Jim Hammitt sends along this interesting report comparing different measures of risk when evaluating public health options:
There is long-standing debate whether to count “lives saved” or “life-years saved” when evaluating policies to reduce mortality risk. Historically, the two approaches have been applied in different domains. Environmental and transportation policies have often been evaluated using lives saved, while life-years saved has been the preferred metric in other areas of public health including medicine, vaccination, and disease screening. . . Describing environmental, health, and safety interventions as “saving lives” or “saving life-years” can be misleading. . . . Reducing the risk of dying now increases the risk of dying later, so these lives are not saved forever but life-years are gained. . . .
We discuss some of these issues in our article on home radon risks. Beyond this, I have two comments on Jim Hammitt’s paper:
1. I wish he’d talked about Qalys. I just like the sound of that word. Qaly, qaly, qaly. (It’s pronounced “Qualy”)
2. He talks briefly about “willingness to pay.” I’ve always thought this can be a misleading concept. Sometimes it’s really “ability to pay.” Give someone a lot more money and he or she becomes more able to pay for things, including risk reduction. True, this induces more willingness to pay, but to me the ability is the driving factor. I think the key is what comparison is being made. If you’re considering one person and comparing several risks, then the question is, what are you willing to pay for. But if you are considering several people with different financial situations, then the more relevant question might be, who is able to pay.
In the US, I think that today's credit-happy culture makes your point #2 even more complicated…
You're right that willingness-to-pay depends, in part, on ability to pay. That's why it's important to evaluate WTP for small risk reductions (or small amounts of whatever item your wish to value), which most everyone can afford. Thus, it's reasonable to ask someone for his WTP for a risk reduction of 1 in 100,000 (most people will answer around $60, which is pretty affordable), but not reasonable to ask him for his WTP for a risk reduction of 1 in 1,000, since the (presumably high) answer will highly depend upon income.
Another point: Environmental economists prefer WTP measures over life-years saved or quality-adjusted life-years saved for a simple reason: WTP reflects people's actual feelings, whereas life-years and QALYs simply do not. Life-years saved, quality-adjusted or not, have no basis in economic reality. They substitute expert opinion for people's true feelings. For example, life-years saved undervalue measures to protect old people: they have fewer life-years remaining. Makes sense, right? But if you ask old people directly, they do not value risk reductions much less than everyone else. In other words, their WTP for risk reductions is still quite high. If you use life-years saved or QALYs to value a policy meant to help the old folks, you're replacing their opinions about what should matter with yours. But you know better now, don't you? So simmer down, Gramps. Who cares what you really think…
Environmental,
It sounds a bit circular to say that willingness to pay "reflects people's actual feelings," considering that you can get different answers depending on how you ask the question.
I agree with you, though, that getting such survey responses–even if they're imperfect–is hugely important in understanding how people think about risks. As a statistician, I think it's funny that responses to the 1 in 100,000 risk question are not sensitive to the respondent's wealth, since some people can afford the $60 more than others can (especially considering that it's not just $60, since there are lots of 1 in 100,000 risks out there). I'm not saying people are "wrong" here, just that my own failure of intuition here illustrates why such surveys are useful.
But I think you're too glib on your second point. For one thing, it we want to make calculations comparing lots of different risks, it can help to use an objective measure such as lives or live-years, or a somewhat objective measure such as Qalys. You can also do the surveys or whatever, but I'd still like to see the hard numbers. Beyond this: yes, just about everyone cares about their own pain and suffering (as in the famous Mel Brooks quote). But I don't quite see your point about Gramps. Nobody's telling him how he can or can't spend his own money; the issue is presumably how a hospital or government or other organization wants to spend its funds. In that case, Gramps's opinions have to be weighted with Dad's and Junior's and so forth. I don't see willingness-to-pay measures as a magic bullet here for solving distributional questions.
Andrew,
I was with you until the last sentence of the post: "… the more relevant question might be, who is able to pay." I thought the point was that this precisely isn't the relevant question; it's just that it's the one we inadvertently get answers to when we focus on willingness to pay in an interpersonal context?
Environmental,
There's a huge amount of evidence that people's attitudes to risks (and small risks in particular) are often irrational, subject to framing effects etc. Does the literature attempt to address this in any convincing way? Also, you haven't really addressed Andrew's point if the WTP of the old simply represents ATP.
Willingness to pay for risk reductions is usually sensitive to income (even for small risk reductions). However, the efffect is quite small.
And since economics, which is where the WTP-approach to valuing lives comes from, usually considers individual preferences as the relevant critera: of course WTP is more appropriate than QALYs.
Using QALYs as a decision critera implies (among other things): (i) linearly decreasing weight with age (usually linearly), and (ii) less weight to an individual with e.g. diabetes compared to an individual with perfect health. Both (i) and (ii) have no foundation regarding individual behavior and preferences. It's "experts" that have decided that this is correct. Yes, economists have a big problem with this approach.
Mikael,
I don't know why you put "experts" in quotes. If statisticians can be experts in statistics, and economists can be experts in economics, I don't see why Jim Hammitt et al. can't be experts in decision analysis.
Andrew,
Jim Hammitt et al. can of course be experts (well, he/they are) in decision analysis. That was not my point (sorry about that).
However, the economists perspective is that individual preferences are important (i.e. here WTP), which gives us some conclusions about societal risk-management. One could disagree with the framework of economics of course, but that's another issue.
QALY is an approach based on a framework that doesn't seem to fit well with individual preferences, so there must be some other argument for using the QALY-approach. However, I haven't yet understood which argument that is, from reading the literature…
I think that this literature is avoiding the main question here, and is too focused on technical details…what principle should guide societal risk-management?
Andrew—
I couldn't get past your first comment. For some bizarre reason, I was reminded of the poems written entirely from anagrams of the word "mathematics." What better way to popularize "quality adjusted life years"?
La fussy jade laity requited,
Edify equity, readjusts…la-la.
Eyeful stye, radial? Just quaid.
Rad, sad eel, justify equality.
Well, it ain't exactly poetry, but I'm not exactly a poet, either.
Mikael,
Life-years saved has benefits compared to lives saved, in that, as Hammitt puts it, "lives saved" isn't really clearly defined, given that it's really "death deferred." I'd rather defer my death by 10 years than by 5 years, for example. Similarly, I'd rather live for 5 years with all four limbs working, than live for 5 years bedridden, hence Qualy and similar measures. Not that Qualy or Daly solve all problems, but there is a motivation for them. It's not a matter of giving less weight to someone with diabetes; from a decision-analytic framework, it's saying that people would rather not have diabetes if they could avoid it.
"It's not a matter of giving less weight to someone with diabetes; from a decision-analytic framework, it's saying that people would rather not have diabetes if they could avoid it."
– As I understand it, it is…let's say we compare intervention A and intervention B. Both will save 5 life years and both will have an incremental cost of 5 million dollars. Hence, the cost-effectiveness ratio is 1 million per life year for both interventions.
Now, let's introduce QALYs. Intervention A is specifically targeted at a population with a QALY-weight of 0.5 (they have some illness), while intervention B is targeted towards a population with perfect health (QALY-weight 1.0). Now, the cost-effectiveness ratio is 2 million per QALY for intervention A, and 1 million per QALY for intervention B.
Comparing these two in a "league-table", intervention B is the preferred choice. This violates individual preferences and behavior, i.e. there is no evidence (as far as I know) that an individual with QALY-weight 0.5 values a prolonging of their life less. Of course I prefer a QALY-weight of 1, but if I'm ill, this is irrelevant, sine this alternative is not in my choice set.
Perhaps my example is not a big problem from a pragmatic perspective, but I consider it a perverse effect from the use of QALYs. When moving from QALY-weight 0.5 to 1 is in my choice set, as may be the case when evaluating pharmaceuticals or medical technology, then I'm fine with it. But otherwise, I think that my example shows a strange effect, no?
'It sounds a bit circular to say that willingness to pay "reflects people's actual feelings," considering that you can get different answers depending on how you ask the question.'
Suppose the Federal Government mandates a safety feature on all new automobiles, and that it will increase the cost of production by $1,000. One might assume that the $1,000 will be fully passed on to the consumers by increasing the cost of new automobiles by that amount. One would be wrong: the increase in price will be something less than $1,000, the rest of the cost being carried by the manufacturers. In one case involving mobile homes, for which I do not have any sort of citation whatsoever, manufacturers wanted to increase the price by $1,000, but in the end the increase came out to be about $400.
The proportion of the cost borne by the consumer is determined by the responsiveness of quantities demanded and supplied with respect to the price of the good, and because consumers respond to prices, it should be rare to see a production-cost increase fully passed on to them.
In deciding whether to mandate the new safety feature, we may expect, say, 100,000 life-years per year to be saved at a total cost of $1,000 × # of new cars per year. Since I don't know how this is done, I'll assume that we take the typical wrongful-death judgment of, say, $1 million, divide it by a life-span of 75 years, multiply by life-years saved, and get a benefit of about $1.3 billion.
To make a decision, can we really compare that benefit to the cost, $1,000 × # of new cars? After all, the consumers are only willing to pay, say, $400 for the extra safety: the mandate is over-providing safety, and (in this example) manufacturers are paying the lion's share of the cost.
Yeah, willingness-to-pay offends the conscience because not everyone has equal ability to pay. But it also reflects the fact that people do not have lexicographic preferences for safety; rather, safety is but one element in weighing their choices.
Those are my thoughts, and they do not reflect the thoughts of any previous poster, this blog, or the Internets — a series of tubes.
I know everybody in this discussion already knows what I'm going to say, but I still thought it would be helpful to make it explicit: There is not an all-or-nothing choice between making a decision based on "lives saved" or on "QALY"s or on any other single thing. We might create a metric based on alpha*(lives saved)+beta*(QALYs), for example.
On a separate subject… Some of the commenters here have the same conceptual problem that "environmental economist's" do: you suggest looking at the attitudes of the person whose life is being saved when deciding whether to use QALYs versus lives-saved, but not the attitudes of the people paying the money. Yeah, a 90-year-old man might value the lives of 90-year-old men just as much as 7-year-old boys…but _I_ don't, and if I'm the one paying, I'm going to value the latter higher than the former.
Different cultures have different attitudes about valuing life. We're told that Eskimo elders (or perhaps Inuit, if Eskimo is no longer an acceptable term) used to walk out on the ice and let themselves die of exposure rather than burden their families, an example perhaps of decision-making based on QALYs or something like it. In contrast, currently in the U.S. most medical decisions are based on something closer to "lives saved" — although not quite: I think terminal cancer patients are removed from lists to receive donated organs, for example.
Perhaps one of the reasons that environmental and transport policies worry about lives saved is that the distribution of ages of people who die due to environmental or transport causes is more representative of the general population than, eg, vaccination (these days, at one end of the age spectrum) or end of life interventions (at the other). Therefore, lives saved is a good enough approximation to life years saved in one, but not the other?
As an economist, got to point out that the validity of the QALY as a basis for making policy decisions probably depends on whether the perspective taken is ex ante or ex post (preferences over years of low quality life do appear to change once you have some experience with low quality life), and are more likely to be useful when comparing otherwise fairly similar interventions. Willingness to pay becomes a really big deal, eg, you're looking at whether to implement the same policy intervention across different countries.