Still more Mr. P in public health

When it rains it pours . . .

John Transue writes:

I saw a post on Andrew Sullivan’s blog today about life expectancy in different US counties. With a bunch of the worst counties being in Mississippi, I thought that it might be another case of analysts getting extreme values from small counties.

However, the paper (see here) includes a pretty interesting methods section. This is from page 5, “Specifically, we used a mixed-effects Poisson regression with time, geospatial, and covariate components. Poisson regression fits count outcome variables, e.g., death counts, and is preferable to a logistic model because the latter is biased when an outcome is rare (occurring in less than 1% of observations).”

They have downloadable data. I believe that the data are predicted values from the model. A web appendix also gives 90% CIs for their estimates.

Do you think they solved the small county problem and that the worst counties really are where their spreadsheet suggests?

My reply:

I don’t have a chance to look in detail but it sounds like they’re on the right track. I like that they cross-validated; that’s what we did to check we were ok with our county-level radon estimates.

Regarding your question about the small county problem: no matter what you do, all maps of parameter estimates are misleading. Even the best point estimates can’t capture uncertainty. As noted above, cross-validation (at the level of the county, not of the individual observation) is a good way to keep checking.

5 thoughts on “Still more Mr. P in public health

  1. Somewhat tangentially, Sullivan's article is another nice example of "story time". It's an extremely common pattern in public debates about healthcare. The commenter starts with making some good observations about life expectancy in the US, and then somehow pivots to absurd conclusions about the state of healthcare.

    Some questions that never get addressed:

    – If it is the private sector that is doing so horribly, how come is it that blacks' life expectancy is the lowest of all races, even though blacks are the most likely of all major races to be covered by Medicaid, and they are significantly less likely than Hispanics to be uninsured?

    – Is there any direct evidence that American medicine results in worse outcomes (life expectancy wise) than European medicine? Not at all – in fact, there is evidence to the contrary: for example, five-year survival for breast cancer in the US is highest among all major developed countries (Iceland excluded):

    http://info.cancerresearchuk.org/cancerstats/type

    – And why exactly is there such a difference in life expectancies? Is it possible that it has nothing at all with the actual state of healthcare, but instead it is largely explained by a confluence of lifestyle factors, namely, obesity, smoking, traffic accidents, and the risk of violent death due to homicide?

    As far as I know, the only real healthcare-related factor that might contribute significantly to lower life expectancy in the US is infant mortality. The difference between infant mortality in the US and, say, France accounts for 0.25 years of the gap. And even there it's not certain to me that the difference is caused by bad healthcare rather than lifestyle factors.

  2. BTW, I quickly ran numbers comparing some mortality factors in the US and France. Here's what I get.

    According to Wikipedia, we need to explain the gap of 2.4 years (80.7 in France vs. 78.3 in the US).

    Of these:
    – 0.25 years are due to infant mortality
    – 0.28 years are due to the difference in mortality due to lung cancer among smokers (assuming 15 years lost per death). That's not including all other smoking-related causes of death.
    – 0.19 years are due to the difference in traffic fatality rates
    – 0.13 years are due to the difference in homicide rates.

    That adds up to 0.85 years, or about one third of the total effect.

    But the biggest effect is still left unaccounted for: obesity. Here quantifying the effect is more controversial, but just the difference in class III obesity rates (BMI > 40, affecting just under 6% of American adult population) should explain another 0.6 years or so.

    There's very little room left for the differences in quality of healthcare.

  3. Nameless,

    I'l disagree – based on some proprietary modeling I have seen, I'd guess that obesity may account for multiple years. Diet and stress account for even more – they take a tremendous toll, and Americans do a horrific job. The actual effect of health care may be the opposite of what you are looking for – America's health care system may confer multiple years of extra life, but it gets masked by health choices.

  4. I’m an avid reader of your blog, and while I am not a statistician, I do work with mathematical modeling for tail events (terrorism specifically,) and spend a lot of time thinking about uncertainty.

    I had an idea reading your paper: it should be possible to use an alpha channel to display uncertainty. Basically, on a red-green scale of, for instance, mortality, you would include a “shading” effect, so that high uncertainty would be more transparent than low uncertainty. This should lead to a display that more correctly plots the values desired. For instance, parts of the map with high mortality, and high sampling variation with low confidence intervals would be mostly light pink, showing that the confidence is weak but the mortality is high. Places where the data is better would be less transparent, and therefore darker in color. I have not done this, since I lack the software and data, but it seems that it should work.

    This obviously will not solve any problems with the models creation of spatial artifacts, but it will show where the data is meaningful in, well, a more meaningful way.

  5. Gerd Gigerenzer has made some pertinent comments about the value of diagnostics such as "5 year survival rates". The simple point is that if you aggressively screen people at an early age, you'll find lots of benign and early-stage cancers that would not kill anyone in 5 years (if ever), and also give lots of invasive treatment with unwelcome side effects, without this necessarily (or even likely) leading to improved health outcomes.

    The issue for me is not whether US healthcare results in worse outcomes (probably not a huge difference on average) but that twice the spend (compared to the rest of the West) doesn't seem to result in a significantly better outcome.

Comments are closed.