An unconvincing analysis claiming to debunk the health benefits of moderate drinking

Posted on by

Daniel Lakeland writes:

This study on alcohol consumption (by Craig Knott, Ngaire Coombs, Emmanuel Stamatakis, and Jane Biddulph) was written up in the BMJ editorials as “Alcohol’s Evaporating health benefits.”

They conveniently show their data in a table, so that they can avoid graphing a “J” shape that they constantly allude to being wrong… But their own models (see the links under table 3 and table 4) show that the hazard ratio relative to the “never drinker” category for males 50-64 years old declines and then goes up for the “heavy drinker” category, like… I dunno, kind of a J shape??? yes.. yes definitely like that.

Ok, so how about table 4 for women… aged 50-64 years:

Ok, decline with minimum at around 10-15 units /wk… increasing for the heavier drinkers….

Ok, model 2, model 2 is going to do it right??? well… kind of constant, but a definite low point at 10-15 units/wk….

Ok, looking across all the different models in all the different tables…. all the different age groups… yep… pretty much every group has lowest risk in the same range, around 10-15 units per week, or 1-3 units per day or whatever way you want to look at it… pretty much just exactly where the traditional J shape puts things… one or two drinks per day 3 to 5 days a week or something like that.

Nevertheless… the editorial claims: “if there is any beneficial dose-response relation, it is limited to women aged 65 or more — and even that association is at best modest and likely to be explained by selection bias.”

and “for a range of reasons, including confounding and selection bias in the papers generally cited, even low level alcohol consumption is unlikely to protect drinkers from cardiovascular disease”…

so… do a study… don’t like the results? Bury them in a table and just claim you found the opposite of what you found? Or if you don’t claim it yourself, maybe at least get your friend to write an editorial or something.

The editorial that Lakeland cites has an explicit political agenda. But setting this aside, I think the larger point is that the effects of any drug will depend on its context. My guess is that the authors of the research paper and the editorial aren’t concerned about moderate drinking. It’s more that they’re worried that the news about the health benefits of moderate drinking will be used as an encouragement for people to drink heavily.

This contextual effect can arise at the individual or societal level. A person might hear about alcohol being good for you and then lapse into alcoholism—at least, the’s the concern. Or, at the national level, the news about the benefits of moderate drinking will get in the way of public healths efforts to combat problem drinking.

I can share my own experiences here. A few years ago I was talking with my cardiologist and he asked me about my alcohol consumption. I said I drank rarely, probably less than one glass of wine a week. He said I should drink a few times a week, that it would be good for me. Then when I was in France getting a health checkup for my employment, the doctor asked me how often I drank alcohol. I said I had a glass or two of wine a few times a week, cos it was recommended by my cardiologist. She told me not to drink so much, it was bad for my liver. I asked her about the benefits to my heart and she said, no, don’t believe that.

This is just n=2, of course, but perhaps it makes sense that my cardiologist’s recommendation made sense given that I rarely drank, and the French doctor’s recommendation made sense given that, for all she knew, maybe I was an alcoholic and was just trying to justify my addiction.

In any case, I agree with Lakeland that it’s better to report results clearly and graphically rather than contorting the data to support some particular claim.

43 thoughts on “An unconvincing analysis claiming to debunk the health benefits of moderate drinking”

  1. >”I think the larger point is that the effects of any drug will depend on its context.”

    Yep, in some contexts drinking only beer is the best thing you can do for your health:

    “On 31 August 1854, after several other outbreaks had occurred elsewhere in the city, a major outbreak of cholera reached Soho. John Snow, the physician who eventually linked the outbreak to contaminated water, later called it “the most terrible outbreak of cholera which ever occurred in this kingdom.”[1]

    Over the next three days, 127 people on or near Broad Street died. In the next week, three quarters of the residents had fled the area. By 10 September, 500 people had died and the mortality rate was 12.8 percent in some parts of the city. By the end of the outbreak, 616 people had died.

    […]

    There was one significant anomaly – none of the monks in the adjacent monastery contracted cholera. Investigation showed that this was not an anomaly, but further evidence, for they drank only beer, which they brewed themselves. Residents near or in the brewery on Broad Street were also not affected as a result of the fermentation of the contaminated water. The beer was safer to drink than the dirty water from the Broad Street Pump.”
    https://en.wikipedia.org/wiki/1854_Broad_Street_cholera_outbreak

    Reply ↓

  2. The key message I take from Andrew’s story is how little of clinical practice is evidence based.

    In various other contexts I’ve seen mutiple doctors give highly contradictory opinions.

    I’m not sure how much of this variation reflects the true uncertainity in our collective knowledge versus how much is pure ignorance on the part of individual doctors.

    Actually, “ignorance” is too harsh a term. Should I even be expecting doctors to have these complex, noisy conclusions of literally thousands of studies on their fingertips? What we really are lacking is good-qualtity, up-to-date meta-analyses and a user-friendly way for clinicians to use them.

    Reply ↓

    • There are actually entire industries built on giving doctors up to date information, one of the companies is in fact called “Up To Date”. Most university hospitals have subscriptions to at least one of these. My experience is that most doctors don’t use them often enough.

      Reply ↓

        • That second bit of advice in France strikes me as… poorly thought through?

          Conditioned on you being a male who has a healthy liver (i.e. no known jaundice, hepatitis, etc.), the mortality risks due to liver ‘problems’ for non-alcoholics, aren’t even on my radar.

          The risks for men related to heart attacks (to oversimplify a bit) are most always in the top 3 mortality risks in developed countries as I recall.

          Then again, it may be a bit simpler, and the French Doctor just read some nonsense in the press about 3 drinks a day causing liver cancer. ( http://understandinguncertainty.org/sensationalist-promotion-world-cancer-research-fund )

        • Too much to paste here, but here’s a snippet from their summary on “Cardiovascular benefits and risks of moderate alcohol consumption”:

          “The cardiovascular benefit of moderate alcohol intake must be balanced against the multiple deleterious effects of alcohol. The net risk-benefit balance associated with moderate alcohol consumption is likely to differ in various age groups and populations. The American Heart Association recommends against advising people who don’t currently drink to initiate light alcohol use. We suggest that carefully selected non-drinking patients with a high risk for CVD be advised that light drinking (no more than one to two drinks per day in men, and one drink daily in women) may decrease their cardiovascular risk (Grade 2B). Selected patients might include those with an established relationship with a clinician, no personal or family history of substance abuse, no history of depression or bipolar disorder, and nonsmokers. (See ‘Advice to patients regarding alcohol use’ above.)”

  3. I imagine many internists hearing “a glass or two of wine a few times a week” tend to mentally double or triple the amount.

    Given the fact that the health benefits of moderate drinking are moderate and the negative health effects of heavy drinking are severe, this seems like appropriately conservative medical practice.

    Reply ↓

  4. “In unadjusted models, protective effects were identified across a broad range of alcohol usage in all age-sex groups. These effects were attenuated across most use categories on adjustment for a range of personal, socioeconomic, and lifestyle factors. After the exclusion of former drinkers, these effects were further attenuated.”

    I just had an idea, tell me if there is a name for this or why it wouldn’t work. I think these authors who want to conclude “no effect” should use their model with all the same adjustments to assess a behaviour they do think causes lower mortality. Authors who claim an effect should use the same model to check something they do not think has an effect. That may be a good approach to model checking.

    Reply ↓

      • I didn’t think of it that way, but you’re right. This way of analyzing observational data is like running experiments missing positive and negative controls. If no one takes the latter seriously, why is it so common for the former? In this case, they could have used the same data to report “Obesity’s Thinning Health Risks” by adjusting for alcohol consumption.

        Reply ↓

  5. Andrew, I was with my father-in-law when he received 2 letters from his medical specialists in the mail. One said (paraphrasing here) “Eat a banana every day” while the other said “Never eat a banana.” I sprang into action to research the evidence and declared that he should confront the doctors with their contradictory advice. He called me off my mission, declaring the solution was simple: “I’ll eat a half a banana every day.” And so he did.

    Reply ↓

  6. I remember that study, and I don’t think this post really hits the nail on the head. What appears to have happened is this: (i) The authors found the same result as previous studies; (ii) they didn’t like it; (iii) they divided the sample into groups until the effect was not significant anymore; (iv) they interpreted the results as though insignificant effect = zero.

    See here: http://understandinguncertainty.org/misleading-conclusions-alcohol-protection-study

    Reply ↓

    • Lemmus:

      I think Lakeland was pretty much making the point that you say in your comment: the results of the new analysis seem consistent with health benefits for moderate drinking, but, as Lakeland put it, the researchers “don’t like the results” so they “bury them.”

      Reply ↓

      • I do think selection bias and confounding are the _real_ or _important_ uncertainties here.

        Am I sure about the direction – not entirely.

        Have I worked out the informative priors for selection and confounding (no and I don’t have the expertise myself anyway).

        Sander Greenland argued once that in the face of such real uncertainties – giving advice that it is good for you is unethical.

        But so is encouraging mistaken impressions about what is being learned from the study (insignificant effect = zero.)

        Reply ↓

      • Yes and no. A table’s fine, as far as I’m concerned – indeed, a look at the table immediately shows that the effect sizes are really large. The sin’s to divide the sample until nothing’s significant anymore and then pass the large insignificant results off as quasi-zero.

        Reply ↓

  7. One of the things we should remember here is that many different studies, including this one, have found a J shape with declining mortality at around 10 standard servings per week for those who never have more than 2 or 3 servings in a given day… And I believe this is a BIG population. It’s probably more than 20% of the total population. So telling these people that any amount of drinking is bad for you and they should cut back their drinking is probably doing harm to these people’s longevity, AND reducing their everyday welfare as well (assuming these people are consuming these beverages because they enjoy them).

    The population of severe drinkers is very small, and is also unlikely to be influenced by these kinds of prescriptions. No-one thinks heavy drinking is good for you.

    So I think the actual net result is to wind up doing harm to a large population, and not helping the target population at all… bad all around.

    Reply ↓

  8. Unfortunately much of public health research is driven by advocacy rather than data.

    Strong advocacy priors + researcher degrees of freedom = foregone conclusions.

    Arguably, never have so many people with such good intentions done so much harm to scientific progress.

    Reply ↓

  11. The criticism of the specific paper seems valid, but the evidence for the health benefits of moderate drinking are based purely on observational studies and there is a clear danger of selection effects. The main evidence used to argue against the J-curve claim does not come from yet more observational studies of drinking levels and health – a brief and clear blog-summary of the arguments in use can be read at http://davidroodman.com/blog/2015/05/01/are-the-benefits-of-moderate-drinking-a-myth/

    In brief: There is evidence of clear selection mechanisms that would produce a J-curve, and Mendelian randomization studies (that use a gene affecting how alcohol is metabolized as an instrumental variable affecting drinking levels) fail to find any positive marginal health benefits of alcohol drinking at any drinking level.

    Not saying the matter is settled – just saying the best arguments against the J-curve seem stronger than the paper discussed here ;P

    Reply ↓

  12. Hi Ole,

    I do not understand the logic of the alcohol metabolization gene being used as an indicator of drinking. Is it that people with this gene do not drink? Drink less? Get drunk faster? Also observational studies of drinking can in fact be quite good information when responses are appropriately adjusted for known biases using empirical distributional differences between reported and actual. They do need to be adjusted however in order to be representative of the behavioral distribution and not simply the reported.

    Best,

    James

    Reply ↓

      • The thing that is especially suspicious is that the J shape is pretty damn robust through time. Even these guys who obviously set out to debunk it wound up finding a J shape and then basically denying it in what I think is a dishonest and politically motivated way.

        I’m perfectly fine with the idea that alcohol consumption benefits and risks vary from person to person, and I can certainly believe that this is partially genetic. But it seems that in the current population if you’re choosing to drink about 10 servings a week not in the form of binges (ie never more than 3 servings in 24 hours) you are generally going to be overall healthier than other groups. Whether the alcohol is the causal factor, or whether the alcohol related genes are the causal factor… or some other thing, like maybe you have genes that give you better self-control which applies across multiple aspects of your health… I don’t know. But this study was particularly bad and got a lot of press.

        Reply ↓

    • Hi James,

      sorry for the late answer – but I’m in a different time zone ;)

      This is not a field I’ve worked in directly, but with that caveat I’ll try and explain their arguments as best I can.

      if I understand the assumptions in these Mendelian randomization studies correctly there are two genes that are used. One affects how fast alcohol is metabolized (and thus removed) from the body. This means that the alcohol from any consumption level would “hang around” in the body of a slow metabolizer for a longer time. For a given consumption level (units per week), these people would have a larger “effective dose”. Alcohol intake is not affected, so a comparison of health outcomes for slow and fast metabolizers (given identical drinking levels) is interpreted as the marginal effect of alcohol.

      The second gene that is used in some of these studies is associated with facial flushing and hangover severity, which affects drinking levels but not how long the alcohol persists in the body. Let’s call these non-tolerators (since they experience more negative effects that cause them to drink less). If tolerators and non-tolerators are randomly distributed within a population, you can compare health outcomes between the two, which is interpreted as the marginal effect of a slightly different alcohol intake.

      Also – I don’t think the point is to ignore or dismiss all observational studies. If you look at the Fekjaer piece from Addiction (linked at the bottom of the post I linked to), his scepticism is largely based on what he sees as curious patterns in these studies: A protective role of alcohol has been found for a large number of conditions, from asthma, cancer and gallstones to hearing loss, rheumatoid arthritis and osteoporosis. He notes that 12 of the 16 conditions classed as “lifestyle diseases” on Wikipedia are among those for which a J-curve for alcohol has been found, and claims that there is no known “biological mechanism explaining a causal preventive role for alcohol” for most of these conditions. Fekjaer also claims that the J-curve found for all these conditions seem to have a similar optimal level, around the “level that is most accepted in our present-day societies, more socially accepted than both abstention and higher consumption,” and that there are a number of observational studies indicating that such norm-conforming consumption is correlated with a variety of health-promoting behaviours.

      Reply ↓

      • Ole: Again thanks.

        I think though we need to keep in mind that many (most?) statisticians have yet to grasp the real difficulties involved in discerning causality from observational studies, getting past the naive idea that responses can be _adeqautely_ adjusted for appropriate covariates (confounders). Hence, confidence interval coverage, type one and two errors, posterior probabilities of causal effects and type M and S errors are almost always undefined – yet interpreted as if helpful.

        This papers essentially argues that even most experts should not be trusted http://sekhon.berkeley.edu/papers/opiates.pdf and current work by David Madigan tries to make the challenges clear and suggests possible ways forward e.g. -http://www.dcscience.net/Madigan-stats-obs-studies-annurev-statistics.pdf

        Now my likely most important contribution to medicine came from an observational study – so I do not mean to suggest they should ignored! http://www.ncbi.nlm.nih.gov/pubmed/10825042

        Reply ↓

        • If the genetic differences were correlated with other behaviours or traits that influenced outcomes that would make the method inappropriate and yield misleading results.

      • As someone who knows a little about biology and a fair amount about system dynamics (ODEs etc), it seems like a poor model to say that a genetic variation that reduces metabolism rate is equivalent to different dosing. People who experience *longer* intoxication from the same dose are likely also to reduce their rate of consumption in response. The assumption that “alcohol intake is not affected” is probably based on say total weekly dose which is the usual reported stats, and does not take into account several things:

        1) Reporting error (there is basically a societal norm towards which people shrink their reported rates both upwards and downwards)

        2) Differences in small time scale (ie. hourly) patterns of consumption. I can well imagine that someone with a higher metabolism rate might be the guy who walks into a bar and orders a shot of whisky and a beer chaser, whereas the slow metabolizer orders a glass of wine and another 2 hours later.

        As I understand it, the clearly predictive thing for problematic drinking is peak blood alcohol concentration (BAC) but it is almost never measured. However people who dose up and burn off quickly will have higher peak concentrations than people who slowly dose and burn slowly. So the assumption that the “high metabolizer” has a lower “dose” may well be entirely BACKWARDS from reality, when you measure the important dose statistic, PEAK BAC.

        It’s a big big difference if you drink 10 servings a week in the form of a beer with lunch and a glass of wine with dinner, or a glass of wine with dinner and a cocktail with dessert, 5 days a week, or if you consume 10 servings a week in the form of 5 servings in a binge friday night, and 5 servings in a binge saturday night. However, the first group may well have larger area under the curve (AUC) for weekly dosing, whereas the second are going to have much higher peak BAC.

        Also, the assumption of random distribution of toleration and non-toleration is just clearly false. non-toleration is often associated with Asian racial background, which is also associated with all kinds of other things, such as education levels, income, social norms for behavior, social norms for alcohol consumption.. not even close to random. Even within say a single relatively homogenous background, like Japanese people or something, you will find a mix of toleraters and non-toleraters, and my general impression is that toleraters generally put social pressure on non-toleraters to consume more than they otherwise would in certain contexts (ie. parties and work related celebrations etc). In other contexts the non-toleraters will behave differently. So, social context is going to be very important.

        All of this is to say that we have a complex issue which goes all the way from basic genetics through individual behavior to social behavior, and confounding with other health issues… Going 1 step down that road doesn’t really mean you’ve done a significantly better job. Particularly if your assumptions sound plausible, but are not only wrong but potentially backwards from reality.

        Ultimately, I agree with your prior on the idea that “norm-conforming consumption is correlated with a variety of health-promoting behaviors” plus possibly, or possibly not, some potential alcohol related biochemistry that helps with cardiovascular disease. Reducing cardiovascular disease can itself help with kidneys (high blood pressure is caused by kidney disease, but also high blood pressure causes kidney damage, in a feedback loop). Many of those diseases you mention can be inflammatory related (asthma, arthritis, hearing loss, etc), and it’s entirely plausible that alcohol consumption changes hormone levels, and changes inflammatory processes. It’s also entirely plausible that pain from arthrithis or feeling of isolation from hearing loss, could drive increased alcohol consumption. To think that there’s some kind of simple directed acyclic time-invariant causal network that could explain this, rather than a complex time-series feedback trajectory for each person is just … too simplistic. It ignores too many sources of uncertainty, largely at a model-selection level. By selecting a single model, and working within just that model, you’ve ignored the very plausible possibility that there’s some entirely different model that explains things correctly.

        The thing that seems pretty clear though, is that discouraging people from consuming in a healthy manner (ie. the 1-2 servings per day 5 days a week or so) has NO scientific evidence for health benefit, and clear evidence for harm in the form of reducing an activity that people enjoy based on a “prohibition” mentality that demonizes alcohol in any form.

        From this study it seems pretty clear that the goal was to go out and find a way to discourage alcohol use. It’s like a racist group going out and finding a way to discourage playing basketball because they dislike the fact that it’s enjoyed mainly by the race they have animosity towards. Never mind that moderate basketball “use” has a plausible positive effect on overall health, they focus on the idea that “overuse” tends to cause severe injuries (ACL tears, broken ankles, whatever) or is associated with other dangerous behavior (gang related activities? crime, whatnot) but the real goal is to just demonize something they know an out-group enjoys.

        Reply ↓

        • Well – as I started out saying: the criticism of this individual study seemed appropriate. It seems motivated by a desire to reach a certain conclusion.

          The appropriateness of the assumptions for a valid IV in the case of Mendelian randomization (in this specific instance or more generally) is not something I am in a position to judge properly. I’ve just seen the method pop up once in a while (there was a paper on alcohol use during pregnancy and child outcomes in the Economic Journal, for instance, and a paper on alcohol and heart disease in the BMJ) . The point I wanted to make is that the main criticism of the J-curve in use today comes from something else than (yet another) observational study.

          Cheers :)

          Ole

        • Cheers as well. I certainly am glad that people *are* doing things other than observational studies on these issues. I think alcohol use is something where good evidence based guidelines have a lot of potential benefit. It’s my impression that many people think that if they just don’t drink too much total alcohol in a week, then their drinking is “under control” or “healthy” and as I said, there’s a big difference between a twice a week 5 serving binge vs a 5 days a week 2 serving routine, yet both are within a simplistic <= 10 servings a week prescription. More nuanced and better understandings could benefit a lot of people. That was my impression of the Canadian guidelines linked above.

Leave a Reply

Your email address will not be published. Required fields are marked *