Larry Gonick asks what I think of this research article, Association between low density lipoprotein cholesterol and all-cause mortality: results from the NHANES 1999–2014.
The topic is relevant to me, as I’ve had cholesterol issues. And here’s a stunning bit from the abstract:
We used the 1999–2014 National Health and Nutrition Examination Survey (NHANES) data with 19,034 people to assess the association between LDL-C level and all-cause mortality. . . . In the age-adjusted model (model 1), it was found that the lowest LDL-C group had a higher risk of all-cause mortality (HR 1.7 [1.4–2.1]) than LDL-C 100–129 mg/dL as a reference group. The crude-adjusted model (model 2) suggests that people with the lowest level of LDL-C had 1.6 (95% CI [1.3–1.9]) times the odds compared with the reference group, after adjusting for age, sex, race, marital status, education level, smoking status, body mass index (BMI). In the fully-adjusted model (model 3), people with the lowest level of LDL-C had 1.4 (95% CI [1.1–1.7]) times the odds compared with the reference group, after additionally adjusting for hypertension, diabetes, cardiovascular disease, cancer based on model 2. . . . In conclusion, we found that low level of LDL-C is associated with higher risk of all-cause mortality.
The above quotation is exact except that I rounded all numbers to one decimal place. The original version presented them to three decimals (“1.708,” etc.) and that made me cry.
In any case, the finding surprised me. I don’t know that it’s actually a medical surprise; I just had the general impression that cholesterol is a bad thing to have. Also, I was gonna say I was surprised that the estimated effects were so large, but then I saw the large widths of the confidence intervals, and that surprised me too at first, but then I realized that not so many people in the longitudinal study would have died during the period, so the effective sample size isn’t quite as large as it might seem at first.
The researchers also fit some curves:
Next, the inferences that the curve came from:
The data are consistent with high risks at low cholesterol levels and nothing happening at high levels, also consistent with other patterns, as can be seen from the uncertainty lines.
The published paper does a good job of presenting data and conclusions clearly without any overclaiming that I can see.
Anyway, I don’t really know what to make of this study, and I know nothing about the literature in the area. I’ll still go by my usual algorithm and just trust my doctor on everything.
I’m posting because (a) I just think it’s cool that the author of the Cartoon Guide to Statistics reads our blog, and (b) it can be helpful to our readers to see an example of my ignorance.
A lot of arbitrary cutoffs at which medical interventions (medication, modification, etc.) are supposed to start, sit at the bottom of a mortality U-curve. So, the optimal conditions are considered abnormal. BMI of 25 is when attention is given to a patient, but the same BMI is associated with the lowest overall mortality.
However, I find this study well-balanced and they mentioned major issues in the discussion.
“…Finally, we can’t deal with the problem of causality, because the design of research is observational…” – Good job at pointing out not only the limits of observational studies, but mentioning that causality shouldn’t be implied.
“…we cannot rule out that the results may be affected by the start or stop of lipid-lowering therapy during the follow-up period and did not observe the dynamic changes with time…” – I thought this one would be easy to provide data for.
The biggest issue I have with the study is the assumption that increase in LDL, which is a proxy for a variety of things, is somehow assumed to be a one-to-one relationship with all cause mortality. It is same with any other blood test used to start some intervention. A1C gives you an average, but not variability. Not to mention it is all based on a very small volume of blood only and the distribution throughout the body is unknown.
I don’t envy the doctors, though. They ultimately have to do or not do something, despite all the unknowns and uncertainty.
Doc Here: all kinds of severe disease, including cancer can reduce you cholesterol. The effect seen in the study is most likely reverse causation. People who have very low LDL due to genetic mutations have lower mortality, see e.g.
https://jamanetwork.com/journals/jamacardiology/article-abstract/2801002?utm_campaign=articlePDF&utm_medium=articlePDFlink&utm_source=articlePDF&utm_content=jamacardio.2022.5271.
100% agree with this suspected interpretation. Furthermore intersect few people dying with the fact that many who did die probably had other diseases (as suggested by Anonymous Doc) and it’s a recipe for misinterpretation.
Seems like a “nothing to see here” finding.
On the other hand, the fact that off-the-charts high levels of LDL (so-called “bad” cholesteral) seems to be no problem is quite interesting. I feel like they buried the headline with this one.
I sometimes wonder whether looking at an individual study without reference to the full literature adds any value. But then again, I sometimes wonder whether meta-surveys have value. Sometimes I think the older I get, the more confused I get. Sure enough, I Googled and this was the first hit:
Evidence from clinical trials has demonstrated that benefits of LDL‐C lowering therapy on all‐cause and CVD mortality are observed in individuals with baseline LDL‐C≥100 mg/dL, but not in individuals with LDL‐C<100 mg/dL over treatment periods of up to 7 years.
https://www.ahajournals.org/doi/10.1161/JAHA.121.023690#:~:text=Evidence%20from%20clinical%20trials%20has,of%20up%20to%207%20years.
Sometimes I think that medical analysis should necessarily be based on clinical trials, but sometimes I think that clinical trials almost invariably fail to control for relevant confounding variables.
I think we’ve known all of this for a long time: https://www.theatlantic.com/magazine/archive/1989/09/the-cholesterol-myth/669117/
I don’t think much has changed from this 34 year old article except that statins are pretty safe.
(I also agree that the modest low cholesterol mortality increase is almost surely from other severe health issues which coincidentally lower LDL.)
To all the commenters above:
Yeah, I get all your skepticism. Still, my “bad” and total cholesterol numbers were gradually going up year after year, and after a couple years of watchful waiting my doctor scared the hell out of me and I radically changed my diet and also went on a low dose of a statin. My cholesterol numbers improved a lot after that. I have no idea what this implies for my life expectancy but I’m planning to stay the course.
As a total non-expert, ass to me this type of risk calculator can be useful for understanding cholesterol levels in context. One cholesterol number in isolation doesn’t seem to be terribly useful.
https://statindecisionaid.mayoclinic.org/statin/index
This isn’t even the “good” cholesterol, but finding seems to be that high levels are…ok? Wild.
Jonathan says that “we” have known this for at least 34 years, but here is the website for Cheerios showing boxes with a prominent banner advertising their usefulness in lowering cholesterol: https://www.cheerios.com/hearts-matter (I’ve seen the same on boxes in stores, not to mention all of the TV ads).
Adede:
Yeah but it seems there’s not a lot of data at the high end: the highest category in the table is “>= 160,” but according to my health plan, you should be concerned if your level is more than 200. (Mine was 201, then 218, then 225, then dropped to 144 after the diet change and drug.) This study doesn’t seem to say much of anything conclusive about levels above 200.
P.S. I have not added Cheerios to my diet, though. Maybe that’s the next step…
Also, it seems to me that using ‘all cause mortality’ adds noise to the more specific hypothesis (cardiovascular disease) that would be tested with LDL at the high end.
Do they not yet have a theory to test for why low LDL would kill you?
Anything that has a normal value will kill you if you are way off one way or the other. My favorite is salt. Pretty much the only study ever done on salt consumption concluded that too little salt was worse for CVD than too much, but the US recommendations for salt consumption are for ridiculously low levels that are, according to that study, seriously bad for you. (And almost impossible to achieve, fortunately.) The Japanese diet has incredibly high levels of salt. And one of the world’s longest life expectancies. (I asked a cardiologist how old her patients were. “One is 46 or so, all the others are well over 90. And they all die of cancer before heart disease becomes problematic.”)
They still seem to make a big deal about hypertension over there:
https://pubmed.ncbi.nlm.nih.gov/32636526/
I believe 140/90 is a more lax definition than used elsewhere (130/80 mmHg) as well. Of course, hypertension itself is astmptomatic and not a problem. But it is assumed to be upstream of cardiovascular disease.
But I also doubt the low salt diet concept. In general, if your body has a given acute response, then the chronic response will be the exact opposite. This is a general principle for complex systems of feedbacks.
So we would expect chronic high sodium diet to *reduce* baseline blood pressure.
What I’m saying is that ‘all cause mortality’ seems like an imprecise target to use with a fairly precise theory of ‘high LDL leads to cardiovascular disease which can lead to death by heart attack or stroke’.
So perhaps we shouldn’t be surprised that when throwing in all causes, the connection was weak?
The same could be said of the low LDL hypothesis if they’ve formed one yet. They probably don’t think low LDL causes death by sepsis or gunshot wounds.
FWIW, the Japanese seem more concerned with cancer and metabolic syndrome (the latter rightly so, since it hits folks of Asian descent at much lower BMIs than Caucasians). But your GP/PCP will check your BP every time they see you.
(The top three causes of death in Japan (2019) are Cancer, CVD, and dementia (with lower respiratory infections fourth). In the US it’s CVD, cancer, and respiratory disease, (with dementia a distant fourth). So the lower apparent concern (in Japan’s media) for CVD seems justified.
This inversion of the top two causes of death is big: the difference (between first and second) is 20% (very roughly) in both cases.
@Anon:
For sure, the cause of death is interesting. But obviously if people diagnosed with a disease then get injected with poison and die of something else first that could reduce the death rate for that disease. But it doesn’t mean the treatment was beneficial.
@David
WHat you say about Japan makes me question not only the sodium -> hypertension idea but also the hypertension -> cardiovascular idea.
I think the concern about being over 200 refers to total cholesterol, not LDL. Anything over 130 is high for LDL, and that’s for no risk factors. Docs want you under 100 if you have a risk factor, and under 70 if you’ve already had a heart attack, it used to be thought that high LDL could be compensated by high HDL (good cholesterol), but recent research has shown that is not true.
Larry:
Yes, you’re right, I was confused on that.
Moore’s 1989 book (which is excerpted in the lengthy Atlantic article I linked to) discusses the lowering-cholesterol-complex that actively promotes Cheerios, among other things. The article points out that cholesterol scares worked for years to suppress egg consumption, even though a central focus of the article is that cholesterol levels are barely affected by diet at all…. not even Cheerios.
I think Moore’s article is a masterpiece of science reporting (it had a huge impact on me 34 years ago) but of course Moore is a scientist, not a journalist, so maybe the surprising thing is that either (a) Moore can both do science *and* write; or (b) the Atlantic at the time had really good science editors. (I lean towards (b)).
Jonathan said: “(I lean towards (b))”
I dunno. Every writer benefits from a good editor. It’s true that many research papers are poorly written, but for my money most of the good science writing I read is written by people with PhDs in some field of science. An editor can tighten and smooth a scientists’ writing, but they can fix the science in a journalists’ writing.
I’d start with what they actually measured. In particular with adherence to fasting (if they told them to do so, I couldn’t easily find that out) and that big “5” constant that goes with triglyceride levels. Then there is the issue that these are arbitrary coefficients from an equation that implicitly sets the coefficient for obvious stuff like statin use to zero.
Sorry, forgot to add this one:
It may be fair to exclude them, but tell us what the results were for that group.
Young adults, middle-age adults, and older adults die of different things. I would expect the relationship between mortality and levels of LDL-C to be quite different across these age groups, potentially including positive associations in one group and negative in another. They included everyone >18y and adjusted for age, but I think it needs to be stratified by age group.
Their “fully-adjusted model” (Model 3) controls for many conditions that are potential outcomes of LDL-C levels, including and very surprisingly cardiovascular disease, which is certainly a bad control/collider in this context. The same case could be made for other of their covariates and the have no information on statin use which is a huge confounding issue here. I was interested to read that high statistical ability is required to do non-linear Mendelian Randomization, which might be true but isn’t much of an argument in favour of the current approach: “However, in Mendelian random analysis, the modeling of U-shaped association requires high statistical ability”
Tears are running down my cheeks as well regarding the decimal places.
I find it mystifying that a single HR can be used to summarize an adjustment model with so many variables. It would seem like in the chart broken into classes of LDL-C, the estimated HRs assume the average values of continuous variables and some unreported distribution of categorical variables? (This situation is not specific to this paper but to many medical science papers I have read.)
What is the convention of summarizing complex models into a single number? Many of these articles do not actually provide all the coefficients in the models so it’s impossible to tell just by what’s published.
They also missed the opportunity to model interactions – sure the sample size is not high but as you said in the causal quartets paper, would rather accept lower precision than to use the wrong model.
I very much share the confusion produced by these results, but any skeptical non-expert like me ends up thinking two things: epidemiological studies only measure associations and have unknown confounders, and clinical trials provide the most reliable evidence.
Clinical trials show consistent and powerful results of lowering cholesterol, e.g. https://jamanetwork.com/journals/jama/fullarticle/2556125
Speaking of which, don’t hunter gatherers (e.g. the Ache’ and Hadza) have really low LDL (50 to 75 mg/dl) and have perfect heart health until their 70s? But then again, I cited another association which could be driven by cohort selection (only the lucky get to their 70s), so… I end up thinking the clinical trials are sufficient to guide my lifestyle choices right now.
Yea, people die of many things, so all cause mortality is critical to look at but not a super precise endpoint here. Reverse causality and all that. The other issue is that it is LDL particle count rather than cholesterol content per se that matters. So, LDL-C will track LDL-p, but it’s adding another layer of measurement error here.
The Mendelian randomizations, ecological comparisons to HGs with very low burden of CVD, and statin RCTs all suggest an important causal role for LDL particles in CVD. That and it’s an area under curve problem. CVD is slow and progressive, so interventions generally require a long time to add up to significant risk modification.
Looks like only around 1% of patients benefit from statin (and 2% develop diabetes):
https://thennt.com/nnt/statins-for-heart-disease-prevention-without-prior-heart-disease-2/
So maybe that explains about 1% of CVD.
For anyone tuning in, this is dangerous nonsense being peddled. Responding to below first, cholestyramine is not a statin and it’s mechanism of action makes it non responsive to the prevailing consensus view of atherosclerotic disease. The sine qua none is ldl particles trafficking into sub endothelial spaces. Yes it’s a complex process and other factors also important, especially those regulating inflammatory responses in endothelia.
The genetic data alone are overwhelming. Familial hypercholesteremia has ungodly hazard ratios and is characterized by massive ldl particle counts. Conversely, alleles that upregulate ldl receptors in liver confer protection including subgroups who never get atherosclerosis. Everything in between falls out nicely in Mendelian randomizations.
The statin mechanism is that it upregulates ldl receptors, increasing clearance rate. This is very good and protective but obviously the amount of protection depends on context. NNT isn’t some static thing you can look at the way anoneuoid is pretending here.
Make informed decisions with your doctor :)
So if Andrew wants to be informed, he may ask his doctor “What is the chance that taking this statin saves me from dying in the next 5 years? How about ten years?”
What should the doctor’s response be?
Because the data says something like 0%. It has about 1% chance of saving him from a heart attack and about 1% chance of causing some other health issue that kills him instead. That is what informed people would know.
Anon:
I guess it depends on the dose? My doctor said I’m on a low dose, which is kinda reassuring. I’ll see how my cholesterol levels are when they’re tested again this fall.
The outcome surely depends on the dose and the reason why your cholesterol was rising. But the data provided to us is the typical result for many people.
I want to say just come up with an algorithm that has predictive skill and plug in your particular situation, but I’m not convinced we even know what needs to be plugged in and whatever algo will be gamed by researchers trying to publish.
> whatever algo will be gamed by researchers trying to publish.
A one-sized fits all answer, based on mind-reading and bad faith.
That’s quite an algo.
I think the explanation is that they used all cause mortality as the criterion, not cardiac mortality. It may be that reduced LDL reduces cardiac events, but may protect against cancers and other diseases. It reminds me about the early studies done by cardiologists showing aspirin reduced heart attacks. But when others used all cause mortality as the criterion, aspirin barely had an effect. It reduced heart attacks at the cost of increasing abdominal bleeding and strokes. The cardiologist must have thought that was someone else’s problem.
I looked more into this use of all cause mortality discussed above. This seems to have been a problem plaguing this cholesterol idea from the beginning:
https://jamanetwork.com/journals/jama/article-abstract/391065
Tables 3 and 5 show the results. In total, 71/1,900 in control died vs 68/1,906 in treatment.
Breaking it down further we get 44 vs 32 heart disease related deaths but 27 vs 36 non-heart related deaths.
It is amazing to me that a difference of 12 deaths spawned an entire cholesterol-lowering industry while a difference of 9 deaths was “non-significant”. Thus is the power of NHST.
They also say:
Cholesterol is a major component of every cell in the body, including the nervous system (eg, lipid rafts). There are an essentially endless number of plausible ways lower cholesterol can be linked to higher accident or violence rates via attention/emotion/etc. It is also the precursor for all steroid hormones..
Why does cholesterol rise with age and become elevated in some people anyway? There is probably higher turnover under certain conditions, so the body absorbs/produces more to compensate.
Yup. I lot of these studies are just documenting noise, NHST or not. The results usually sound good when only relative risk is reported. Too much complexity to tease apart causal influence of anything when it comes to ‘all cause mortality’