“The Saturated Fat Studies: Set Up to Fail”

Russ Lyons points me to this recent magazine article by Martijn Katan and a research article, “Diet and Serum Cholesterol: Do zero correlations negate the relationship?” by David Jacobs, Joseph Anderson, and Henry Blackburn, and this video by Michael Greger.

This is interesting stuff, especially as the ultimate truth is still very unknown. It’s good to have stories like this where there is still the fog of uncertainty and no unambiguous good guys and bad guys.

15 thoughts on ““The Saturated Fat Studies: Set Up to Fail”

  1. It is interesting to read something (like the magazine article) which bears a message consistent with my priors, but which also comes off as partisan. Immediately makes me more sympathetic to the other side.

  2. This is more of the between-within issue that is barely acknowledged in nutrition. The hypotheses in nutrition are about within person change, and yet their data overwhelmingly involve between-person covariation. Experiments are often in good shape, but the observational literature debates “puzzles” such as weak correlations between food intake and weight status. It’s true that self-reported intake is noisy, but it’s also important to acknowledge that many diet-health associations are likely non-ergodic.

    Molenaar, P. C. (2004). A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement, 2(4), 201-218.

  3. Thanks a lot for posting this. Very interesting.

    Readers may also be interested in:
    JPA Ioannidis: Editorial: Implausible results in human nutrition research. British Medical Journal 2013, 347, 14 Nov 2013
    http://www.bmj.com/content/347/bmj.f6698
    responses to this editorial:
    http://www.bmj.com/content/347/bmj.f6698/rapid-responses

    Don’t know about this (maybe worth a blog post & reader discussion by itself):
    Dean Ornish: Large Randomized Controlled Trials.
    http://edge.org/response-detail/25497
    from: This Idea Must Die: Scientific Theories That Are Blocking Progress. edited by John Brockman, Harper Perennial, 2015
    http://edge.org/responses/what-scientific-idea-is-ready-for-retirement

    • Ornish: “In any scientific study, the question is: “What is the likelihood that observed differences between the experimental group and the control group are due to the intervention or due to chance?” By convention, if the probability is less than 5% that the results are due to chance, then it is considered statistically significant, i.e., a real finding.”

      Arrgh. How can I ever convince my kid this isn’t right.

      • I disagree with Dean Omish. He does not justify that claim, what is the reasoning behind it? A focus on “chance” should be expected to be extremely damaging. In any scientific study, the real questions are: “What processes may explain the observations? How can we distinguish between these possibilities in the next study?”

        Any experimental design capable of suggesting answers to those questions will rule out chance incidentally if it is possible to do so. On the other hand, if you design your research around the idea that ruling out chance is the goal it will be difficult to say much more.

        To rule out chance, it is often sufficient to aggregate a few superficial measurements made at a single timepoint. That is insufficient to determine the most likely non-chance explanation or yield enough insight into what process generated the data. Since the latter is required to construct quantitative theories, research that focuses on chance will make very slow (if any) progress and be incapable of precise predictions. Since little real progress is being made but there are limited funding resources, the research field will devolve into a marketing competition with all the associated hype and misinformation. After a few generations of this, the “scientists” will be an active obstacle to progress.

        • Apropos Ornish’s ignorance of statistics, this comes to mind:

          Statistics is a subject that many medics find easy but most statisticians find difficult.
          Guernsey McPearson (alias Stephen Senn)

  4. Andrew: The “ultimate truth is still very unknown”

    Yes. And that is scandalous.

    Scandalous because the government has been issuing all kinds of diet advice on the basis of pretty shaky evidence one way or another for more than one-half century. What were scientists up to during this time?

    Scandalous because universities and research institutions continue to put out poorly designed, unregistered, epidemiological studies that buttress the reputations of the authors, journals, and institutions, but do nothing to resolve the substantive uncertainty we face.

    Scandalous because they do this with our tax dollars and until recently with zero accountability. And when post peer reviewers criticize they sue http://retractionwatch.com/2015/03/05/judge-rules-most-of-pubpeers-commenters-can-remain-anonymous/

    PS Here is a another foggy situation related to circumcision http://mosaicscience.com/story/troubled-history-foreskin? 100 years giving recommendations in the US, only three trials done — recently, and in Africa. Go figure.

  5. This is all very interesting, but putting my “we all gotta eat” hat on, it seems the thrust of the argument is that observational studies fail to find an effect because the effect is small and/or the variance in our diets is too small. But that is precisely the point!

    Along the same lines, while I sympathize with the sentiment that we should focus on randomized control trials, I would also question how sure we are that a trial of extremely high risk individuals (as most all trials focus on these) really has much to say about the average person.

    Whatever is going on, it suggests to me that the risk of eating bacon instead of broccoli tonight, and every night hereafter, is really quite small. And is almost certainly outweighed by my increased enjoyment!

    • @jonathan

      The problem you refer to about trials is not a problem of trials but how trials are financed and practiced.

      If we want to make population level diet recommendations (and maybe we shouldn’t) then we better use generalizable trial designs.

Leave a Reply to Vijayant B Cancel reply

Your email address will not be published. Required fields are marked *