This has come up a few times on the blog already:

Carroll/Langer: Credulous, scientist-as-hero reporting from a podcaster who should know better

7 steps to junk science that can achieve worldly success

A suggestion for Freakonomics and Sean Carroll: Interview Nick Brown

Two researchers in the Harvard psychology department published a paper reporting that they could make people heal faster by telling them that more time had passed. Nick Brown and I looked at this paper carefully and didn’t think that it offered good evidence for its claims. Meanwhile, the paper was promoted uncritically in various media outlets.

As I wrote a couple years ago, to the extent that healing is important, I think it’s important not to overstate evidence for speculative claims about what works. Individual and societal resources are limited. If you want to say something like, “Sure, this is pie-in-the-sky research, but if it works it would be wonderful (‘kind of amazing,’ as physics podcaster Dean Carroll might say), so it deserves our attention, respect, and funding as a high-risk, high-return possibility” . . . go for it. That argument could be made. But then that argument should be made. Don’t fudge it by acting as if there’s evidence that isn’t really there.

Nick and I published an article in a psychology journal discussing the problems with the paper in question, framing it as a more general exploration of how scientific errors can propagate. One of the authors of the original paper then published an article in that journal arguing that we had gotten it wrong and that they really did have strong evidence. Nick and I didn’t find their response convincing on scientific or statistical grounds, but we thought it could possibly be rhetorically effective: just as a piece of writing, if you read it in isolation, it might make you think that we were full of crap. So we closed the loop by replying in the journal, basically restating what we’d said in our earlier article.

The four articles are in different places online and I thought it could be helpful to have all of them in the same place. So here they are:

Peter Aungle and Ellen Langer (2023), Physical healing as a function of perceived time:

In this study we wounded study participants following a standardized procedure and manipulated perceived time to test whether perceived time affected the rate of healing. We measured the amount of healing that occurred across three conditions using a within-subjects design: Slow Time (half as fast as clock time), Normal Time (clock time), and Fast Time (twice as fast as clock time). Based on the theory of mind–body unity—which posits simultaneous and bidirectional influences of mind on body and body on mind—we hypothesized that wounds would heal faster or slower when perceived time was manipulated to be experienced as longer or shorter respectively. Although the actual elapsed time was 28 min in all three conditions, significantly more healing was observed in the Normal Time condition compared to the Slow Time condition, in the Fast Time condition compared to the Normal Time condition, and in the Fast Time condition compared to the Slow Time condition. These results support the hypothesis that the effect of time on physical healing is directly affected by one’s psychological experience of time, independent of the actual elapsed time.

Andrew Gelman and Nicholas Brown (2024), How statistical challenges and misreadings of the literature combine to produce unreplicable science: An example from psychology:

Given the well-known problems of replicability, how is it that researchers at respected institutions continue to publish and publicize studies that are fatally flawed in the sense of not providing evidence to support their strong claims? We argue that two general problems are: (a) difficulties of analyzing data with multilevel structure and (b) misinterpretation of the literature. We demonstrate with the example of a recently published claim that altering patients’ subjective perception of time can have a notable effect on physical healing. We discuss ways of avoiding or at least reducing such problems, including comparing final results to simpler analyses, moving away from shot-in-the-dark phenomenological studies, and more carefully examining previous published claims. Making incorrect choices in multilevel modeling is just one way that things can go wrong, but this example also provides a window into more general problems with complicated designs, cutting-edge statistical methods, and the connections between substantive theory, experimental design, data collection, and replication.

Peter Aungle, Daniel Chen, and Nicholas Holmes (2026), Beyond Statistical Myopia: Replying to a Misguided Critique of Mind-Body Research:

In response to Gelman and Brown’s recent critique of Aungle and Langer (2023), we argue that their article illustrates how narrow statistical reasoning and selective literature review can misrepresent and undermine credible scientific findings. Using their discussion of perceived time and physical healing as a case study, we identify three general problems: (a) a failure to accurately characterize the methods and results of the study they critique, (b) misinterpretations and omissions in their review of the relevant literature, and (c) a tendency to generalize from isolated statistical issues to sweeping claims about the invalidity of mind-body research. We adopt Gelman and Brown’s recommended model and find that the main effect remains robust. We also document errors in their interpretations of other cited studies and demonstrate that they ignore decades of rigorous, well-replicated research on placebo effects and health mindsets. By examining their critique in detail, we highlight how methodological skepticism, when untethered from accurate reading and balanced appraisal, can mislead rather than clarify.

Nicholas Brown and Andrew Gelman (2026), This is the reason for external replication: Response to Aungle et al. (2026):

In an earlier article we addressed a controversy regarding a form of mind-body healing, arguing that a recent paper had overstated evidence from experiments and from literature review. In reaction, one of the authors of that paper disputed our claims. Here we explain why we remain skeptical.

The short answer is that, no, we don’t see any evidence that manipulating people’s subjective experience of time will help them heal better, nor do we see evidence that telling people that they’re exercising will get them to lose weight without their being any changes in their diet or exercise, or various other things claimed in that original paper. I do think it’s possible for researchers, through a combination of sloppy statistics, forking paths, and inaccurate literature review, to create an impression of a strong body of evidence even when nothing is going on–this was a point made eloquently in the classic 2011 article by Simmons, Nelson, and Simonsohn. And I think this combination is enough not just for people to mislead others, but, more importantly, to fool themselves, which can then allow them to spread misunderstanding in the scientific literature, the popular press, and, yes, NPR, Ted, and podcasts.

The whole thing makes me sad, to see researchers caught in a loop of misunderstanding so that, even after their mistakes are pointed out to them, they double down and remain confused. There’s no way that the authors of the above papers will agree with me on this point, and maybe they will find all this to be condescending, but I’m completely sincere here. It makes me sad to see people aim their careers in this direction. The good news is that over the years I’ve received many many emails from young researchers who see this sort of thing going on in their labs and want to do better. I guess the best way to get a grip on this problem is to see how others have been trapped in it.