More than a decade after the earthquake that was the replication crisis (for some background, see my article with Simine Vazire, Why did it take so many decades for the behavioral sciences to develop a sense of crisis around methodology and replication?), it is frustrating to see junk science still being published, promoted, and celebrated, even within psychology, the field that was at the epicenter of the crisis.
The crisis continues
An example that I learned about recently was an article out of Harvard, Physical healing as a function of perceived time, published in 2023 and subsequently promoted in the news media, that claimed to demonstrate that healing of bruises could be sped or slowed by manipulating people’s subjective sense of time. All things are possible, and never say never, but, yeah, this paper offered no good evidence for its extraordinary claims. It was standard-issue junk science: a grabby idea, a statistically significant p-value extracted from noisy data, and big claims.
Someone pointed me to this paper, and for some reason that I can no longer remember, Nick Brown and I decided to figure out exactly what went wrong with it. We published our findings in this article, How statistical challenges and misreadings of the literature combine to produce unreplicable science: An example from psychology, which will appear in the journal Advances in Methods and Practices in Psychological Science.
In short, the published article was flawed in two important ways, first in its statistical analysis (see section 2.4 of our paper, where we write, “We are skeptical that this study reveals anything about the effect of perceived time on physical healing, for four reasons”) and second in its interpretation of its cited literature (see section 3 of our paper, where we write, “Here we discuss three different examples of this sort of misinterpretation of the literature cited in the paper under discussion”).
I don’t have any particular interest in purported mind-body healing, but Nick and I went to the trouble to shepherd our article through the publication process, with two goals in mind:
– Providing an example of how we, as outsiders, could look carefully at a research article and its references and figure out what went wrong. This is important, because it’s pretty common to see papers that make outlandish claims but seem to be supported by data and the literature.
– Exploring what exactly goes wrong–in this case, it was a mis-analysis of a complex data structure, researcher degrees of freedom in decisions of what to report, and multiple inaccurate summaries of the literature.
What does it take for junk science to be successful?
All this got me thinking about what it takes for researchers to put together a successful work of junk science in the modern era, which is the subject of today’s post.
Before going on, let me emphasize that I have no reason to suspect misconduct on the part of the authors of the paper in question. It’s a bad paper, and it’s bad science, but that happens given how people are trained, and given the track record of what gets published in leading journals (Psychological Science, PNAS), what gets rewarded in academia, and what gets publicity from NPR, Ted, Freakonomics, and the like. As we’ve discussed many times, you can do bad science without being a bad person and without committing what would usually be called research misconduct. (I actually don’t think that bad data analysis and inaccurate description of the literature would usually be put in the “research misconduct” category.)
This is also why I’m not mentioning the authors’ names here. The names are no secret–just click on the above link and the paper is right there!–I’m just not including them in this post, so as to emphasize that I’m writing here about the process of bad science and its promotion; it’s not about these particular authors (or any particular authors).
7 steps to junk science
So here they are, 7 things that allow junk science to thrive:
1. Bad statistical analysis. Statistics is hard; there are a lot of ways to make mistakes, and often these mistakes can lead to what appears to be strong evidence.
2. Researcher degrees of freedom. Garden of forking paths. As always, the problem is not with the forking paths–there really are a lot of ways to collect, code, and analyze data!–but rather with selection in what is reported. As Simmons et al. (2011) unforgettably put it, “undisclosed flexibility in data collection and analysis allows presenting anything as significant.” And, as Loken and I emphasized in our paper on forking paths, “undisclosed flexibility” could be undisclosed to the authors themselves: the problem is with data-dependent analysis choices, even if the data at hand were analyzed only once.
3. Weak or open-ended substantive theory. Theories such as evolutionary psychology, embodied cognition, and mind-body healing are vague enough to explain just about anything. As Brown and I wrote in our above-linked article, “The authors refer to ‘mind–body unity’ and ‘the importance of psychological factors in all aspects of health and wellbeing,’ and we would not want to rule out the possibility of such an effect, but no mechanisms are examined in this study, so the result seems at best speculative, even taking the data summaries at face value. During the half hour of the experimental conditions, the participants were performing various activities on the computer that could affect blood flow, and these activities were different in each condition . . . there are many alternative explanations for the results which we find
just as scientifically plausible as the published claim.”
4. Inaccurate summaries of the literature. This is a big deal, a huge deal, and something we don’t talk enough about.
It’s a lot to expect the journal editors and reviewers to check citations and literature reviews. It’s your job as an author to read and understand the work you’re citing before using those papers to make unsupported claims. For example, don’t make the claim, “If a person who does not exercise weighed themselves, checked their blood pressure, took careful body measurements, wrote everything down, maintained their same diet and level of physical activity, and then repeated the same measures a month later, few would expect exercise-like improvements. But in a study involving hotel housekeepers, that is effectively what the researchers found,” if you’re citing a study that does not support this claim.
5. Institutional support. Respectable journals are willing to publish articles that make outlandish claims based on weak evidence. Respected universities give Ph.D.’s for such work. Again, I’m not suggesting malfeasance on the part of the authors; they’re just playing by the rules that they’ve learned.
6. External promotion. This work was featured in Freakonomics, Scientific American, and other podcasts and news outlets (see here and here). This external promotion has three malign effects:
– Most directly, it spreads the (inaccurate) word about the bad research.
– The publicity also provides an incentive for people to more sloppy work that can yield these sorts of strong claims from weak evidence.
– Also, publicity for sloppy, bad science can crowd out publicity and reduce the incentives to do careful, good science.
7. Celebrity culture. This is a combination of items 5 and 6 above: many celebrity academic and media figures prop each other up. Some of it’s from converging interests, as when the Nudgelords presented the work of Brian Wansink as “masterpieces,” but often I think it’s more just a sense that all these media-friendly scientists and podcasters and journalists feel that they’re part of some collective project of science promotion, and from that perspective it doesn’t really matter if the science is good or bad, as long as it’s science-like, by their standards.
Anyway, this continues to bug the hell out of me, which is why I keep chewing on it and writing about it from different angles. I’m glad that Nick and I wrote that paper–it took some effort to track down all the details and express ourselves both clearly and carefully.
Number 8, maybe, is that very highly educated people tend to trust (or at least not distrust) NPR and Ted talks, and scientific journals in general. Our first instinct is not to say, “That is wrong.” There is no level of education high enough to induce skepticism, unless someone happens to have tuned into the replication crisis or be a reader of Andrew Gelman’s blog.
Great list. I find myself wanting the computer science or more specifically AI and machine learning version! I think it would be similar but with a few tweaks. E.g., not just inaccurately summarizing the literature, but not trying to so hard to summarize it at all! Throwing in inaccurate analogies to human cognition or human processes for solving the task. Designing experiments around straw men baselines or unrealistic synthetic data. Unnecessarily “branding” techniques with slick sounding phrases as if you’re trying to convince the reader to invest in your startup.
Careful there! You’re sounding like me. (I don’t advise it, but if you insist, I’ll cheer. Loudly. (Especially the “inaccurate analogies to human cognition” bit.))
I’ve been somewhat changing my tune, at least recognizing that gradient descent plus domain-specific search does kewl things.
Still, I took great glee in the iPhone AI news summary app crashing and burning.
In this day and age, having something to laugh at is seriously appreciated. Thank you, Apple!
Appropriate use of terms would help as well. We do not currently have ‘artificial intelligence’, and LLMs in particular aren’t even close to it. They’re just Searle’s Chinese Room running on massive servers. However, marketing has taken over so everything from chatbots to digital picture tools built into a phone gets called ‘AI’, with researchers, engineers and other experts all playing along.
Searle’s Chinese Room thought experiment posits systems indistinguishable from human experts at the level of input-output behaviour (the “Chinese Room” name refers to linguistic competence, but the argument is developed to incorporate all externally observable behaviour). Searle’s contention was that even systems exhibiting such behaviour would not possess “understanding” (note: many contemporary commentators and, later, Searle himself acknowledged that his argument was better understood as referring to consciousness). Thus, if we really did have “Searle’s Chinese Room running on massive servers”, we would have systems exhibiting behaviour precisely as intelligent as any human behaviour you care to commend as worthy of that label.
The above is not directly relevant to the tricky question of how we should evaluate the intelligence of current AI systems, but I thought it worth pointing out the ironic lack of accuracy in a comment supposedly troubled by the lack of “appropriate use of terms”.
Adobe Acrobat popped up with “Short on time? Ask for a quick summary”. So I wrote “Summarise this article and identify the theoretical weaknesses”, and honestly it didn’t do the worst possible job, even though the ending is a bit generic:
This study by Peter Aungle and Ellen Langer investigates the impact of perceived time on physical healing. Participants were subjected to cupping therapy to create standardized wounds and were then placed in three different conditions: Slow Time (perceived as half the actual time), Normal Time (equal to actual time), and Fast Time (perceived as twice the actual time). Despite the actual elapsed time being 28 minutes in all conditions, the study found that wounds healed faster in the Fast Time condition and slower in the Slow Time condition, supporting the hypothesis that perceived time significantly influences physical healing.
The study is grounded in the theory of mind-body unity, which posits that the mind and body interact bidirectionally and simultaneously. The results suggest that psychological experiences, such as the perception of time, can directly affect physiological processes like healing.
Theoretical Weaknesses:
Conditioned Expectations: The study’s design included at-home exercises that may have conditioned participants’ expectations about healing time, potentially confounding the results.
Lack of Direct Measurement of Perceived Time: The study did not directly measure participants’ perception of time, making it difficult to assess the effectiveness of the time manipulation.
Generalizability: The study’s findings may not be generalizable to populations without internalized notions of clock time, such as those in non-industrialized societies.
Temperature Influence: The study did not measure body temperature, which could be a confounding factor as warmer temperatures are associated with faster tissue repair.
Limited Scope: The study focuses on a specific type of wound (cupping marks) and may not be applicable to other types of injuries or healing processes.
Overall, while the study provides compelling evidence for the influence of perceived time on healing, these theoretical weaknesses suggest areas for further research to validate and expand upon the findings.
Nick:
That is a terrible, terrible summary. I’d say it pretty much does “do the worst possible job.” The summary says, “the study provides compelling evidence for the influence of perceived time on healing.” That’s not at all true. It’s the result of the sort of shallow associative reasoning that you will get from a chatbot–or an inattentive journal editor!
Well, maybe apart from the “compelling evidence” bit. But I thought it made a good job of “understanding” the limitations.
However, I agree that it’s too trusting of the paper. But then, why would we expect an LLM to be skeptical about an article in a Leading Peer Reviewed Academic Journal™? After all, it’s probably been trained on 25 years of PPNAS and friends. I wonder what will happen when the AIs have been trained on the p-hacking (etc) literature?
I asked ChatGPT to check some tables with GRIM the other day, and it failed badly. But I have been contacted by someone who is interested in building a model that can check published statistics at this basic level for their final-year undergraduate computer science project. I’m eager to see the results.
Nick:
The AI could be trained on the complete works of Cass Sunstein . . . or maybe that would be too large of a corpus for it to swallow.
One problem with actually existing chatbots is that with current incentives they will lead to more quickly-produced papers based on System 1 thinking, when what we really need is to cut research output and think harder about the research we read. One function of academic publication is to limit the flow of information that we need to process (ie. we agree to consider and only consider peer-reviewed publications, or works from these 10 journals and five presses, because considering anything, from any source, in any format is very time-consuming).
Sean:
For some reason I’m reminded of the scene in the 1950s sci-fi classic, Earthman Come Home, when the heroes learn of the collapse of the germanium standard. That episode has always resonated with me.
You want inaccurate summaries of the literature? A recent paper I reviewed (excerpts generalized to remove identifying information) said “A can do B (citation)”. I was surprised that A might be able to do B, so I checked the citation. It did not make that claim. A few lines later in the introduction: “We chose dataset C because it’s less biased than dataset D in measuring E (citation).” I was surprised that C was less biased than D, so I checked the citation. It did not compare the datasets’ measurement of E. With antennae sufficiently stimulated, I read the next sentence “Dataset D also is better than other datasets because of F (citation).” Yep, checked the citation, and it didn’t evaluate F across datasets.
On to the methods: “G is defined as H exceeding J at least K times during the months L-M for years N-O, following (citation).” Sure enough, the cited paper didn’t use J or K as criteria and didn’t focus on the months L-M. If you’re wondering whether the cited paper at least studied years N-O, nope. O was 6 years after the cited paper was published. But at least it did study G.
This was the first paper I ever reviewed where I didn’t make it to the results section.
John,
Yeah, that’s perhaps even worse than claiming that “the suggestion that one had touched poison ivy resulted in stronger symptoms than actually touching poison ivy,” with this claim entirely based on an uncontrolled and never-replicated experiment on 13 people from 1962 that did not even involve poison ivy (see page 9 of our paper).
It is frustrating indeed. But why are we frustrated? Is it because we still expect social science to move into a more fruitful direction? And social science fails again and again? As you write here:
“[bad science] happens given how people are trained, and given the track record of what gets published in leading journals […] what gets rewarded in academia, and what gets publicity […].”
In other words, scientists are simply following incentives and doing what they always did. If you think about the world as institutions evolving through time and you look at various institutions like science, soccer, liberal democracies, technology and so on, then some institutions seem to do better than they did 10 years ago. I think this is true for much of professional sports and technology, where are strong incentives to do better than the last time (fame and money being the incentives here). But our liberal democracies are in danger, because it pays off to not play by the rules. And science? Well the incentive structure is broken. It pays off to publish quickly written papers that make strong claims from noisy data. Since incentives do not change, we can not expect science to change. So maybe we are frustrated because our wish that science does better is just hopelessly unrealistic. Maybe we should expect that all this garbage continues till eternity and than we can celebrate in joy every single paper that does better.
Very wise post.
#3 is a big part of the problem in economics. Economists used to do economics, which meant there was clear theory and the idea was to test some implication of that theory. Sometimes the implication was specific (as in “structural micro”) and sometimes less formal. But there was theory. Now the only “theory” is “X matters” and that is an open invitation to p-hacking and other misconduct.
#4 was always a weakness in economics. Economists tend not to read earlier work, in economics or in other disciplines. Now that they poach on every other discipline, they are strip-mining claims from other disciplines to use as support for their stories. Even the most well-intentioned don’t know the pitfalls of this approach. And the malign can misrepresent the earlier literature with ease because there are few in the discipline who can call them on it.
#5 may be the core problem for economics. There are huge rewards to publishing in a few journals. Those journals either cannot or do not care to provide meaningful evaluation of “cute” work. You can make a great career in economics doing work you know is terrible.
Having read the paper I thought it was pretty good and it was an appropriate paper to write about. I made some notes in case you care for some post publication feedback early on the paper itself and not necessarily this blog post.
I can see the sentiment of being uncritical of Langer when she was cooperative about the data for that and a prior paper. But this is just the most recent in a line of work just like it from her. I do not believe anything she has done about the mind-body connection has ever been consistently replicated, or replicated at all. Yet, she remains a luminary in psychology for many. Of course, it was appropriate for you to stay on topic Andrew. Going through all of her work was clearly beyond the scope of your project. Commenting further would be ad hominem without much more work.
I can see skeptics of your approach finding some inconsistencies with arguments that large effects are implausible and small effects are implausible (in t-scores anyway). I’m not sure you make the conditions that accompany them and the why really clear to the uninitiated.
It’s unclear in your paper what a responseID is and what a subject is. Someone’s a rater and someone get’s sucked on but it’s unclear which is which.
I could also see Langer, or someone else, trying to hang you on the suggestion that the hotel workers in the treatment group got more exercise based on their rating. They were told what they were doing was exercise and, in so far as it was news to them, it would have shifted their ratings against groups not so informed without any change in exercise. I think you should have been more clear on that. The way you wrote it suggests you think the rating of their belief regarding exercise is a rating of actual exercise increase.
I was very glad to see an example of assessing the cited literature in more detail. I must say that I have been guilty of letting it slide myself when coauthors want to stick in a paper to support a claim that doesn’t really support it. It has usually been in the context of something we were going to say anyway and has other support. But I try to limit my citations to strong papers. Anyway, typical critiques go into the results and methods and not so much background literature. I’m glad you looked at that as well. It’s an even flimsier foundation than you convey.
Psyoskeptic,
Thanks for the comments. In quick response:
– Regarding the question of addressing Langer’s work more broadly, we did have some discussion about this, actually I think it might have come up from a reviewer comment too. I thought it would be better to focus just on this one article, so that our paper would be more useful as an example of how to carefully look into this sort of outlandish claim. There’s room for another paper–not by me!–on the larger problems with this mind-body-connection work, focusing on the larger theoretical claims rather than on the statistics.
– Regarding the large effects etc.: A key part of our paper is section 2.4, where we give all our reasons for skepticism. One problem with a lot of research criticism and debate is that there’s this idea that there should be a “slam dunk” or a “smoking gun” such as a t-statistic going from 2.5 (“significant!”) to 1.5 (“not significant!”). Not having a single “smoking gun” made our paper harder to write, but ultimately I think it made it better, in that we couldn’t rely an an easy “debunking” and instead we had to be more careful and thoughtful about the problems we saw.
– Our point was not that the hotel workers in the treatment group necessarily got more exercise but rather that it was possible–after all, they said they got more exercise! And that’s even beyond the issue that not-statistically-significant is not the same as zero. Basically, there’s a pile of messy evidence that was arranged to tell a story. Again, researchers can do this sort of thing without any malign intent.
– Finally, yeah, I’m glad we went to the trouble to look at the literature review carefully. I remember having a similar feeling after reading the Nudge meta-analysis, which included papers by Ariely etc. as well as many papers by non-disgraced researchers that had lots of problems. And when reading Bem’s ESP meta-analysis, which included all sorts of weird things like a study of spiders or something like that. Tracking down references is like pulling up a rock and looking at all the slimy things wriggling underneath.
Ha… slimy things.. good. I’ve felt similar.
I just want to note that most of my comments were about clairity and how things are conveyed, not what the intent was. I know it’s hard to close all of the loopholes for the skeptics when things are true, like large effects and small effects are suspiciousgiven the right circumstances for each.
Psyoskeptic:
Agreed regarding loopholes–there’s always a way out of these things. People are still out there defending “power pose,” a decade after the first author on the original paper publicly disowned the entire enterprise! And Brian Wansink’s still out there, etc. With sufficient chutzpah, all things are possible.
As I said in my above comment, I think it was an unexpected benefit for us that we weren’t able to achieve a simple “smoking gun” or “slam-dunk,” as this meant we didn’t have the easy option, as it were, but rather we were forced to think more carefully about the evidential claims that were being made in that paper.
Indeed, the literature is full of bad research making outlandish and unsupported claims, but without the smoking-gun flaw. So it’s good for people to have example of reasoning showing how, even without the smoking gun, we can feel ok about dismissing the published claims in that paper.
Just speaking generally, there’s this annoying dynamic whereby, before publication, any trivial objection made by a reviewer can be enough to hold up a paper, but, after publication, there’s the implicit standard that only the smoking gun will be enough to cause disbelief. Had Nick and I found the smoking gun, we would’ve reported it, but, as I said, I think it ultimately worked better that we didn’t.
Unfortunately, I don’t know how many people will read an article in “Advances in Methods and Practices in Psychological Science,” but ya gotta start somewhere. Maybe someone will pick up a baton and write an article for a larger audience about the abject failure of the mind-body-connection literature. Really, it could deserve a whole book! The author could devote a whole chapter to the way in which people who should know better, such as Steven Levitt and Sean Carroll, decided to get sucked into all this.
Just saw that this blog post was picked up by Retraction Watch in the Worthwhile Reads Elsewhere section.
The best way to fame from junk science!
Dear Andrew,
I have read your (with Nicholas Brown) severe criticism of the paper by Aungle and Langer (2023). I am not too surprised by their largely insufficient mixed-effects modeling with lme4 in R. Indeed, as you noted–and try to remedy this with a Bayesian analysis–when we go from an analysis with random variation of the intercepts to a more complete analysis with (also) random variation of the slopes, lmer displays, in red, a message: “boundary (singular) fit: see help(‘isSingular’)”. As a result, many researchers think it is better to publish the analysis with random variation of intercepts alone, which has the merit of perfectly run, rather than the questionable analysis with random variation of slopes. Many researchers have certainly resorted to such pragmatic “reasoning”. The psychological literature must therefore be full of results as far removed from reality as Aungle and Langer’s.
On the other hand, I am more astonished, especially by an experienced researcher like Ellen Langer, when Aungle and Langer report df in excess of 2000 with an observation of around 30 participants (32 or 33). This number of df seems to be roughly obtained by a calculation of 3 (conditions) x 25 (judges) x about 30 (participants) > 2000. Such a calculation, which includes the number of judges, is obviously suspect, because if Aungle and Langer had used 10 times as many judges (or, 1000 times, in thought!), they would have obtained a perfectly stabilized measure of the healing rate (i.e., SE = 0) and so we could simply compare the difference between two conditions, calculated for each of the 32 or 33 participants, with a t-test comparing the mean of these differences to a fixed mean 0. Such a comparison leads to a number of df of the order of 30 (32 or 33). In fact, I verified, using your Model_varying_slopes model and an estimate of the df with Satterthwaite’s method, that this is indeed the order of magnitude that we obtain: 32.0845 and 33.2710 precisely for the comparisons 28 min vs. 14 min and 56 min vs. 14 min, respectively.
Jean-Paul:
That’s funny–I hadn’t even thought of the error message as something that would spook people! In practice, this degeneracy is pretty much irrelevant, given that it’s arising with the varying slopes for rater, and these are basically indistinguishable from zero anyway, but lots of people won’t realize that. I guess that’s something we need to emphasize in our next book. Also I can write a post about it.
I remember that whole problem. All these arcane terms and choices, you are never sure you “did it right”. This causes anxiety and a need to rely on authority/consensus heuristics.
I did like 2 years down this rabbit hole until finally accepting the horrifying truth. Its all BS.
<b<The only metric to judge these arbitrary models by is out-of-sample predictive skill. The coefficients are entirely arbitrary. There are effectively an infinite number of equally valid model specifications, all with different values for the coefficients (that share the same name). These coefficients have as much meaning as the weights of a neural network.
And if that wasn’t enough, these arbitrarty numbers are then compared to another arbitrary number (ie. 0.05).
My point being, fixing the issues you point out is still just as much BS as the one used by psychologists who got scared of the error.
If you want meaningful coefficients, you must come up with some premises and derive a quantitative model of the process that generated the numbers.
Attempting to avoid this with arbitrary statistical models has been a huge mistake. You can feel assured future generations (assuming the misinformation doesn’t destroy our civilization/species first!) will be ignoring anything related to that, like today we ignore the millenium of (considered very important at the time) theological ramblings that came before.
I suspect things are getting much worse. There is no incentive to change in a self regulated and profitable industry where tangible outcomes have been dissociated from default markers of success or prestige.
When publishing garbage under a veneer of gimmicky sounding pseudoscientific terms like ‘organoids’ or ‘connectomes’ progresses careers and profits by elite journal vetting with profound coi issues and neo phrenological practices like fMRI investigations of cognition can make you tenured or bestow on you a Kavli award, where illegitimate masking is a routine protocol to get desired results. Where critics and peer reviewers (anonymous and self invested) and too few to be taken seriously) all have their snouts in the trough. Where extremely poor replication rates are simply abstracted away by not naming culprit papers and only brought to light by industry that is wondering why all its Nature and Science publications aren’t translating into profits cf Amgen. Where an individual can give a 5 minute TED talk and claim consciousness is a controlled hallucination (complete nonsense) and get 15 million views a statistic that makes you a sought after academic. Where jealously guarded cabals of so called scientists with appropriate ties to powerful institutional and publishing instruments completely control the narratives. In summary, where really good ideas or theories are not valued as much as bad ones, one because the former are quite rare and two because the latter are very profitable and default markers take the place of real outcomes. All this means is that academia is divesting itself of science as useful interpretation and investing more in a church with high bishops and the simonism of medieval times. Not as an objective yardstick but more and more as a tenuous inconvenient reality marketable only in name.