tl;dr. To the extent that healing is important, I think it’s important not to overstate evidence for speculative claims about what works. Individual and societal resources are limited. If you want to say something like, “Sure, this is pie-in-the-sky research, but if it works it would be wonderful (‘kind of amazing,’ as Carroll might say), so it deserves our attention, respect, and funding as a high-risk, high-return possibility” . . . go for it. That argument could be made. But then that argument should be made. Don’t fudge it by acting as if there’s evidence that isn’t really there.
Pointing to a podcast of Sean Caroll interviewing psychologist Ellen Langer, commenter Mark writes:
I [Mark] found myself wanting very much to ask her to substantiate the grand claims she was making about how mindfulness (as she defines it, which itself was a bit squirrelly) makes people heal faster, reverse age, and feel perpetually alive and happy.
Sean Carroll is a physicist, so I was hoping for a more rigorous dive into the science and data that Langer asserts is overwhelmingly supportive of her claims, but was notably absent from her explanations. Lots of anecdotes and stories and pithy phrases, though!
“Brash and unscientific” indeed. Also alarmingly overconfident in her views (in direct contra to her description of the “mindlessness” – lack of curiosity and doubt – that she alleges virtually all humans have about their own views). Frustrating to listen for so long with no substantive response by Carroll.
I agree. We’ve written about Langer before (see above link), and more recently Nick Brown and I did a deep dive into a couple of her papers claiming to find evidence of mind-body effects. We concluded that those papers are fatally flawed in the sense of not providing evidence to support their strong claims.
Carroll’s podcast is here, and it conveniently has a transcript, so I could find some of the relevant parts.
Here’s the key bit from Carroll, right at the beginning of the interview:
Ellen came out with a new book at the end of last year called The Mindful Body: Thinking Our Way to Chronic Health, which is about the physiological, the health benefits of mindfulness. And it’s very interesting, she has a lot of studies, right? This is very data-based, and some of the results of these studies are kind of amazing. . . . You can think of it as kind of like the placebo effect. You take some pill that really isn’t anything at all and your mind coaxes your body into getting better. But turning that on its head to make it much more intentional and cognitive and active rather than tricking yourself, just thinking yourself into feeling younger, healing faster, generally being more healthy. So I mean the data are there. . . .
I don’t think the data are there. To be precise, some relevant data exist, but, from the published papers, I don’t see these data providing good evidence for many of the claims being made.
More generally, statements such as “This is very data-based” and “the data are there” are nothing but empty hype if you can’t point to the actual data and their relation to the (justly) controversial scientific claims. Otherwise, you’re just bullshitting. You could just as well interview someone about the Loch Ness Monster or whatever and say “This is very data-based” over and over and hope your listeners don’t go and check.
From the podcast interview, here’s Langer:
All right, we take chambermaids, and we first ask these chambermaids these are people who are cleaning hotel motel rooms, whatever, how much exercise do you get? Well, because the surgeon general describes exercise as what you do after work, and they just too tired, they don’t think they’re getting any exercise. All right, so we take lots of measures, the study is very simple. We divide them into two groups, and one group we teach them that their work is exercise, making a bed is like working at this machine at the gym, and so on. So we have two groups there. One group that knows their work is exercise, the other group that is unaware of it. We want to make sure they’re not eating any differently, they’re not exercising any more, they’re not working any harder, everything is the same except for they change in their mindset. Now that they saw their work as exercise, they lost weight, there was a change in waist to hip ratio, body mass index, and their blood pressure. Which is remarkable, right?
Carroll responds with complete credulity:
Oh, yeah.
That’s a bad response! Nick Brown and I looked into that paper carefully and, no, there’s no evidence that “they’re not eating any differently, they’re not exercising any more, they’re not working any harder, everything is the same except for they change in their mindset.” The most “remarkable” thing about this podcast is that the interviewer just accepts these claims.
And here’s Langer with another example:
So we have people, we have three groups of people. Unbeknownst to them, to all of them. For a third of them, the clock is going twice as fast as real time. For a third of them, the clock is going half as fast as real time. For a third of them, it’s real time. The question we’re asking is, will that wound heal based on perceived time, what the clock tells you, or real time? And the answer is perceived time.
Again, no.
On the plus side, they didn’t mention the so-called “poison ivy” study (see section 3.2 of the above-linked article with Brown).
P.S. The webpage for the podcast has a comment section, and most of the commenters express strong skepticism of Langer’s claims, picking up on the lack of persuasive data and the potential risks of taking these speculative ideas too seriously. It’s an interesting example where the commenters are much more grounded than the main post.
I wonder if Carroll will read those comments and reassess.
My recommendation is that he conduct do a followup interview with Nick Brown, a person who, unlike Langer, has not received her Ph.D. from Yale, is not a professor at Harvard, has not had multiple gallery exhibitions, and has never received a Guggenheim Fellowship or a Genius Award—but who is careful not to exaggerate what can be learned from data.
Andrew–
A bit orthogonal to the main point, but, in your cited paper with Nick Brown, you write (p. 5), “. . . the estimates of 0.23 ± 0.09 and 1.05 ± 0.09 correspond to the effects of the 56-min condition compared to this [14-minute] baseline.” Perhaps I’m misreading the R output that is being explicated here, but don’t the estimates “correspond to the effects of the *28-min condition and* the 56-mim condition *respectively,* compared to this baseline.” If this were in a blog post, I’d chalk this up to my misunderstanding or a typo, but since the sentence is in a paper in press, I thought I’d ask about it.
Carrol usually wants to find out what the quest thinks and don’t push hard. That’s OK when he has a subject he is interested in and prepared. It’s learning experience, not fact finding experience. But sometimes it seems he just takes one person from book/movie promotion circuit and just “Joe-Rogans” (verb) it trough. This interview sounded like he was doing just that, or he was taken for the ride when he could not make himself to push back.
Langer has been doing very expensive counterclockwise retreats and started to cash in in mindfulness. The interview works as unpaid advertising. https://themindresearchfoundation.org/langer-mindfulness-institute-2/
“Dr. Langer is also the Founder of The Prestigious Langer Mindfulness Institute. The Langer Mindfulness Institute focuses on three main activities:
– Research testing the impact of mindfulness without meditation on challenging health issues including breast cancer, prostate cancer, PTSD the common cold, diabetes, and sleep;
– Workshops and Retreats teaching participants how to live boldly based on validated research, and
– Redesigning the workplace through offering “mindfulness at work” conferences and consulting services.”
The site very conveniently has ‘Consult Now’, followed by ‘Pay Now’, even above the thorough ‘Disclaimers’ in the Contact options!
Would this type of business present a conflict of interest for publications supporting workshop principles?
Reminds me of blue zone dot com
“All right, we take chambermaids, and we first ask these chambermaids these are people who are cleaning hotel motel rooms, whatever, how much exercise do you get? Well, because the surgeon general describes exercise as what you do after work, and they just too tired, they don’t think they’re getting any exercise.”
But, that is not how the surgeon general defines exercise, as far as Google can tell. The Google AI summary says “The Surgeon General defines physical activity as anything that gets your body moving,” I didn’t find anything real with a clear definition of exercise, but here is some language from the forward of a report when Donna Shalala was surgeon general.
“Support is greatly needed if physical activity is to be increased in a society as technologically advanced as ours. Most Americans today are spared the burden of excessive physical labor. Indeed, few occupations today require significant physical activity, and most people use motorized transportation to get to work and to perform routine errands and tasks. Even leisure time is increasingly filled with sedentary behaviors, such as watching television, “surfing” the Internet, and playing video games.”
My google search on
how much exercise do americans get
led to a bunch of news articles on this and other CDC-published studies:
https://www.cdc.gov/mmwr/volumes/72/wr/mm7204a1.htm?s_cid=mm7204a1_w#contribAff
“NHIS 2020 data were used to conduct multivariate logistic regression analyses by rural-urban status and U.S. Census Bureau region of the prevalence of meeting the aerobic, muscle-strengthening, and combined aerobic and muscle-strengthening guidelines during leisure time among adults aged ≥18 years, controlling for demographic characteristics.”
The guidelines themselves don’t constrain themselves to leisure activities, but apparently the National Health Interview Survey asks about exercise during leisure time so that’s what people study.
Langer “checked out” from publishing things scientifically since a very long time.
Science isn’t a prison. but avoiding to defend your studies, and many times avoiding to write up your studies as academic papers, implies you you (Langer) deserves little credibility beyond “he says she said”.
she’s hasn’t scientific, period.
from memory. I’ve looked into her too many years ago. everything isn’t convincing at the least.
One thing that stood out for me in this blog post is the comment about the person being alarmingly overconfident in her views. I have thought about this in relation to problematic issues and people in academia in general, and psychological science in particular. I also noticed this in recent discussions with, and concerning, certain psychological scientists.
I wonder whether certain personality characteristics and certain actions might have some sort of “survival benefit” in academia and/or psychological science. One of those could be being, or coming across as, very confident. Perhaps this feature interacts with several features of academia and psychological science to produce a situation in which people get away with, or even benefit from, this arguably unscientific behaviour.
For instance, perhaps it’s hard in psychological science to determine who or what is right or wrong and it might be a field in which confidence is often mistaken for competence. Now, when taking into account several features of present-day academia in which power differences and dependencies exist it might be even harder to be critical of certain work and certain people. This might further build the confidence of the (over-)confident scientists because they never receive any real criticism. And as long as they receive attention and grant money, the universities might not care. Or perhaps they aren’t even aware. The scientists can just go on and on and on with telling their stories, and doing it (over-)confidently. And there might be many folks who listen to those stories and mistake their confidence for competence.
Tying into the discussion on conspiracists, how should a layperson evaluate claims in the current environment?
Great question. I am skeptical about lots of things so I don’t know if my answer is in line with what’s the best answer to your question, but I do think my answer might be useful to post.
When I still wanted to try and become a “scientist” at a university or such a place I thought about how I would handle things like dealing with the media when they would have questions, or presenting my findings in papers, or talking about my research, etc.
I was interested most at the time in things like PTSD and depression and anxiety, and I thought about how my research might affect those people dealing with that stuff (including myself to a certain degree). I have some values and views which make me not want to be directive, and let people make their own decisions. I also tend to want to be careful in what I state. The combination of this led me to think that I would try to not propose things that might follow from my findings when you take a few steps in your reasoning. I would try to limit that, and keep it as close as possible to merely presenting my findings. No recommendations, no further steps in reasoning, no tying things to certain other things. Just keep it as simple as possible.
Then I would present my findings in papers that would be accessible to members of the general public. I would view my job as being one that somehow is in service of the general public which means to me that I would want them to be able to go and read my stuff. I would even try and go further by not submitting my papers to peer-reviewed journals because I reason they might have caused and/or sustain many problematic issues. But let’s leave that aside for now, I would want to make sure my findings and papers would be available to those that want to read it.
As said earlier, I would want to be careful in what I claim or state. I think there are enough examples that can be found that make clear that people thought they were doing science optimally, but in fact may not have been. Or that certain methods or analyses are later on found to be lacking in validity or whatever. Given all of that, I would try to make clear that this is the best I can currently do, but that may not imply that it is correct or optimal.
So, those are three things I think I would try and do if I were to be a scientist that receives money to perform research into things like PTSD and writes down findings of experiments in papers, etc. I would try 1) to be careful in what I claim or how I phrase things and always allow for uncertainty and make this all clear to the reader, 2) to make my papers, and possibly data and materials, available to those that want to read my stuff, and 3) to not be directive, to not take too many steps in reasoning, and stick as close to what I found and how I found it as possible. Keep it as clean and simple as possible, and let the reader conclude things for themselves.
Carroll does address the criticisms in the July 2024 AMA episode of Mindscape around 2 hours 16 minutes in. The podcast and transcript can be found here: https://www.preposterousuniverse.com/podcast/2024/07/08/ama-july-2024/
Essentially he still thinks Langer “has something worthwhile to say” and says the episode had “good, insightful things” but he acknowledges that the claims may be overblown and that he should have pushed back more.
Chris:
In that case he should do a goddamn interview with Nick Brown already. Break out of the celebrity-science b.s. circuit and talk with Brown, who’s a thoughtful outsider.
In the linked paper with Nick Brown, this is my favorite sentence:
“Weak theory often goes with noisy data: it is hard to know to collect relevant data to test a theory that is not well specified.”
But before I put this on the slides of my intro stats course: Is there a “how” missing in “to know to collect”? Or is it correct as it is, and I just don’t get it (very well possible)?
I think the interpretation is “it is hard to know [that you should] collect relevant data…”
Not so much how but even that it should be done.
Donna Shalala was never Surgeon General of the U.S.. She was Secretary of Health and Human Services.
Carrol would probably know the psychology of science news well enough not to turn his podcast into a debunking podcast like The Studies Show with Stuart Ritchie (a great listen, by the way!). Disconfirming (negative) findings simply travel less far, as few people like to be downers. I’m reminded of this recent article:
Expressions of uncertainty in online science communication hinder information diffusion
Olga Stavrova, Bennett Kleinberg, Anthony M Evans, Milena Ivanović
PNAS Nexus, Volume 3, Issue 10, October 2024, pg. 439
https://doi.org/10.1093/pnasnexus/pgae439
The next step now is to create channels to discuss the problem in a way that is productive *and* palatable to the public.
P.S. The review for Langer’s recent book on mindfulness contains an endorsement by…. who else…. Dan Ariely.
“What matters more: mind or body? Filled with original research and thought-provoking insights, The Mindful Body shows that the two are not just connected but are actually one, opening us to vast potential for health and happiness.”—Dan Ariely, New York Times bestselling author of Predictably Irrational
Jay:
Celebs stick together, just like Cass “Nudge” Sunstein was a pal of Henry Kissinger.
> Sean Carroll is a physicist, so I was hoping for a more rigorous dive
> into the science and data
Sean Carroll is a proponent of the Many-Worlds theory of Quantum Mechanics, which is nonsense. So, being a professional physicist does not imply that one thinks clearly.
+1