Last year we discussed the problem of scientists who host podcasts in which they credulously and uncritically interview celebrity scientists who are promoting junk science. There was Sean Carroll, a physicist who should know better, fawning over a Ellen Langer, Harvard psychology professor who was making wild claims about mind-body healing and also uncritically pushing the ridiculous claims by Robert Sapolsky, a Stanford biology professor who’s notorious for relying on bogus science.
Both these academic science superstars–the one from Harvard and the one from Stanford–have also been featured uncritically on the Freakonomics podcasts.
As I wrote a few months ago, If you’re a well-trained physicist or economist and you have a public platform and you use it to promote junk science . . . really, what’s the point of it all?
I mean, really, what’s the point? I can think of three reasons:
1. You’re invested in the scientist-as-hero narrative (which I hate), and these people are NPR and Ted-certified heroes with great stories to tell.
One reason why these celebrity scientists have such great stories to tell is that they’re not bound by the rules of evidence. Unlike you or me, they’re willing to make strong scientific claims that aren’t backed up by data.
So it’s not just that Sapolsky and Langer are compelling figures with great stories who just happen to be sloppy with the evidence. It’s more that they are compelling figures with great stories in large part because they are willing to be sloppy with the evidence.
2. Once you have a podcast, you want more listeners. (I have a blog here, I get it.) You get more listeners with good stories. The truth or evidence of the stories is not so important.
3. You outsource your judgment to the academic community, peer-review process, NPR, Ted, and other podcasts. If someone’s a decorated professor at a top university, with papers published in top journals, further validated by top-grade publicity, then it’s gotta be solid research, right? These science-podcasters are too busy to actually look into the evidence that purportedly supports the wild claims they’re promoting.
The question then is, what to do about it?
My original thought was that, if you’re gonna interview people who make outrageous-but-wow-it-would-be-amazing-if-true claims, you should grill them a bit. Express some skepticism and don’t let them just wave away objections.
The trouble is that if you do this your interview would not go well. If you had me on a podcast and asked me tough questions passed along by skeptics who don’t trust Bayesian inference or don’t like polling or whatever, that’s fine–I can respond to such things. That would be fine. But if you push hard against people who have the habit of stretching the evidence, I don’t know what would happen. I’m pretty sure they wouldn’t just collapse and admit that their claims are unsupported. My guess is that they’d refer to other studies that they claim would back them up, to which the podcast host would be able to instantaneously respond. So it would just push things back one more step. Either a waste of time or a disaster if the person being interviewed gets angry.
So I don’t think the strategy of pushing harder in the interview would work.
I’ve listened to lots of podcasts, and I’ve never heard a single one in which the interviewers really challenge the people being interviewed. It’s just not done. I don’t recall even soft questioning, of the form, “People sometimes disagree with you regarding X . . . how would you respond to that?” For better or worse, podcasts just don’t do that.
But here’s something that Carroll and Freaknomics could do. They’ve already done podcasts promoting the work of notorious science exaggerators. Follow this up with interviews of skeptics.
In particular, I recommend interviewing Nick Brown, my coauthor on this recently published paper and an articulate explainer of the problems with junk science in psychology.
Nick isn’t an Ivy League professor, but . . . you’re not gonna tell me that Carroll and Freaknomics are status-obsessed, right? If anything, it’s a great populist story, that Nick Brown, this guy from nowhere, was able to puncture the bubbles of so many highly-credentialed purveyors of junk science. It’s the emperor’s new clothes!
So, Sean Carroll and Freakonomics, here’s your opening. Invite Nick Brown on to your podcast. Go for it.
He’s already been on a podcast, so he has practice!
https://www.michellemcquaid.com/podcast/happiness-equation-wrong-podcast-nick-brown/
“I’ve listened to lots of podcasts, and I’ve never heard a single one in which the interviewers really challenge the people being interviewed. It’s just not done. ”
I think that’s a little overstated, or perhaps your sample is too small to have included any. For example, Freddie Sayers often pitches hardball questions at his guests on the Unherd podcast. Of course, it’s a political podcast, not a scientific one. But Freddie is quite good at that–he even does it with people he agrees with!
That said, as a broad generalization, I think what you say is true. And perhaps it is suitable for the podcast medium. Unlike blogs, which are read deliberately and fully engage the reader’s mind, it is my understanding that most people listen to podcasts while they are doing chores, or going for walks. They are typically only partly paying attention to the podcast–so perhaps this is just the wrong venue for serious intellectual debate.
Clyde:
I think the other thing is that, if a science podcaster asks tough questions, most scientists won’t want to be interviewed in that way. Politicians are used to being grilled by the news media; scientists are used to being deferred to as experts. I’ve been interviewed on podcasts a few times, and I will get some good questions, but I’d be pretty flabbergasted if the interviewer came at me saying that it sounded like I was just talking b.s.
Clyde –
I have listened to Sayers a few times and never heard him push back substantively against anyone who comes from a contrarian perspective (particularly covid-related, “anti-woke,” or economically libertarian). Do you have an example you could offer?
I can’t give you a counter-example that is directly responsive to your question. I tend not to listen to covid-related podcasts any more (by Freddie or anyone else–it’s all become a rehash of the same arguments and nobody seems to have new evidence to bring, nor a new perspective.) I’m not sure what you have in mind with “anti-woke” because it means different things to different people. And as for economically libertarian, well, you may be right–I can’t think of an example there, and Sayers is about as economically libertarian as they come.
However, he recently did a podcast in which he separately interviewed John Mearsheimer and Matthew Syed about the Ukraine war. I skipped the Mearsheimer segment because I subscribe to his substack and have listened to so many of his discussions about Ukraine that I can practically give them myself! On Ukraine, Sayers mostly agrees with Syed’s positions. Yet he repeatedly hit Syed with tough questions and challenged him on his asserted facts. My impression is that he does this sort of thing pretty often on his podcast. (This is not true of the other interviewers on Unherd.) I don’t have a sense of the overall frequency–it may be that I have a recall bias for such events precisely because, as in Andrew’s original post, it is a pretty rare event in podcasting as a whole.
I don’t listen to Freakonomics but I do listen to Carroll. I have to cut you some slack on Langer – I made it through about five minutes before giving up in disgust because it was obvious woo. And to your point, if was obvious to me, a rando on the Internet, why wasn’t it obvious to Carroll?
That’s an extreme exception though. Contrary to your claim, Carroll pushes back on his interview subjects *all the time*. Sometimes he is just teeing up an objection he anticipates from listeners (“why assume there is dark matter, why not just re-examine the laws of gravity?”), but quite often it is because he is interviewing someone with whom he disagrees (about many worlds quantum mechanics, for example.)
I don’t recall the Sapolsky interview, but the claim about chess doesn’t appear in the transcript. I infer that what you are saying is that Carroll ought not to have interviewed Sapolsky because Sapolsky generally makes up junk science for his personal benefit and Carroll ought to have known that by reputation. That seems self-refuting? Carroll is supposed to ignore the reputation about “a decorated professor at a top university, with papers published in top journals” and chase down this *other* reputation promoted by Nick Brown? How does he decide which reputation is right? Is he supposed to “both sides” every interview subject? Doesn’t that just push responsibility down to his audience, who are the most poorly placed to resolve it? And I should add that I find it difficult to imagine a *better* science interviewer than Carroll; it is almost always obvious that he has prepared for his interviews by reading the book being promoted or the papers describing the research.
Ultimately, there just isn’t One Weird Trick for getting around bogus science; the only workable solution is for academic science to do a better job of not producing bullshit in the first place. Maybe Nick Brown can help make that happen and maybe by interviewing Brown Carroll can give him a little bit of an assist, but meanwhile there are still going to be podcast interviews.
Philip:
Everybody makes mistakes, and Sapolsky and Langer are glib self-promoters with lots of experience promoting junk science in media interviews, so it makes sense that they could’ve fooled Carroll. My problem with Carroll’s interviews is that these people are making big claims: as a physicist, I’d hope he’d consider the possibility that he’s interviewing people who are playing fast and loose with evidence. Again, what’s the point of a physicist doing a podcast if he’s just going to act like a credulous NPR interviewer?
In any case, I think Caroll could follow up by interviewing Nick Brown, which could give Carroll an opportunity to reflect on what got wrong when he got fooled by Langer. We can learn from our mistakes, but only when we first confront them. For Carroll to just move on as if nothing has happened, that would be a wasted opportunity on his part.
Phillip –
I think “both sides(ing)” interviews misses the point. Usually podcasters just don’t push guests very deeply. It’s not that they should necessarily push back from the “other side,” but that they shouldn’t just be credulous and passively accept what their guests say.
Admittedly, there’s a structural problem in the that interviewers lack technical background and usually don’t do much work to research the topics. A good example would be Sam Harris’ interview with Matt Rodney and Alina Chan about covid origins. He was just unable to really interrogate the opinions they were expressing (and clearly had no real interest in doing so).
Andrew –
Carroll certainly pushed back against big claims made by Eric Weinstein. It caused quite a stir.
I often listen to Freakonomics Radio just to hear its benign and inspirational ending:
“Take care of yourself, and if you can, someone else too.”
Somehow, this quiet benediction brings solace.
The only better one is: “correct your own English, and if possible, somone else too”
I just read Ezra Klein’s podcast in the NYT interviewing Ben Buchanan about among other things the Biden administration’s approach to the coming of A.G.I.. (Did I just do an Oxford period there?) “The Government Knows A.G.I. Is Coming”, https://www.nytimes.com/2025/03/04/opinion/ezra-klein-podcast-benbuchanan.html. There were three things I liked: 1) he tried to ask hard questions of Ed Buchanan, 2)) it was the most interesting discussion about AI and it’s potential effects I’ve heard, 3) Buchanan’s third book recommendation — “A Swim in the Pond in the Rain” by George Saunders. I took a couple years of Russian in college so I’m excited to see which stories he picked and his analysis…
By the way, I like to read podcasts, am I crazy or what?
As to the “Oxford Period” mentioned by Bill Bottenberg,
https://cherwell.org/2022/09/23/the-oxford-period-su-launches-university-wide-campaign-for-free-menstrual-products/
Tyler Cowen is good at pushing back on guests. He seemed to really object to Jonathan Haidt’s theses about “The Anxious Generation” when he had Haidt on the other month. I don’t know whose side I ultimately come down on (my instinct is to say that smart phones are really bad for us as a society, but I don’t know whether Haidt really proved that case). But I like that Cowen was expressing some skepticism, whereas every other podcast that Haidt was on took his claims as given.
I agree that Cowen does push back – and intelligently so. And so does Ezra Klein. The interviews they’ve done of each other are good examples.
This topic seems to be a part of the larger question: “How should non-experts build informed opinions about scientific claims?” These days (and perhaps always), it’s hard not to feel like everyone is just trying to sell something. The popular press wants new, exciting, and weird (need to sell papers/clicks). Even high profile journals like Science and Nature publish “controversial” (wrong) science for its shock value. “Debates” are a joke (particularly because lying is now normalized). Most of these venues are popular because they provide entertainment, not knowledge. (And scientists are now being criticized for not doing a better job of reaching out to the public.) So, other than getting more degrees, how can we become better informed?
Critical readers like Brown might want to start their own podcasts to set an example for others to follow. I like The Studies Show for a skeptical examination of science: https://www.thestudiesshowpod.com/ It’s a wonder why more sleuths don’t start podcasts like these for outreach. It’d scale better than PubPeer, microblogging, and Substack.
Reaching the broader scholarly community with audio and video media seems like a win-win.
This is long and in the weeds but it’s a very good window into just how bad it is with respect to podcast hosts who are credulous dupes at best, and (more likely) cynical grifters.
For all the problems with the “mainstream media” I find it deeply disturbing that incredibly shitty podcasts are a primary source of information for so many people:
https://www.mtracey.net/p/darryl-cooper-has-some-questions