I heard about the above-titled book by science journalist Jesse Singal when it came out, actually before it came out, as the author had talked with me about some of the topics in the book and had run some passages by me to check before publication. But I only happened to read it recently. As I often say with this kind of book, I’m not the intended audience; still, I find it interesting to see what people have to say about the replication crisis and related problems with public science.
One thing I like about this book is its modesty: it doesn’t push any big theories, instead going through several examples of hyped psychology research that found its way into public attention and policy. The examples are “self-esteem,” “superpredators,” “power pose,” “positive psychology,” “grit,” “implicit bias,” “social priming,” and “nudging.” I’m putting all these terms in scare quotes because they’re slogans as much as anything else.
I don’t want to say that Singal offers no big-picture perspective. His larger theme, beyond his discussion of how shaky ideas can get publicity and influence, is that that this sort of push-a-button-change-your-life version of psychology is appealing as a way for people to make sense of their lives in a time of uncertainty:
The world is a big and scary place, and imposing structures often circumscribe our room to maneuver. That’s why the reductive storytelling of grit is so appealing; that’s why we turn to figures like Martin Seligman for positive thinking that can shield us from trauma, or Amy Chua for parenting advice, or Angela Duckworth for the secret to grittiness. There’s always the idea of the side entrance; there’s always something you, the individual, can do to regain control in a world that sometimes seems hell-bent on robbing you of it.
Well put. The question, then, is what’s new about this? Dale Carnegie’s “How to Win Friends and Influence People” came out in 1936, has sold a zillion copies since then, and remains in print. Before that there was Christian Science and all sorts of self-help movements. One thing that distinguishes “grit,” “nudge,” and “positive psychology” from its predecessors in the bestseller list and the pulpit is that these more recent ideas are coming from academia and have elite media endorsements. Some of this could just be that the academic world is much bigger than it used to be. Just as many people who in the past would have been professional novelists but now support themselves as university teachers, so it could be that professors such as Duckworth, Sunstein, etc., would in the past have been freelance authors or speakers. Indeed, the recent rise of celebrity podcasters has perhaps taken us back to the Dale Carnegie era.
Some of this came up in our Rationalizing the World course: the idea that intellectual movements can come from academic social science or from the popular or commercial world. It may be that the “Quick Fix” era of 2010-2015–the high-water mark of Psychological Science, PNAS, Ted, and NPR–was a transient period.
The celebrity-academic-social-science era took off in 2005 with Freakonomics and ended in 2020 with Covid. If ever there was a time for society to cash the check of the promise of academic social science for society, it was the pandemic. And academic social science pretty much failed the test. Science didn’t fail–vaccines were rapidly developed, tested, and implemented–but social science didn’t do so well, and the authority of science in society failed, as evidenced by vaccine deniers taking over our public health establishment.
The thing that really struck me about Singal’s book, though, was how it seems like a product of a much more innocent time than today. The book came out in 2021 so it was finished in 2020, but I wonder if much of it was written a few years before that. I say this for two reasons. First, he was so gentle on many of the researchers. He keeps giving them the benefit of the doubt. He even had good things to say about the notorious Roy Baumeister!
Second, Singal’s political take on junk science was so mellow. Yes, he considers it to be an important problem–he did write a whole book on the topic!–and he makes the political point that supposed “one quick trick” tweaks (or as Brian “Pizzagate” Wansink would say, “tweeks”) can be viewed as a distraction from serious political reform, in the same way that do-it-yourself environmentalism could be a substitute for real action on global warming. As Singal says, one of the problems with the “quick fix” mentality is that it places the burden of societal problems on individuals, which might not be such a bad idea, except that these quick fixes don’t work. There are also some logical issues that we’ve discussed which he doesn’t get into, such as the piranha problem that all these purportedly large effects can’t coexist and the competition problem that, even if something like “power pose” worked as advertised, it won’t help you get that job if your competitors are power posing too.
OK, yeah, so there’s some political content. But, as of 2017 or 2020 or whenever most of the book was written, there was less of a sense that the political stakes were high. Yes, there was political polarization; yes, the 2016 election featured two unpopular candidates; etc. But it wasn’t like now, where the government is shooting protesters on the street, shooting people in boats, starting wars, etc. Not to mention government science policy: global-warning denial, vaccine denial, and public health authorities that are such a joke that the best thing that even their supporters can say is that the new dietary guidelines is that they’re “not crazy”–yeah, that was supposed to be a positive reaction.
Now, partisans of different sides will have different reactions here. The Democratic take is that the Trump administration is a bunch of corrupt liars. The Republican take is that the science establishment has been in the tank for the Democrats for awhile now, and there’s no good alternative but to shake things up: it’s too bad about the stupid dietary guidelines, but the only way forward is to start over and show the public some respect.
My point here is not to try to resolve this political debate, but rather to emphasize that it exists, and to a much greater extent now than six or ten years ago.
One way to see this is to compare Singal’s book, The Quick Fix, to the podcast If Books Could Kill by Michael Hobbes and Peter Shamshiri. In content, Singal’s chapter on “grit” is very similar to Hobbes and Shamshiri’s recent episode on the topic–but the tone is much different. Part of this can be attributed to politics–Singal is center-left, while Hobbes and Shamshiri are far-left (at least, in the U.S. context)–and part could be just differences in personality: maybe Singal is just a mellower person. But I don’t think it’s just that, and I say this in part because I think Singal really is bothered by science hype. I think it’s mostly that the stakes are higher now. As Hobbes and Shamshiri say, the “grit” phenomenon is mostly played out, but educational fads haven’t gone away. (A weakness of If Books Could Kill is that it’s so politically slanted that even when the hosts are getting all the facts right, they can be missing important context, for example not talking about various liberal-leaning educational fads of recent decades.) But the issue here isn’t really “grit” or the associated publicity campaigns or even education policy; it’s more the sense from people on both sides that the stakes are high.
I think it would be harder for a journalist like Singal to write a book like The Quick Fix with such a mild tone today.
Not that the replication crisis has gotten worse–it’s my impression that in most ways the promotion of junk science has diminished. Gladwell, Freakonomics, and Ted are still around, but I think they take up a smaller fraction of the public bandwidth nowadays; NPR and other journalistic outlets seem more likely to approach new claims with skepticism, even the Association for Psychological Science has reformed a bit, finally publishing my letter refuting their earlier-published lies about me. Nobody’s calling us terrorists or Stasi anymore, either.
So, yeah, I do think social-science hype has diminished. But, outside of academia and the news media, bad things are happening: vaccine denial and all the rest. And I think it’s hard to write about problems in science without all that casting a shadow. The Quick Fix was written in a more innocent time.
To put it another way, examples such as Stanford’s Andrew Huberman shilling supplements or Harvard’s Cass Sunstein partying with Henry Kissinger may have in the past seemed like peccadillos, somewhat relevant to the exploration of bad judgment by credentialed scholars but not so important to the big picture. But now that the manufacturers of supplements have taken over the public health authorities and the modern-day Kissingers are starting new wars, all this takes a more serious turn. I’m not trying to blame Huberman or Sunstein for our government’s misdeeds (or for the misdeeds of the Biden administration, for that matter); I’m just saying that the corruption of science and the science media is concerning now on a new level beyond what it was as of the writing of The Quick Fix, and I think that any new book on unreplicable science would have to address these larger issues.
I’m having trouble reconciling your perception that the stakes are now higher with your perception that the replication crisis is somewhat less than it was. While political polarization and scientific practice are not the same thing, I believe they are connected. And the connection makes me suspect that increased polarization would worsen scientific practice (and that is my perception as well).
A factor missing from this post is that you (Andrew) are getting older. Unfortunately, I am as well. And I am constantly wondering how much aging affects my perceptions of things – things like polarization and scientific practice. I definitely feel like both are getting worse (and rapidly so), but am unable to separate these perceptions from the fact that I keep aging. Objective measurement seems more elusive than I would like.
Are they regularly checking replication rates in psych now? Or perhaps requiring it in grant proposals.
If its not being monitored the problem is likely getting worse. The one trick that will solve ~50% of the problems is to always fund a replication along with the original study. Same overall budget, but half the projects.
The other 50% is testing predictions of quantitative models, rather than doing arbitrary regressions/p–values. But we need reliable data to base these models on first.
Dale –
As flawed as our scientific institutions have always been, I think the current state with Bobby and Jay et al. is objective evidence that polarization and practice at those institutions has gotten worse. If “the science” more generally hasn’t gotten worse, the surrounding environment surely has. Hopefully, the damage can be somewhat mitigated although I think it will be long-standing and deep.
Research methods in psychology have improved markedly in my estimation.
At least with empirical research.
People are differently aware that multiple tests need Bonferroni adjustment. Nobody knew about this concept 20 years ago…….
Polarisation is a different issue, and likely getting worse. But this is challenging to solve…..
Jazi:
I disagree with you on the multiple testing, for reasons discussed in this article published in 2012. But I do have the general impression that psychology research is less junky than it was ten or fifteen years ago.
A problem with the approach Singal takes, like others in his lane, is it opens the door to and predictably helps empower people like Bobby.
I don’t really see Singal as a journalist so much as someone curating a particular storyline. He’s good at pointing out failures, but at least sometimes doesn’t dig deeply enough to give a thorough treatment of alternative views and interpretations. That kind of work has value but it carries risk.
I’m not saying criticism of scientific institutions or academia isn’t appropriate and important, but critique that’s not done carefully causes blowback and drifts into normalizing the idea that our scientific institutions are fundamentally unreliable. If not done carefully, what gets lost is that uncertainty and inconsistency are inevitably part of how science works. If those features are treated as evidence of extraordinary unreliability, it becomes easily exploited.
Joshua:
I see what you’re saying about criticism of problems in science opening the door to uninformed skepticism of serious science. This concern is one reason my colleagues and I are writing an article, A statistical case for qualified scientific optimism. So, sure, I see the concern.
My general here is that there’s a division of labor: I conduct applied research in social science, help others on their research projects, develop research methods, help design software, and write textbooks and tutorials; I also work on metascience and learn by carefully examining and criticizing published work in the literature; other people work on science policy and science communication. I recognize that, by criticizing the work of Brian Wansink or Satoshi Kanazawa or whoever, I’m also implicitly supplying support for bad actors our there such as RFKJ or Dr. Oz who are interested in discrediting science as a whole so as to support their political or financial agendas. I’ll still do my work, though, on the hope that more serious and honest political figures will be able to move forward with a better understanding of the scientific process.
Regarding your statement that Singal “sometimes doesn’t dig deeply enough to give a thorough treatment of alternative views and interpretations”: Sure, Singal is a journalist, not an academic researcher or a careful “data thug” in the model of James Heathers or Nick Brown. As a journalist, Singal’s motivation is to uncover and tell stories, so once he gets to a certain point, he might want to move on. One thing I liked about Singal’s book is that (unlike Nate Silver, say), he didn’t purport to be an expert; he does come to his own conclusions, but with some uncertainty and discussion with experts. This doesn’t always work–as noted above, he treats Roy Baumeister as an expert!–; still, his overall approach seems reasonable to me. And he seems pretty careful, more so than many science journalists, who seem committed to the simplistic scientist-as-hero narrative.
+1
My current reading of things (me being a social scientist in a Western European country) is that while there is a pretty wide-shared sense that science, and the social sciences in particular, is facing serious threats from current political and social political contexts, there is a fundamental divide among (social) scientists regarding how to combat those threats.
On the one side, there are many calls to a closing-of-ranks, appeals to solidarity, and often coupled with that, a push towards explicit political and ideological position-taking. A kind of ‘unite, show no weakness, don’t give “them” anything “they” can attack us with, “our” side is right’-attitude.
On the other side, you have people (me included) that feel like it’s 2 minutes to midnight for us to address the dysfunctions of our field ourselves, or possibly face the consequences of outside actors with nefarious motivations stepping in. That means recognizing and openly engaging with these dysfunctions, stimulating pluralism, improving scientific standards and showing our academic disagreements. Because you can’t just sweep these dysfunctions, and the politicization of the scientific endeavor, under the rug and hope no one will notice. Because we will need to continued support of the general public, and contrary to what some social scientists seem to assume, it’s not entirely made up out of complete idiots. People might not have our training and vocabulary, but in my experience, they can often smell a rat where there is one. So, if science wants to maintain its status as an independent institution, along with rights and resources that come with that, it might be absolutely critical that the scientific community collectively demonstrates a commitment the pursuit of truth (as uncertain, ambiguous and complex that truth will remain to us) above all else. In my neck of the woods, this increasingly appears to be a minority position, if not numerically, then at least in the current public discourse among academics. Somewhat surprising to me, it sometimes almost feels like some academics just want to speed-run the dynamics that are playing out in the US and have already played out in e.g., Hungary.
So, I fully agree with the need for criticism to be done carefully, and the emphasize the uncertain character of scientific knowledge. But I also think that this should not become an excuse to close our eyes – our mouths – to the dysfunctions in our fields.
Ten years ago I would have said a lot of the problems around political opposition to science revolved around the social assumptions about education and public discourse being developed before there was a middle class that was an educated class of any significant numbers. This kind of public needs something in between HS courses and original research, and I don’t think we’re getting it, or moving in the direction of that.
Now I’d add that the politics around what groups are opposing science and what the politics of various groups (not excluding scientists) are has gotten a lot more complicated. The idea that the politics of opposing the modern world entirely would gain any power, this would have been unthinkable ten years ago, and seems to be what we have now.
I haven’t listened to “If Books Could Kill” in a while, but I may have listened to the one on “Grit.” It feels to me like they treat each new pop-psychology or self-help book as a one-off that was written out of the author’s purported desire for the big books that come with support for the worst of capitalism. That might be why I enjoy the business and marriage books more. (The funniest one was “Who Moved My Cheese?”) They probably have less reverence for capitalism and traditional sex roles than they do for psychology as a field (which is new enough that I feel like it was taken a lot less seriously in the 70s when I was in grade school than it is now). I sometimes feel like they assume everyone who isn’t an awful person won’t have their lives affected by the bad aspects of the popularity of a book like “Grit,” and IIRC some of the passages they ridicule seem less well chosen than others.
With a book like “Grit” the actual social science content is, of course, only part of the book. Part is the choice of anecdotes/case studies that get included. Part is the social context that’s chosen for framing, etc.
“big bucks” not “big books”
J: It’s sort of the opposite of “treat each book as a one-off.” They have a semi-joking “One book theory”, where the book they’re reviewing is a pale reflection of the one true Platonically-Ideal pop-psychology / self-help book. Described (by someone else) on [Reddit](https://www.reddit.com/r/IfBooksCouldKill/comments/1p7mgeb/what_is_the_one_book_theory_exactly/):
> Every problem can be solved through willpower, a new attitude, a new consumer product, or This One Weird Trick […] you just need to believe in yourself and develop these passive income streams. To find out how, buy my book and read my unoriginal views which could have been covered in one blog post.”
(I think the description is rather weak, but don’t care to try and summarize it myself.)
I don’t disagree with you on the ideology they find in these books, but I meant a one-off as science, that is, they don’t treat the book as an example of bad science due to poor use of statistics or a now-refuted theory, but as something where the writer veered away from science entirely and into something completely different.
I guess the podcast on Grit is new, and not a repeat? So I can’t have heard it already.
One thing the quote from Singal neglects is the extent to which these are are parenting books, as well as books on teaching philosophy (my kid’s school had the teachers read Grit one year and create a schoolwide program around it), and not just the kind of “how to get ahead” books we think of as self-help. I think the consensus around these kinds of books is kind of inconsistent. On the one hand we think parents should use science and listen to professionals, but on the other hand we ridicule the idea that they should read books on raising children. I think we avoid addressing whether we really disagree with people like Duckworth by blaming parents for sending her royalties, but that’s not really on-topic for this blog.
For what it’s worth, having now actually listened to that podcast, there’s a questionnaire that was mandated for some education programs quoted in the podcast that doesn’t come from Duckworth’s book, I’m pretty sure. “I can do anything if I put my mind to it,” is a core part of “Growth Mindset,” not “Grit.” I don’t think it matters though. I don’t think my thoughts on the subject could fit in this comment box.
When people list the “usual suspects” of junk psychology (e.g. the list in this post), it is interesting (to me) that they tend to think of work with no real connection to basic psychology.
By basic, I mean the general study of fundamental psychological capacities like memory, attention, visual perception, et cetera, that are cobbled together to generate complex behaviours. Basically, what the Germans call “Allgemeine psychologie 1” – General psychology 1.
I sometimes think that the problem might not fundamentally be about research practices per se, but that we do not “microfound” theory in Allgemeine psychologie 1. We’re trying to understand how apples fall to the ground rather than how gravity works.
Mattias wrote: We’re trying to understand how apples fall to the ground rather than how gravity works.
I am not sure that I understand this statement. If he means that psychological researchers are studying basic problems that can be examined with the tools we have today, rather than developing grand theories of how the mind works, then I think I agree. (I probably don’t know enough about the field to have an opinion.) On the other hand, if he is saying that psychological researchers should focus on fundamental issues rather than consequences of fundamental structures, then I think the parallel with physics is weak.
Physicists can tell you a lot about how apples fall to the ground (Newton, Principia, Book II, Section III, Proposition XV describes the physics of decaying orbits such as those of low-earth-orbit satellites) but I understand they are weak on how gravity works.
In 1995, Chandrasekhar quoted Newton’s general Scholium which begins “Hitherto we have explained the phenomena of the heavens of our sea by the power of gravity but have not yet assigned the cause of this power.” and ends “And to us it is enough that gravity really does exist, and act according to the laws which we have explained, and abundantly serves to account for all the motions of the celestial bodies, and of our sea.” (S. Chandrasekhar Newton’s Principia for the common reader at p. 535)
Chandrasekhar then tells us, “There is only one additional statement on can make, at this date, three hundred years later: the quest for the ’cause of gravity’ still continues.”
He did indeed not mean either of those things.
Rather, I’m struck by how there *are no* fundamental structures in junk psychology.
We know a lot about memory, attention, and the rest of allgemeine psychologie 1. We even know a lot about basic learning, on a good day. Yet, this knowledge isn’t used (in any serious way) in junk psychology.
One way of avoiding junk psychology pitfalls, I’ve been thinking, might be to think of complex behaviours in light of the core capacities we know they must arise from. ”Given what we know about basic psychology, how could this work?”
Answering the question of how apples fall by reference to the physics of decaying orbits sounds like the physics equivalent of doing exactly that – unless Newton goes on to postulate different physical laws for oranges in the next chapter.
I’d watch out regarding any basic theories of learning. Like in many other fields, averaging has distorted the data and hidden the nature of the underlying phenomenon. This is known of course, but goes unacknowledged for the most part:
https://www.sciencedirect.com/science/article/pii/S002224962300072X