“Why do people prefer to share misinformation, even when they value accuracy and are able to identify falsehoods?”

I found this question and answer from a leading social science researcher at a a trustworthy internet source:

Why do people prefer to share misinformation, even when they value accuracy and are able to identify falsehoods?

Researchers point to distraction and inattention. When they prompted Twitter users, even subtly, to think about accuracy before sharing content, the quality of the postings improved. Now we just need to get social media platforms to do something that reminds us of our preference for accuracy before we share.

I’m thinking this is a more general problem. NPR and Ted are more like traditional media than social media, but they exist within a social media environment, and both these media are notorious for (a) promoting junk science and (b) not coming to terms with the fact that they promote junk science. Much of the time, the junk science they promote stays up on their sites indefinitely (as with the notorious Why We Sleep), and when they do issue some sort of retraction, I don’t feel that these organizations fully address the problem of their credulity with respect to all the bad studies they didn’t get around to retracting.

I get it: when we produce reports, we can make errors. I’ve published errors myself, and I’ve issued corrections for four of my published papers. Errors are inevitable, and we need to think about the next step, which is how to learn from them. I don’t know that subtle prompting to think about accuracy is enough.

I also don’t think that “lie detection tests” are the answer. First, there’s no such thing as a lie detector. Second, the big problem is not lying but the general free-lunch attitude that there is this large and readily accessible set of small painless interventions can have large and consistent effects, which in a social science context violates the piranha principle, for reasons we’ve discussed in various places on this blog. Ethical problems and statistical challenges are connected through the Armstrong principle and Clarke’s law, so it’s complicated—but I don’t think that lie detection would help so much, even if it were possible, because it’s my impression that lots of people who do junk science are sincere, and I bet that they can justify to themselves even the most extreme cases of data faking as just some jumping through hoops in order to satisfy the persnickety Stasi-type statisticians who are going around insisting on statistical significance for everything.

Yes, I’m a statistician and I hate statistical significance, but I doubt these researchers distinguish between different statisticians. To them, we’re all a single annoying mass, an institution that annoyingly sends mixed messages, sometimes requiring statistical significance and other times telling people not to compute p-values at all. To ordinary researchers who are just trying to do their job, get on NPR a lot, give Ted talks, and get rich and famous, the entire field of statistics is like some sort of pesky IRB that keeps throwing up ever-changing roadblocks. So, to some of these researchers, I imagine that faking your data is kind of like backdating an expense account—a step that, sure, is ethically questionable but really it’s just a matter of fiddling with the paperwork in order to get to what is ultimately fair. And everybody does it, right? As the professor of marketing says, “what separates honest people from not-honest people is not necessarily character, it’s opportunity.”

My point is, yes, this can be viewed as a moral issue, but ultimately I see it as a statistical issue, in that once you start with certain misconceptions (the expectation that effect sizes are huge, that statistical significance can and should be routinely found, that measurement doesn’t really matter if you have causal identification, that everything you want to find is right there, if you put in the effort to look hard enough, that social scientists are heroes who deserve to be celebrated in the national media) and then you operate by avoiding opportunities to learn from your mistakes, then all these other problems flow from that. If you get into this mindset that everything you do is fundamentally correct, then “questionable research practices,” misrepresentation of data and the literature, and out-and-out fraud won’t seem so bad. We discussed this general attitude last year, the view that the scientific process is “red tape,” just a bunch of hoops you need to jump through so you can move on with your life.

As I wrote a few years ago in the context of pizzagate, it’s fine to shine a light on bad behavior, but I think it’s a mistake to focus on that rather than on the larger problem of researchers being trained to expect routine discovery and then being rewarded for coming up with a stream of apparent discoveries in a scientist-as-hero narrative.

P.S. At the link two paragraphs above, commenter Sebastian draws a TV Tropes connection. I love TV Tropes.

23 thoughts on ““Why do people prefer to share misinformation, even when they value accuracy and are able to identify falsehoods?”

  1. Andrew, BRILLIANT post This so frustrates me b/c I do hang around with people who are far more credentialed than I.

    Cyber hug to you.

    As I have implied before, we some screwy notions of falsehood and truths. I believe it’s related to the acceptance of dishonestly in all societies/cultures.

    I can’t wait to read what all ya all have to say.

    I guess that lies fuel markets which is why we accept them so easily.

  2. Didn’t one of the last great psychology frauds also study dishonesty? What are the odds, eh?

    Re: being an institution that sends annoyingly mixed messages, to me it feels like an ethos of “two statisticians, three opinions.” It’s something I like about being a statistician, but I see why others might find it a downside of working with a statistician.

    • That one was weird. Who commits fraud, points out oddities with their own fraudulent data and then publishes the raw fraud data itself? The fraud would have never been uncovered if the researchers had not been honest and transparent, but if they were honest and transparent, who did the fraud?

  3. As a journalist who writes about science, I think journalists who write about science have a role here. You don’t have to be a statistician to spot shoddy work — you only have to know statisticians who don’t mind when you call. If shoddy work stopped getting air time, we might have less of it.

    • Here’s my advice to science writers:

      If it sounds too ridiculous to be true, it probably is. So just call up the author and tell them straight up: “I’ve personally never noticed that hopping my left foot for five minutes after looking at pictures of yellow squares increased my sex drive even a little, let alone by ten times. It seems really unlikely. How confident are you in this conclusion? How much of your own money would you bet on it? Is there work by other research groups that has corroborated this conclusion or suggested it would be true? If not what inspired you to test this seemingly strange claim? Is it possible that your conclusions depend on unverified assumptions about how people behave, or about the types of statistical analyses that have been performed? Can you highlight some of the critical assumptions that your conclusions depend on? Do you expect other research groups to attempt to replicate your work? Are there any researchers that you think would be likely to disagree with your conclusions?

      That’s a pretty generic set of questions for the typical – as Andrew calls it – cargo-cult social science paper.

      • Bubbles:

        Sure, but typically it’s not so easy. First, claims are usually presented directionally rather than in terms of magnitude, and, as we’ve discussed, just about any directional claim can be plausible. Second, it’s usually nothing quite so silly as hopping or yellow squares. Even ridiculous-in-retrospect ideas such as the subliminal smiley faces are supported by some theory. Third, when getting to the criticisms it often helps to get quantitative. For example, the problem with the ovulation-and-voting study was not the idea that women’s attitudes could change at different times of the month—I guess that would happen with men’s hormones too!—but the magnitude of the effect, the implausible 20 percentage point shift. Or with the beauty-and-sex-ratio study, it was a claim of a 36% shift. I’ve read a bit about sex ratios and knew that 36% was ridiculous—but the authors of Freakonomics fell for it hook, line, and sinker, never even questioning it! And Steven Levitt is, we can only assume, at least as quantitatively literate as the average science reporter.

        For all these reasons, I’m loath to recommend that reporters dig into these claims in such detail. I mean, if they can, sure, it’s great, but that’s why I focus on the last question you have: “Are there any researchers that you think would be likely to disagree with your conclusions?” I’d be suspicious of any scientist who can’t recommend someone with a different perspective.

        And, of course, when people refuse to engage with their errors, that’s a terrible sign. It’s too bad that the Freakonomics people had no interest in looking at what they got wrong. They had so much going for them. But not being willing or able to go back and look at errors, that’s a disaster.

        • Hi Andrew,

          Just wanted to check, by “claims are usually presented directionally rather than in terms of magnitude”, is that essentially the difference between the positive/negative correlation of something, and its effect size?

        • When faced with an implausible effect size, there are two different kinds of responses (aside from the most common one where they do not think about effect size or question its plausibility)

          A. That effect size is too large to be true. This research is likely fundamentally flawed and should not be taken seriously.

          B. That effect size is too large to be true. But wouldn’t such an estimate imply that the main claim is directionally true?

          I think there is a tendency for people to lean towards B especially in topics that are further away from their expertise.

        • Andrew:

          I feel like you’re granting such studies more plausibility than they deserve. The refilling soup bowl? That’s plausible? Or:

          “Ovulation led single women to become more liberal, less religious, and more likely to vote for Barack Obama. In contrast, ovulation led women in committed relationships to become more conservative, more religious, and more likely to vote for Mitt Romney. ”

          This is plausible? I don’t agree that it is, theoretical support or not. It seems *remotely* possible that *some* individual women experience such an effect. But that’s not the claim here at all. The claim is “two studies with large and diverse samples”. Populations, not individuals. “Women” as in all people not men. This is a massive piranha swimming in a pool of goldfish for all of history that never made any waves or ate any goldfish – just swam about happily and invisibly.

          People already know these claims are fishy because they are ridiculously simple and surprising, like a dream come true, in conflict with most people’s experience – that’s the reason they’re being made in the first place. They’re headline grabbers and clickbait.

          Whatever the case why not default to the old saw: extraordinary claims require extraordinary evidence. Why can’t reporters press hard for corroborating evidence? Why can’t they press hard for the fundamental assumptions? Why can’t they demand to know why the effect has remained hidden for all of time? Why can’t they press hard to find out what other effects this purported mechanism might create?

          They can and they must. If they don’t, they’re propagating a harmful mythology, because that’s exactly what these claims are: mythology being propagated under the veil of science. They cause harm. They claim people can ignore their hard-earned and valuable experience and believe in numbskull fantasies. They draw legislators and public officials into misdirecting society’s hard-earned resources on policies that fritter those resources away. They undermine real science by co-opting the title “science”. It must be stopped.


          Andrew:

          Should we just put the honesty warning at the top of the treaty with the Taliban?

          I feel like you’re granting such studies more plausibility than they deserve. The refilling soup bowl? That’s plausible? Or:

          “Ovulation led single women to become more liberal, less religious, and more likely to vote for Barack Obama. In contrast, ovulation led women in committed relationships to become more conservative, more religious, and more likely to vote for Mitt Romney. ”

          This is plausible? Really? I don’t agree that it is, theoretical support or not. It seems *remotely* possible that *some* individual women experience such an effect. But that’s not the claim here at all. The claim is “two studies with large and diverse samples”. Populations, not individuals. “Women” as in all people not men. This is a massive piranha swimming in a pool of goldfish for all of history that never made any waves or ate any goldfish – just swam about happily and invisibly.

          The difference between a response in three or five or seven people and a response in entire populations is not merely a difference in magnitude or scope. It’s a fundamental difference in the mechanisms by which these “treatments” supposedly act: an effect in a small number of people can be written off as variation in factors that are known to vary widely. An effect in a population requires a dedicated mechanism, which requires some sort of evolutionary foundation. You can’t make tools without opposable thumbs. By the same token, any ovulation effects on voting have to tap a mechanism that long precedes Mitt and Barack or modern American political questions about conservatism and liberalism.

          This makes it obvious why the “hidden brain” effect is almost always bogus: substantial selection pressure is necessary to generate or select for a behavioral mechanism. Another way of saying that is that evolution doesn’t generate mechanisms that have small effects. Thus it’s almost certain that there will be a wide gap between behaviors that result from random variation among individuals and behaviors that occur across a population. The reality is that the search for the small effect is almost certainly futile.

          But you don’t have to know all that to see that these claims are implausible. People already know they are fishy because they are ridiculously simple and surprising, like a dream come true, in conflict with most people’s experience – in fact that’s the only reason they’re being made in the first place isn’t it?

          Whatever the case why not default to the old saw: extraordinary claims require extraordinary evidence. Why can’t reporters press hard for corroborating evidence? Why can’t they press hard for the fundamental assumptions? Why can’t they demand to know why the effect has remained hidden for all of time? Why can’t they press hard to find out what other effects this purported mechanism might create?

          They can and they must. If they don’t, they’re propagating a harmful mythology, because that’s exactly what these claims are: mythology being propagated under the veil of science. They cause harm. They claim people can ignore their hard-earned and valuable experience and believe in silly fantasies. They draw legislators and public officials into misdirecting society’s hard-earned resources on policies that fritter those resources away. They undermine real science by co-opting the title “science”. It must be stopped.

        • Bubbles:

          I agree with you regarding the low quality of these studies. Let me explain what I mean by plausibility.

          Consider the claim, “Ovulation led single women to become more liberal, less religious, and more likely to vote for Barack Obama. In contrast, ovulation led women in committed relationships to become more conservative, more religious, and more likely to vote for Mitt Romney.” Is this claim plausible? It depends on the size of the effect. If “more likely to vote for Barack Obama” corresponds to a 20 percentage point increase in probability (as claimed in the article), then no, this is not plausible at all. If it corresponds to a 0.1 percentage point increase, then maybe so. But in that case the study is completely useless as the noise in the data overwhelms any possible signal. At this point someone might say they should just ramp up the power, but that wouldn’t solve anything, because even if they could somehow nail down a 0.1 percentage point effect here, there’s no reason to think it would persist in other settings, other times, or in other populations. The issue here is not “statistical significance” or even “practical significance” but rather a more general scientific issue of replicability.

          So, yes, I agree with your general points. My statement about plausibility of directional effects does not contradict your statements.

  4. This doesn’t seem to link to the paper, but I assume this answer is referring to our paper published in March: https://www.nature.com/articles/s41586-021-03344-2

    In the field experiment on Twitter, it is worth noting that we looked at quality of links to rated domains in the day of the treatment. The theory doesn’t say there are going to be large, long-run effects of a single subtle exposure. If anything, some of the criticism of this work has been that the effects are too small; I’d say of course they aren’t giant and that many psychologists have totally unrealistic effect size expectations.

    The attention to accuracy treatments seem to <a href="https://psyarxiv.com/v8ruj"replicate quite robustly in online lab / survey experiments across several countries and panels. There are more ongoing implementations in the field too.

    One thing that separates these kinds of treatments from, say, “goal priming” research that maybe hasn’t held up so well is that these interventions simply remind people of goals they largely already have, at least on reflection.

  5. I don’t think the blame lies so much with NPR and Ted. Sorting out the dubious scientific studies is not so hard when you know how, but they don’t. Doesn’t the real blame lie with the journals that publish the junk, and with the departments that promote people who publish it? Aren’t they supposed to know how to sort studies and scientists out?

    • John:

      I don’t blame NPR or Ted for promoting bad work. I don’t even blame PNAS for publishing bad work. Often it’s easy to see the problems in retrospect but not so easy when the paper comes out. I don’t even blame various notorious journal editors for publishing bad papers . . . the first time it happens.

      Where I blame NPR, Ted, and certain journal editors is when they don’t learn from their mistakes.

      Remember that Harvard study that put North Korea in the middle of the pack on its “democracy index”—comparable to North Carolina, I believe? When the poop hit the fan, the Harvard team just removed North Korea from the dataset. They actively avoided learning from their mistakes.

      Similarly with NPR and Ted. They promote the work of Brian Wansink, Matthew Walker, Dan Ariely, etc., and then when it all falls apart, they might flag their error or they might not, but they don’t wrestle with the larger questions of how they got conned in the first place.

      I have the same problem with the Nudgelords. First they get rooked by Wansink. Fine, it happens. But then they pop back up with “Candidate for coolest behavioral finding of 2019: If a calorie label is on the left of the relevant food item, it has a much bigger impact than if it is on the right.” I’m not kidding—they really said that. Post-Wansink. And when you call them on it, they call you Stasi.

      So the fault of NPR, Ted, the Nudgelords, and the other amateurs out there is not getting conned the first time, it’s that they get conned over and over again, and in a Stockholm-like way, they keep coming back for more. Fool me once, etc.

  6. “When they prompted Twitter users, even subtly, to think about accuracy before sharing content. . .” This sounds a lot like the nudge of getting people to sign odometer readings before rather than after recording the mileage. Your trustworthy Internet source doesn’t link to the research. Has anyone checked the fonts in the spreadsheets?

  7. We love narratives & dislike uncertainty. It’s why fascism and communism continue to stay relevant despite their historical track record. It’s much easier to sell people ONE story that explains EVERYTHING, rather than a bag of effects of dubious size and origin bundled with a bunch of “it depends”.

    Maybe the answer is to build up our own romantic story about statistics? The brave statistician standing up as a bulwark against the unrelenting onslaught of the forces of entropy and deceit? The quantitatively-minded Heracles slaying the multi-headed hydra of Epsilon? All joking aside, there’s probably an element of that in the subculture already. And maybe it wouldn’t be such a bad thing if it spread into the wider culture – as it currently stands, statistics could use a bit of regularization in the PR department. Who knows, maybe in 10 years, the pendulum will swing the other way & there will be real academic witch hunts & over-exaggerated TED talks about how every single paper in social psychology is bogus. The audience will clap, feel a bit smarter, and the cycle will continue.

  8. Because, like many, like most of us, they cannot resist repeating the outrages they observe.

    In this snippet of a Confucian dialogue is the simple antidote to it:

    顏淵問仁。子曰:「克己復禮為仁。一日克己復禮,天下歸仁焉。為仁由己,而由人乎哉?」顏淵曰:「請問其目。」子曰:「非禮勿視,非禮勿聽,非禮勿言,非禮勿動。」

    In the famous passage here, Confucius says that when individuals aim for decency, then so goes the political ‘body’ (‘heaven’ in the jargon here); and the interlocutor asks, “and how does one go about doing that?” and the famous — and famously misinterpreted response: “What is indecent, that which goes against decency; do not make yourself an audience to it; do not watch it; do not speak it; do not (of course) act it!”

    He does not mean “bury your head in the sand”. He means, do not make yourself the vehicle that carries the stuff along and aloft!

  9. If I’ve understood you properly, what you’re noticing is the distinct lack of intellectual virtues currently being practised, such as intellectual humility and openness to being wrong. Robert E. McGrath (2020; https://doi.org/10.1080/17439760.2020.1752781) provides a fantastic analysis of the role of virtues in the historical development of human communities and concludes by suggesting that what appears to be missing from contemporary discourses is the promotion and emphasis of the intellectual virtues.

    What I think you might also be getting at is that we need to address this problem from multiple levels simultaneously. At the individual level, reminding and encouraging people to act according to human values seems likely to be of some help, but only if we’re also addressing things at, for example, the institutional level. Many of our institutions operate according to a competitive logic that assumes the worst in people and ultimately incentivises the behaviours antithetical to the very virtues we hope to cultivate. It’d be like trying to encourage overweight people to attain their dieting goals while employing them as taste-testers in a cake factory. Don’t.

    Maybe even all of this stems from a deeper issue: Why? Why should researchers do the right-but-difficult thing over the wrong-but-easy thing? When researchers can get away with putting less effort into checking their data than they should, what can be done to inspire them to do differently? Because any measures we put in place, any steps we take to address QRPs, etc., will only be bandaids unless they address the root cause. Of course, I’m not sure if this IS the root cause, or if there could be multiple root causes, etc.

    • Thank you David. Very nicely framed.

      Political philosopher John Rawls sought to emphasize intellectual virtues. But there doesn’t seem any similar figure that has had large influence in academia today. Although not a political philosopher, Possibly John Ioannidis may qualify to some degree, based on the citation counts.

      Who and what do we see on mainstream television anyway? Scandal-mongering and fear-mongering which have made us a cynicism filled society. Add to that the commercialization of nearly every aspect of our lives. It serves to make researchers more careful how they craft their career and their research to avoid losing their jobs.

      In any case, not all researchers are exceptional obviously.

Leave a Reply

Your email address will not be published. Required fields are marked *