Junk science and fake news: Similarities and differences

Jingyi Kenneth Tay writes:

As I read your recent post, “How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions” . . . and still stays around even after it’s been retracted, I realized that there are many similarities between this and fake news: how it is much easier to put fake news out than it is to debunk, how it tends to persist even long after it has been debunked, etc. I was wondering how similar you thought the two issues were, and whether there was anything from how governments/entities are trying to deal with fake news that could be imported to the issue of sloppy/poor research.

I do think there are some connections; see here, here, and especially here.

I don’t know how to think more systematically about these ideas, though.

One difference between junk science and fake news is that most junk science has got to be sincere, with researchers thinking they’re studying real phenomena and then getting fooled by their statistical analyses. In contrast, I’m guessing that most, or at least much, fake news is deliberately made up. I mean, sure, the fakers probably typically think that their larger political goals are worthy, but the news stories are deliberately made up or distorted. Once the news appears, though, there is also a difference: junk science is promoted by credulous organizations such as PNAS or NPR, whereas fake news is promoted by political organizations who have something directly to gain from the fraudulent stories. So I see some similarities, but also some differences between junk science and fake news.

18 thoughts on “Junk science and fake news: Similarities and differences

  1. Good points.

    Another big difference is that fake news usually has a specific result in mind. It is usually crafted to produce a predetermined narrative with predetermined villains and heroes. With fake news, the news story can be produced first, and the available facts and lies are dropped into the story (or omitted) as needed to create the impression the facts came first.

    Much junk science, though, doesn’t need to start with a specific result in mind (except insofar as p<.05 can be said to be a specific result). Any results will do. Metaphorically, the writer detonates an analysis explosion that sends conclusions flying in all directions. The researcher then grabs the pieces that fly the furthest and sparkle the brightest (the ones with the lowest p-values). Much more often, there is no intended direction in junk science.

    The two can be merged. The recent discussion of Cass Sunstein's paper on traffic stops is an example. There, Cass chose only the pieces that flew the furthest in the desired direction and ignored pieces that flew in an undesirable directions.

  2. I wonder if both end up having similar effects on how people consume science/news? Fake news is bad not just because it pollutes the world with falsehoods, but because it can lead to a kind of learned helplessness, i.e., “there’s just so much junk out there, I might as well not care about anything.”

    Could junk science be doing the same thing? In other words, being bad not just in itself, but because it introduces so much noise that it trains members of the public and policy makers to not care about research at all? Anecdotally, I could believe this is the case, but I don’t really know.

    • Good point. Perhaps the process could be called “apathy generation”?

      PS: I didn’t intend a pun, but when I googled the phase, I got “The apathy generation” (referring to the generation called Millennials). So maybe it would be better to say that junk science is an “apathy generator”!

  3. Is all “fake news” deliberately fake? I dont know..is Fox deliberate? Or just an outrageous bias? Does an outrageous bias rise to the level of “fake”? Seems to me that “fake news” is similar to political speech in that its (at least) 2D spectrum: True-False and intentional deception to innocent misrepresentation…

    • I’ve wrestled with this question regularly. I have settled on using the word “dishonest”.

      Sleazy news outlets like Fox and CNN don’t all that often resort to outright lies. It is enough for them to exaggerate and distort points they find helpful, ignore unhelpful points, and avoid lying themselves by quoting liars they know are probably lying.

      We don’t want to limit our scorn to outright, provable lies. That is too small a set, and the boundary between the two is often fuzzy. We want to scorn all the exaggeration and selection bias, and for that I think the best word is to scorn their “dishonesty”.

      Scorning dishonesty also avoids petty bickering about “yes, that is biased, but its not an outright lie,” because something is technically accurate.

      • “We don’t want to limit our scorn to outright, provable lies”

        For sure! But I’m not sure one axis of “honesty / dishonesty” can get us there.

        In the Seattle Times yesterday we have the headline in the NW section:
        “Overdoes deaths Keep Rising in King County”

        in the article we find this data:
        “between mid-June and mid-September, 141 people overdosed and died. During the same period last year, 109 people died as a result of an overdose. Another concerning trend….”

        So, wait, we have one year over year comparison of one period, and that’s summarized as “Overdose Deaths Keep Rising”, and it’s a “concerning trend”?

        It’s true, over one period, deaths rose. That’s what actually happened. There is no “trend” at all. There is other data in the article, but every data quote is a unique variable over a unique time period (e.g., “alcohol and drug related deaths, 2017-2018”) not comparable to any other data.

        So what do we call this? I think “fake news” is a bit strong for the article. There’s no numeric discrepancy. But the title is fake. It’s an inaccurate summary of the data – what precious little there is of it.

        I can’t pull up an example right off hand but this kind of over-extension of very little data to make sweeping claims definitely happens in science, so I think it would be worth finding some way to quantify it.

        • In the above I’m not making any argument about what’s actually happening. My argument is entirely about how what is happening is being reported.

          but it’s an important argument because from the above, we *can’t tell* whether there is a trend or not. Why not? The data is missing. Why? Either the county doesn’t have the data or the reporter failed to find it. Well if we can’t tell from the data that there’s a trend, why are we reporting one?

          There must be some reason. Are we reporting one to grab readership? Because, off the record, officials believe there is one? Because even if there isn’t one there might be and we’re doing the right thing by trying to stop it?

          The whole point of both science and news is to report *what actually happened* first, then to figure out why and what it means.

        • “between mid-June and mid-September, 141 people overdosed and died. During the same period last year, 109 people died as a result of an overdose. Another concerning trend….”

          Based on a population of 2,233,163, that would be a rise of 4.9 per 100k to 6.3 per 100k, right.

          That’s about 100x more common than dying from “vaping illness”, which there is ton of recent news about.

        • I see it’s up to 12 deaths from vaping now, and 3K death per year from food-borne illness. Almost the same.

          I’m personally very impressed with the pathological anti-vaping lobby. I agree it’s politics but you know really I think a lot of these things just come down to people who want to be part of a movement and they’re trying to increase their own status within the movement by being vehement.

          Geologically, we live in the Anthropocene. Anthropologically, we live in the Activist Age.

          It’s interesting that there’s this underlying assumption in the media that all social activists are doing something good and therefore should be excused from scrutiny.

        • I see it’s up to 12 deaths from vaping now,

          Or 12 unexplained deaths in people who happened to vape in the last 90 days. Always watch the definitions, that is where most of the games are played.

  4. This discussion will be more fruitful if Andrew/others provide a specific example of what they consider to be “fake news,” ideally with a link to the source. The community of readers here probably agree on many examples of junk science. I suspect that we may not agree about what is and is not fake news.

  5. My experience has certainly made me more skeptical of scientific claims these days, even claims in fields supposedly with roots in physical science. For example, the following is latest expert assessment on links between climate changes and tropic storms. It was praised even by climate skeptics as being careful and yet I see plenty of warning signs in its liberal and imprecise dependence on NHST.

    T. Knutson, S.J. Camargo, J.C.L. Chan, K. Emanuel, C. Ho, J. Kossin, M. Mohapatra, M. Satoh, M. Sugi, K. Walsh, and L. Wu, “Tropical Cyclones and Climate Change Assessment: Part I. Detection and Attribution”, Bulletin of the American Meteorological Society, 2019. http://dx.doi.org/10.1175/BAMS-D-18-0189.1

  6. @yyw:

    … you brought up Climate-Change

    AGW/Climate-Change is a spectacular example of a massive hybrid Junk Science + Fake News.
    It now nearly qualifies as a mystic religion.

Leave a Reply

Your email address will not be published. Required fields are marked *