Bad science as genre fiction: I think there’s a lot to be said for this analogy!

Posted on by

I came across this blog comment from a couple years ago saying that, whatever was going on in the head of Brian “Pizzagate” Wansink when he wrote up those papers with the fake data, in any case his papers papers are not to be believed; they’re a sort of genre fiction.

I like this idea, not just the bad science it’s false (hence “fiction,” as in psychologist Stuart Ritchie’s recent book “Science Fictions”), but also that it’s genre fiction; that is, it’s written to a certain pattern, to fulfill certain expectations and be published in certain venues.

A p-hacked paper in Psychological Science or PNAS is less impressive than, say, a Nero Wolfe novel, but it’s a form of naturalistic genre fiction in the sense of drawing together real-world experiences in an artificial way in order to create a satisfying theme, plot, and characters. (In this case, the hero/detective is the lead author of the paper, and the Archie Goodwins are the intrepid grad students who do the heavy lifting.) To continue this analogy, hard-core fakes (such as the work of retired political science grad student Michael Lacour or those psychology papers on which Dan Ariely and Francesca Gino both deny fabricating data) are genre fiction in the tradition of The Murder of Roger Ackroyd: literature that follows the standard genre form but plays a bit with our expectations. Just as you’re usually supposed to be able to trust the narrator of a detective novel, similarly you’re supposed to trust the honesty of the authors of a scientific paper. They’re supposed to get into the fiction by working with noisy but real data and then cleverly coming up with story time. To just make up the data—that’s cheating! It would be like Poirot revealing the existence a hitherto-unknown poison in the last chapter of the book. But, sure, every once in awhile you can do this sort of trick.

Genre fiction is not always airtight. It’s typically written to get sold and make money right not, not with an eye on posterity. There can be lots of plot holes. Even Ubik, that classic novel—or, maybe, especially Ubik—has a barely coherent plot, just enough to keep the story going. An analogy in the junk science literature could be the paper climate-science provocateur by Richard “Gremlins” Tol that approached the Platonic ideal of having more errors than it had data points. In contrast, the work of gender-essentialist beauty-and-sex-ratio dude is more like a tightly-plotted series featuring a central character with certain character traits that are required to show up in every book. Maybe like “Spenser”? I don’t really know—I never read any of those books, but that’s how I recall them described in reviews of the time.

Continuing this analogy: What’s the junk-science equivalent of modern mass-produced thrillers of the James Patterson variety? I dunno, maybe just generic science journals. I’m not thinking of bottom-of-the-barrel crap such as the presumably-plagiarized “torment executioners” but rather the kind of journal article that is of zero theoretical or practical interest which can be churned out pretty much automatically by any of thousands of researchers with basic competence in this sort of thing. For example, the immortal “Participants reported being hungrier when they walked into the café (mean = 7.38, SD = 2.20) than when they walked out [mean = 1.53, SD = 2.70, F(1, 75) = 107.68, P < 0.001]" paper. OK, that one’s so ridiculous it’s almost a campy “Showgirls” of its type—to switch genres for a moment—, but you get the idea.

Finally, we can think of inside-out quasi-parodies of genre such as in the work of Paul Auster. I think the nudgelords reached that level with their declaration that, by writing a book on a topic they knew nothing about, they think they’ve “discovered a new continent.” Unlike Auster, the authors of that book seem to have been dead serious. Then again, many non-French literary critics can’t stand Auster, finding him empty and pretentious, so maybe this isn’t such a bad analogy.

Ahhh, the mention of “pretentious” reminds me of one of my favorite stories!

OK, I guess that’s enough for now. I’ve written about 8 blog posts today and done close to zero real work. Time to put this one in the queue and it will appear in a few months.

I’ll just conclude by pointing to this post from a few years ago expressing my annoyance at what might be called “middlebrow” science reporting, the kind of well-meaning NPR sort of thing where they speak in the tones of real journalists but are actually just uncritically doing the equivalent of retyping press releases. In contrast, there’s a lot of middlebrow or mid-list science that I like, including many of my own papers, which aren’t always bursts of originality; sometimes they’re just the scientific equivalent of “the well-made play” in that they do a job well. Without fiction, though, in my case (this paper aside).

Which reminds me that I still want to do some more of those movie and play ideas, following up on “Second Chance U”, “The New Dirty Dozen”, Don’t Call Me Shirley, Mr. Feynman!, and Philip’s movie idea that he shared with us in the Playroom the other day.

14 thoughts on “Bad science as genre fiction: I think there’s a lot to be said for this analogy!”

  1. The Spenser novels (by Robert B Parker) are pretty workmanlike mystery genre fiction. I hate to analogize Kanazawa’s birth sex ratio stuff to that. Parker may not be Hammett or Chandler, but he deserves better than that.

    In some sense, these scientific works are unlike any of the genre fiction. The one thing you think about in genre fiction is that it’s fiction. The one thing you’re supposed to think about scientific papers is that they’re full of fact. It’s almost more like someone passing a true crime story off as fiction where they didn’t have to write any plot or imaginative stuff at all because they could just copy real life. reversal of expectations.

    Reply ↓

    • Daniel:

      I think Kanazawa’s papers on sex ratios are very comparable to Parker’s Spenser novels. In both cases, a form is followed in a way that is acceptable to two audiences. For Parker, the audiences were book publishers and general readers. For Kanazawa, the audiences were journal editors and the press. Both Parker and Kanazawa had success repeating a formula, but with enough variation each time to keep the audiences happy.

      I agree there are differences. I consider Kanazawa’s actions—presenting statistical noise as enduring scientific truth—to be somewhere between incompetent and dishonest. Parker, on the other hand, was honestly telling stories.

      Reply ↓

      • I was confused because I thought that perhaps “The Spenser novels (by Robert B Parker)” was some strange spoonerism of the Parker novels by Richard Stark, also genre fiction! But apparently not. In any case, having finished the last of the original set of 16 this year, I can conclude that the median Stark/Parker novel is a greater contribution to human civilization than the median contemporary scientific article; I’ll try to justify this later.

        Reply ↓

        • Now, if you want a good comparison to Kanazawa, you could do worse than the “Chet and Bernie mysteries” by Spencer Quinn (another Spencer!). Here we have extremely, laughably formulaic stories about a desert dwelling ex army guy told from the slightly confused but incredibly loyal perspective of his sidekick dropout K9 dog. Complete with catchphrases, references to ridiculous business ventures such as a major investment in Hawaiian Pants, and a cast of seedy characters, such as the guy who runs a restaurant in the front and brothel in the back. They’re enjoyable as heck if you’re a dog lover for their portrayal of the mildly airheaded but nevertheless serious and faithful police dog’s internal monologue. Nevertheless, no-one would call them anything but campy schlock.

  2. Great analogy. Reminds me of how weirded out I got recently when I heard an AI-generated podcast from Google’s NotebookLM summarizing a paper I collaborated on. The cringe-worthy stylization of the facts to entertain is something LLMs can do quite well.

    I am still holding out for a book of fake movie scripts…

    Reply ↓

  3. > the presumably-plagiarized “torment executioners”

    Maybe it came up on the blog and I missed it, but personally in the meantime I came across the work of Guillaume Cabanac https://www.irit.fr/~Guillaume.Cabanac/problematic-paper-screener who’s been tracking this kind of problems, and made me aware of some of the tools that could have been used to produce these such as https://spinbot.com/

    Here’s an example with this blog post:
    > I like this idea, not just the bad science it’s false (hence “fiction,” as in psychologist Stuart Ritchie’s recent book “Science Fictions”), but also that it’s genre fiction; that is, it’s written to a certain pattern, to fulfill certain expectations and be published in certain venues.

    Which becomes:
    > I like this thought, in addition to the terrible science it’s bogus (subsequently “fiction,” as in clinician Stuart Ritchie’s new book “Science Fictions”), yet in addition that it’s kind fiction; that is, it’s kept in touch with a specific example, to satisfy specific assumptions and be distributed in specific scenes.

    A bit off-topic from the post but probably of interest to some readers here.

    Reply ↓

    • Jarrod:

      Regarding storytelling: Thomas Basbøll and I have argued that stories are important not just for communicating science to others but also in doing good science.

      My perspective here is:

      1. Fictional stories are an informal sort of predictive model checking, in which the process of working out a story is a way of understanding the implications of a scenario.

      2. Nonfiction stories are a sort of prior predictive check, where the unexpected twist that makes the story so satisfying represents a revelation of some aspect of reality that confounds our expectations, that is, does not fit with our implicit or explicit model of the world.

      Reply ↓

Leave a Reply

Your email address will not be published. Required fields are marked *