Skip to content
 

“The tabloids” and the tabloids: why Nature ran that John Edwards story

The journals Science and Nature are called “the tabloids.” As psychologist Sanjay Srivastava writes:

As long as a journal pursues a strategy of publishing “wow” studies, it will inevitably contain more unreplicable findings and unsupportable conclusions than equally rigorous but more ‘boring’ journals. Groundbreaking will always be higher-risk. And definitive will be the territory of journals that publish meta-analyses and reviews.

Thus: what makes tabloids tabloids is that they regularly publish results that are (a) exciting and (b) false.

“Exciting” and “false” go together: one thing that makes a claim exciting is that it’s surprising, i.e. contradicts what was previously known, i.e. likely to be false.

Tabloids do, however, publish truth every now and then, and this makes sense too. There’d be little reason to read a journal if they only publish errors. To move from the scientific to the political realm, recall that the National Enquirer may have been wrong about some of their UFO stories but they were dead-on right about John Edwards.

P.S. More here by blogger Orac on the idea that cutting-edge research is often false. (You can skip all the rude stuff at the beginning of the linked post to get to the relevant discussion.) Orac puts a positive spin on this, arguing that of course speculative research is typically wrong but that’s ok because speculation is the first step preceding evaluation. In that sense, the problem is not with the tabloids publishing bad stuff, the problem is with with journalists who think that being published in a top journal is some sort of indication of correctness.

2 Comments

  1. dmk38 says:

    This is one of the exam questions, right?

    The article that is the subject of the post by Orac *is* a Nature article that reports the “shocking” finding that 75% of the positive results from “pre-clinical cancer treatment trials” cannot be replicated. Given how often top journals like Nature are incorrect, estimate the likelihood that a a study reporting an “exciting” breakthrough and published in an established medical journal can in fact be replicated. For extra credit, estimate the likelihood (and standard error, and 0.95 CI) that the Nature study on nonreplication will prove to be nonreplicable.

  2. […] reminded me of Andrew Gelman’s recent post about results in headline academic journals such as Science and Nature. He quotes Sanjay Srivastava […]