Skip to content
Archive of posts filed under the Zombies category.

The flashy crooks get the headlines, but the bigger problem is everyday routine bad science done by non-crooks

In the immortal words of Michael Kinsley, the real scandal isn’t what’s illegal, the scandal is what’s legal. I was reminded of this principle after seeing this news article about the discredited Surgisphere doctor (see here for background). The news article was fine—it’s good to learn these things—but, as with pizzagate, evilicious, and other science […]

BMJ FAIL: The system is broken. (Some reflections on bad research, scientism, the importance of description, and the challenge of negativity)

tl;dr It’s not the British Medical Journal’s “fault” that they published a bad paper. I mean, sure, yeah, it’s 100% their fault, but you can’t fault a journal for publishing the occasional dud. And there’s not really a mechanism for retracting a paper that’s just seriously flawed, if no fraud is suspected. So the system […]

Recently in the sister blog

Generic language in scientific communication: There is increasing recognition that research samples in psychology are limited in size, diversity, and generalizability. However, because scientists are encouraged to reach broad audiences, we hypothesized that scientific writing may sacrifice precision in favor of bolder claims. We focused on generic statements (“Introverts and extraverts require different learning environments”), […]

Male bisexuality gets Big PNAS Energy

Do flowers exist at night?—John Mulaney and the Sack Lunch Bunch I have very little to say here, except to let you all know that the venerable PNAS has today published a paper (edited by Steven Pinker) letting use know that male bisexuality exists. Here it is: Robust evidence for bisexual orientation among men (The paper […]

Can the science community help journalists avoid science hype? It won’t be easy.

tl;dr: Selection bias. The public letter Michael Eisen and Rob Tibshirani write: Researchers have responded to the challenge of the coronavirus with a commitment to speed and cooperation, featuring the rapid sharing of preliminary findings through “preprints,” scientific manuscripts that have not yet undergone formal peer review. . . . But the open dissemination of […]

“Sorry, there is no peer review to display for this article.” Huh? Whassup, BMJ?

OK, this is weird. Yesterday we reported on an article with questionable statistical analysis published in the British Medical Journal. This one’s different from some other examples we’ve discussed recently (Surgisphere and Stanford) in that the author list of this recent article includes several statisticians. One way to get a handle on this situation is […]

Please socially distance me from this regression model!

A biostatistician writes: The BMJ just published a paper using regression discontinuity to estimate the effect of social distancing. But they have terrible models. As I am from Canada, I had particular interest in the model for Canada, which is on their supplemental material, page 84 [reproduced above]. I could not believe this was published. […]

Association Between Universal Curve Fitting in a Health Care Journal and Journal Acceptance Among Health Care Researchers

Matt Folz points us to this recent JAMA article that features this amazing graph: Beautiful. Just beautiful. I say this ironically.

What does it take to be omniscient?

Palko points us to this comment from Josh Marshall: To put it baldly, if it’s a topic and area of study you know nothing about and after a few weeks of cramming you decide that basically everyone who’s studied the question is wrong, there’s a very small chance you’ve rapidly come upon a great insight […]

If variation in effects is so damn important and so damn obvious, why do we hear so little about it?

Earlier today we posted, “To Change the World, Behavioral Intervention Research Will Need to Get Serious About Heterogeneity,” and commenters correctly noted that this point applies not just in behavioral research but also in economics, public health, and other areas. I wanted to follow this up with a question: If variation in effects is so […]

Blog about a column about the Harper’s letter: Here’s some discourse about a discourse about what happens when the discourse takes precedence over reality

I read this op-ed by Tom Scocca and I have some thoughts. To start with, as the above title indicates, the topic is very “meta.” Scocca’s article is a response to an open letter which is a response to criticisms of other people’s negative responses to other people’s criticisms. As a statistician, I can relate […]

New England Journal of Medicine engages in typical academic corporate ass-covering behavior

James Watson (not the racist dude who, in 1998, said that a cancer cure was coming in 2 years) writes: About a month ago, when the infamous Lancet hydroxychloroquine/chloroquine paper was still “real science” (i.e. in the official scientific record), we decided to put extra pressure on the authors by writing an open letter to […]

No, I don’t believe that claim based on regression discontinuity analysis that . . .

tl;dr. See point 4 below. Despite the p-less-than-0.05 statistical significance of the discontinuity in the above graph, no, I do not believe that losing a close election causes U.S. governors to die 5-10 years longer, as was claimed in this recently published article. Or, to put it another way: Despite the p-less-than-0.05 statistical significance of […]

Hey, this was an unusual media request

This popped up in the inbox: Hi Professor Gelman – my name is ** and I’m a journalist who reports on issues of ** conducted by **. Recently ** announced the department was working with “research groups” to study and analyze the **. To further my reporting on this issue, I am reaching out to […]

No, there is no “tension between getting it fast and getting it right”

When reading Retraction Watch, I came across this quote: “There is always a tension between getting it fast and getting it right,” said Dr. Marcia Angell, another former editor in chief of the New England Journal of Medicine. “I always favored getting it right. But in the current pandemic, that balance may have shifted too […]

Retraction of racial essentialist article that appeared in Psychological Science

Scene 1: It all started for me on 2 Jan when I received this email from Keith Donohue in Fargo, North Dakota: I am a longtime reader, and I am curious about your reaction to an (in press) journal article that I recently came across. . . . The paper is “Declines in Religiosity Predicted […]

Against overly restrictive definitions: No, I don’t think it helps to describe Bayes as “the analysis of subjective
 beliefs” (nor, for that matter, does it help to characterize the statements of Krugman or Mankiw as not being “economics”)

I get frustrated when people use overly restrictive definitions of something they don’t like. Here’s an example of an overly restrictive definition that got me thinking about all this. Larry Wasserman writes (as reported by Deborah Mayo): I wish people were clearer about what Bayes is/is not and what 
frequentist inference is/is not. Bayes is […]

The point here is not the face masks; it’s the impossibility of assumption-free causal inference when the different treatments are entangled in this way.

Adam Pearce writers: When I read your Another Regression Discontinuity Disaster post last year, I was curious how much shifting the breakpoint would change the fit lines. A covid paper making the rounds this weekend used a similar technique so I hooked it up to an interactive widget that lets you tweak the start and […]

“Worthwhile content in PNAS”

Ben Bolker sends an email with the above subject line, a link to this article, and the following content: Experimental evidence that hummingbirds can see purple … researchers used Stan to analyze the data … The article in question is called “Wild hummingbirds discriminate nonspectral colors” and is by Mary Caswell Stoddard, Harold Eyster, Benedict […]

Surgisphere scandal: Lancet still doesn’t get it

So. I opened the newspaper today and saw this article by Roni Caryn Rabin, “Two Retractions Hurt Credibility of Peer Review.” It was about the Surgisphere scandal, which we’ve discussed a few times in this space, going from Doubts about that article claiming that hydroxychloroquine/chloroquine is killing people to How should those Lancet/Surgisphere/Harvard data have […]