Skip to content
Archive of posts filed under the Zombies category.

Did that “bottomless soup bowl” experiment ever happen?

I’m trying to figure out if Brian “Pizzagate” Wansink’s famous “bottomless soup bowl” experiment really happened. Way back when, everybody thought the experiment was real. After all, it was described in a peer-reviewed journal article. Here’s my friend Seth Roberts in 2006: An experiment in which people eat soup from a bottomless bowl? Classic! Or […]

Replication police methodological terrorism stasi nudge shoot the messenger wtf

Cute quote: (The link comes from Stuart Richie.) Sunstein later clarified: I’ll take Sunstein’s word that he no longer thinks it’s funny to attack people who work for open science and say that they’re just like people who spread disinformation. I have no idea what Sunstein thinks the “grain of truth” is, but I guess […]

As always, I think the best solution is not for researchers to just report on some preregistered claim, but rather for them to display the entire multiverse of possible relevant results.

I happened to receive these two emails in the same day. Russ Lyons pointed to this news article by Jocelyn Kaiser, “Major medical journals don’t follow their own rules for reporting results from clinical trials,” and Kevin Lewis pointed to this research article by Kevin Murphy and Herman Aguinis, “HARKing: How Badly Can Cherry-Picking and […]

This one goes in the Zombies category, for sure.

Paul Alper writes: I was in my local library and I came across this in Saturday’s WSJ: The Math Behind Successful Relationships Nearly 30 years ago, a mathematician and a psychologist teamed up to explore one of life’s enduring mysteries: What makes some marriages happy and some miserable? The psychologist, John Gottman, wanted to craft […]

The garden of forking paths

Bert Gunter points us to this editorial: So, researchers using these data to answer questions about the effects of technology [screen time on adolescents] need to make several decisions. Depending on the complexity of the data set, variables can be statistically analysed in trillions of ways. This makes almost any pattern of results possible. As […]

Plaig!

Tom Scocca discusses some plagiarism that was done by a former New York Times editor: There was no ambiguity about it; Abramson clearly and obviously committed textbook plagiarism. Her text lifted whole sentences from other sources word for word, or with light revisions, presenting the same facts laid out in the same order as in […]

“Guarantee” is another word for “assumption”

I always think it’s funny when people go around saying their statistical methods have some sort of “guaranteed” performance. I mean, sure, guarantees are fine—but a guarantee comes from an assumption. If you want to say that your method has a guarantee but my method doesn’t, what you’re really saying is that you’re making an […]

What’s published in the journal isn’t what the researchers actually did.

David Allison points us to these two letters: Alternating Assignment was Incorrectly Labeled as Randomization, by Bridget Hannon, J. Michael Oakes, and David Allison, in the Journal of Alzheimer’s Disease. Change in study randomization allocation needs to be included in statistical analysis: comment on ‘Randomized controlled trial of weight loss versus usual care on telomere […]

What does it take to repeat them?

Olimpiu Urcan writes: Making mistakes is human, but it takes a superhuman dose of ego and ignorance to repeat them after you’ve been publicly admonished about them. Not superhuman at all, unfortunately. We see it all the time. All. The. Time. I’m reminded of the very first time I contacted newspaper columnist David Brooks to […]

Endless citations to already-retracted articles

Ken Cor and Gaurav Sood write: Many claims in a scientific article rest on research done by others. But when the claims are based on flawed research, scientific articles potentially spread misinformation. To shed light on how often scientists base their claims on problematic research, we exploit data on cases where problems with research are […]

Gigerenzer: “The Bias Bias in Behavioral Economics,” including discussion of political implications

Gerd Gigerenzer writes: Behavioral economics began with the intention of eliminating the psychological blind spot in rational choice theory and ended up portraying psychology as the study of irrationality. In its portrayal, people have systematic cognitive biases that are not only as persistent as visual illusions but also costly in real life—meaning that governmental paternalism […]

Racism is a framework, not a theory

Awhile ago we had a discussion about racism, in the context of a review of a recent book by science reporter Nicholas Wade that attributed all sorts of social changes and differences between societies to genetics. There is no point in repeating all this, but I did want to bring up here an issue that […]

We should be open-minded, but not selectively open-minded.

I wrote this post awhile ago but it just appeared . . . I liked this line so much I’m posting it on its own: We should be open-minded, but not selectively open-minded. This is related to the research incumbency effect and all sorts of other things we’ve talked about over the years. There’s a […]

And, if we really want to get real, let’s be open to the possibility that the effect is positive for some people in some scenarios, and negative for other people in other scenarios, and that in the existing state of our knowledge, we can’t say much about where the effect is positive and where it is negative.

Javier Benitez points us to this op-ed, “Massaging data to fit a theory is not the worst research sin,” where philosopher Martin Cohen writes: The recent fall from grace of the Cornell University food marketing researcher Brian Wansink is very revealing of the state of play in modern research. Wansink had for years embodied the […]

A supposedly fun thing I definitely won’t be repeating (A Pride post)

“My friends and I don’t wanna be here if this isn’t an actively trans-affirming space. I’m only coming if all my sisters can.” – I have no music for you today, sorry. But I do have an article about cruise ships  (This is obviously not Andrew) A Sunday night quickie post, from the tired side […]

How to think about reported life hacks?

Interesting juxtaposition as two interesting pieces of spam happened to appear in my inbox on the same day: 1. Subject line “Why the power stance will be your go-to move in 2019”: The power stance has been highlighted as one way to show your dominance at work and move through the ranks. While moving up […]

Freud expert also a Korea expert

I received the following email: Dear Dr Andrew Gelman, I am writing to you on behalf of **. I hereby took this opportunity to humbly request you to consider being a guest speaker on our morning radio show, on 6th August, between 8.30-9.00 am (BST) to discuss North Korea working on new missiles We would […]

“The writer who confesses that he is ‘not good at attention to detail’ is like a pianist who admits to being tone deaf”

Edward Winter wrote: It is extraordinary how the unschooled manage to reduce complex issues to facile certainties. The writer who confesses that he is ‘not good at attention to detail’ (see page 17 of the November 1990 CHESS for that stark, though redundant, admission by the Weekend Wordspinner) is like a pianist who admits to […]

Harvard dude calls us “online trolls”

Story here. Background here (“How post-hoc power calculation is like a shit sandwich”) and here (“Post-Hoc Power PubPeer Dumpster Fire”). OK, to be fair, “shit sandwich” could be considered kind of a trollish thing for me to have said. But the potty language in this context was not gratuitous; it furthered the larger point I […]

I agree it’s a problem but it doesn’t surprise me. It’s pretty random what these tabloids publish, as they get so many submissions.

Jeff Lax writes: I’m probably not the only one telling you about this Science story, but just in case. The link points to a new research article reporting a failed replication of a study from 2008. The journal that published that now-questionable result refuses to consider publishing the replication attempt. My reply: I agree it’s […]