Skip to content
Archive of posts filed under the Zombies category.

Statistical-significance thinking is not just a bad way to publish, it’s also a bad way to think

Eric Loken writes: The table below was on your blog a few days ago, with the clear point about p-values (and even worse the significance versus non-significance) being a poor summary of data. The thought I’ve had lately, working with various groups of really smart and thoughtful researchers, is that Table 4 is also a […]

One more reason I hate letters of recommendation

Recently I reviewed a bunch of good reasons to remove letters of recommendation when evaluating candidates for jobs or scholarships. Today I was at a meeting and thought of one more issue. Letters of recommendation are not merely a noisy communication channel; they’re also a biased channel. The problem is that letter writers are strategic: […]

Raghuram Rajan: “The Third Pillar: How Markets and the State Leave the Community Behind”

A few months ago I receive a copy of the book, “The Third Pillar: How Markets and the State Leave the Community Behind,” by economist Raghuram Rajan. The topic is important and the book is full of interesting thoughts. It’s hard for me to evaluate Rajan’s economics and policy advice, so I’ll leave that to […]

The neurostatistical precursors of noise-magnifying statistical procedures in infancy

David Allison points us to this paper, The neurodevelopmental precursors of altruistic behavior in infancy, by Tobias Grossmann, Manuela Missana, and Kathleen Krol, which states: The tendency to engage in altruistic behavior varies between individuals and has been linked to differences in responding to fearful faces. The current study tests the hypothesis that this link […]

“Abandon / Retire Statistical Significance”: Your chance to sign a petition!

Valentin Amrhein, Sander Greenland, and Blake McShane write: We have a forthcoming comment in Nature arguing that it is time to abandon statistical significance. The comment serves to introduce a new special issue of The American Statistician on “Statistical inference in the 21st century: A world beyond P < 0.05”. It is titled "Retire Statistical […]

Journalist seeking scoops is as bad as scientist doing unreplicable research

Tom Scocca shares this dispiriting story: Yesterday, as a news day, was an even worse cascade of lies and confusion and gibberish than usual. Yet what stood out the most was a single word: “Clarification.” It appeared at the bottom of a very short Axios post by reporter Jonathan Swan, introducing a note that read, […]

“Yes, not only am I suspicious of the claims in that op-ed, I’m also suspicious of all the individual claims from the links in these two sentences”

Someone pointed me with suspicion to a newspaper article that reported a cool-looking social science result, and asked me for my thoughts. I replied, Yes, not only am I suspicious of the claims in that article, I’m also suspicious of all the individual claims from these links. And I pointed to a bunch of links […]

George Orwell meets statistical significance: “Politics and the English Language” applied to science

1. Political writing: imprecision as a tool for obscuring the indefensible In his classic essay, “Politics and the English Language,” the political journalist George Orwell drew a connection between cloudy writing and cloudy content. The basic idea was: if you don’t know what you’re saying, or if you’re trying to say something you don’t really […]

Statistical-significance filtering is a noise amplifier.

The above phrase just came up, and I think it’s important enough to deserve its own post. Well-meaning researchers do statistical-significance filtering all the time—it’s what they’re trained to do, it’s what they see in published papers in top journals, it’s what reviewers for journals want them to do—so I can understand why they do […]

Evidence distortion in clinical trials

After seeing our recent post, “Seeding trials”: medical marketing disguised as science, Till Bruckner sent me this message: I’ve been working on clinical trial transparency issues for over two years now, first for AllTrials and now for TranspariMED, and can assure you that this is only the tip of the iceberg. This report by Transparency […]

Don’t worry, the post will be coming . . . eventually

Jordan Anaya sends along a link and writes: Not sure if you’re planning on covering this, but I noticed this today. This could also maybe be another example of the bullshit asymmetry principle since the original paper has an altmetric of 1300 and I’m not sure the rebuttal will get as much attention. I replied […]

“News Release from the JAMA Network”

A couple people pointed me to this: Here’s the Notice of Retraction: On May 8, 2018, notices of Expression of Concern were published regarding articles published in JAMA and the JAMA Network journals that included Brian Wansink, PhD, as author. At that time, Cornell University was contacted and was requested to conduct an independent evaluation […]

Statmodeling Retro

As many of you know, this blog auto-posts on twitter. That’s cool. But we also have 15 years of old posts with lots of interesting content and discussion! So I had this idea of setting up another twitter feed, Statmodeling Retro, that would start with our very first post in 2004 and then go forward, […]

More on that horrible statistical significance grid

Regarding this horrible Table 4: Eric Loken writes: The clear point or your post was that p-values (and even worse the significance versus non-significance) are a poor summary of data. The thought I’ve had lately, working with various groups of really smart and thoughtful researchers, is that Table 4 is also a model of their […]

The bullshit asymmetry principle

Jordan Anaya writes, “We talk about this concept a lot, I didn’t realize there was a name for it.” From the wikipedia entry: Publicly formulated the first time in January 2013 by Alberto Brandolini, an Italian programmer, the bullshit asymmetry principle (also known as Brandolini’s law) states that: The amount of energy needed to refute […]

What should JPSP have done with Bem’s ESP paper, back in 2010? Click to find the surprisingly simple answer!

OK, you all remember the story, arguably the single event that sent the replication crisis into high gear: the decision of the Journal of Personality and Social Psychology to publish a paper on extra-sensory perception (ESP) by Cornell professor Daryl Bem and the subsequent discussion of this research in the New York Times and elsewhere. […]

When doing regression (or matching, or weighting, or whatever), don’t say “control for,” say “adjust for”

This comes up from time to time. We were discussing a published statistical blunder, an innumerate overconfident claim arising from blind faith that a crude regression analysis would control for various differences between groups. Martha made the following useful comment: Another factor that I [Martha] believe tends to promote the kind of thing we’re talking […]

Just when you thought it was safe to go back into the water . . . SHARK ATTACKS in the Journal of Politics

We’ve been here before. Back in 2002, political scientists Chris Achen and Larry Bartels presented a paper “Blind Retrospection – Electoral Responses to Drought, Flu and Shark Attacks.” Here’s a 2012 version in which the authors trace “the electoral impact of a clearly random event—a dramatic series of shark attacks in New Jersey in 1916” […]

The butterfly effect: It’s not what you think it is.

John Cook writes: The butterfly effect is the semi-serious claim that a butterfly flapping its wings can cause a tornado half way around the world. It’s a poetic way of saying that some systems show sensitive dependence on initial conditions, that the slightest change now can make an enormous difference later . . . Once […]

A ladder of responses to criticism, from the most responsible to the most destructive

In a recent discussion thread, I mentioned how I’m feeling charitable toward David Brooks, Michael Barone, and various others whose work I’ve criticized over the years, because their responses have been so civilized and moderate. Consider the following range of responses to an outsider pointing out an error in your published work: 1. Look into […]