Skip to content
Archive of posts filed under the Zombies category.

Data quality is a thing.

I just happened to come across this story, where a journalist took some garbled data and spun a false tale which then got spread without question. It’s a problem. First, it’s a problem that people will repeat unjustified claims, also a problem that when data are attached, you can get complete credulity, even for claims […]

Horse-and-buggy era officially ends for survey research

Peter Enns writes: Given the various comments on your blog about evolving survey methods (e.g., Of buggy whips and moral hazards; or, Sympathy for the Aapor), I thought you might be interested that the Roper Center has updated its acquisitions policy and is now accepting non-probability samples and other methods. This is an exciting move […]

Name this fallacy!

It’s the fallacy of thinking that, just cos you’re good at something, that everyone should be good at it, and if they’re not, they’re just being stubborn and doing it badly on purpose. I thought about this when reading this line from Adam Gopnik in the New Yorker: [Henry Louis] Gates is one of the […]

Gremlin time: “distant future, faraway lands, and remote probabilities”

Chris Wilson writes: It appears that Richard Tol is still publishing these data, only now fitting a piecewise linear function to the same data-points. https://academic.oup.com/reep/article/12/1/4/4804315#110883819 Also still looks like counting 0 as positive, “Moreover, the 11 estimates for warming of 2.5°C indicate that researchers disagree on the sign of the net impact: 3 estimates are […]

Post-Hoc Power PubPeer Dumpster Fire

We’ve discussed this one before (original, polite response here; later response, after months of frustration, here), but it keeps on coming. Latest version is this disaster of a paper which got shredded by a zillion commenters on PubPeer. There’s lots of incompetent stuff out there in the literature—that’s the way things go; statistics is hard—but, […]

“Appendix: Why we are publishing this here instead of as a letter to the editor in the journal”

David Allison points us to this letter he wrote with Cynthia Kroeger and Andrew Brown: Unsubstantiated conclusions in randomized controlled trial of binge eating program due to Differences in Nominal Significance (DINS) Error Cachelin et al. tested the effects of a culturally adapted, Cognitive Behavioral Therapy-based, guided self-help (CBTgsh) intervention on binge eating reduction . […]

Why “statistical significance” doesn’t work: An example.

Reading some of the back-and-forth in this thread, it struck me that some of the discussion was about data, some was about models, some was about underlying reality, but none of the discussion was driven by statements that this or that pattern in data was “statistically significant.” Here’s the problem with “statistical significance” as I […]

Claims about excess road deaths on “4/20” don’t add up

Sam Harper writes: Since you’ve written about similar papers (that recent NRA study in NEJM, the birthday analysis) before and we linked to a few of your posts, I thought you might be interested in this recent blog post we wrote about a similar kind of study claiming that fatal motor vehicle crashes increase by 12% after 4:20pm […]

Abandoning statistical significance is both sensible and practical

Valentin Amrhein​, Sander Greenland, Blakeley McShane, and I write: Dr Ioannidis writes against our proposals [here and here] to abandon statistical significance in scientific reasoning and publication, as endorsed in the editorial of a recent special issue of an American Statistical Association journal devoted to moving to a “post p

Prestigious journal publishes sexy selfie study

Stephen Oliver writes: Not really worth blogging about and a likely candidate for multiverse analysis, but the beginning of the first sentence in the 2nd paragraph made me laugh: In the study – published in prestigious journal PNAS . . . The researchers get extra points for this quote from the press release: The researchers […]

Thinking about “Abandon statistical significance,” p-values, etc.

We had some good discussion the other day following up on the article, “Retire Statistical Significance,” by Valentin Amrhein, Sander Greenland, and Blake McShane. I have a lot to say, and it’s hard to put it all together, in part because my collaborators and I have said much of it already, in various forms. For […]

Another bit from Art Owen, this time dunking on ripoff publishers

From Owen’s review of Mayo’s book: Going through this put me in mind of Jim Zidek’s early 1980s work on multi-Bayesian theory. The most cited paper there is his JRSS-A paper with Weerahandri from 1981. From the abstract it looks more like it addresses formation of a consensus posterior or decision choice and is not […]

A comment about p-values from Art Owen, upon reading Deborah Mayo’s new book

The Stanford statistician writes: One of the fun parts of this was reading some of what Meehl wrote. I’d seen him quoted but had not read him before. What he says reminds me a lot of how p values were presented when I was an undergraduate at Waterloo. They emphasized large p values as a […]

Surgeon promotes fraudulent research that kills people; his employer, a leading hospital, defends him and attacks whistleblowers. Business as usual.

Paul Alper writes: A couple of time at my suggestion, you’ve blogged about Paulo Macchiarini. Here is an update from Susan Perry in which she interviews the director of the Swedish documentary about Macchiarini: Indeed, Macchiarini made it sound as if his patients had recovered their health when, in fact, the synthetic tracheas he had […]

“Retire Statistical Significance”: The discussion.

So, the paper by Valentin Amrhein, Sander Greenland, and Blake McShane that we discussed a few weeks ago has just appeared online as a comment piece in Nature, along with a letter with hundreds (or is it thousands?) of supporting signatures. Following the first circulation of that article, the authors of that article and some […]

Statistical-significance thinking is not just a bad way to publish, it’s also a bad way to think

Eric Loken writes: The table below was on your blog a few days ago, with the clear point about p-values (and even worse the significance versus non-significance) being a poor summary of data. The thought I’ve had lately, working with various groups of really smart and thoughtful researchers, is that Table 4 is also a […]

One more reason I hate letters of recommendation

Recently I reviewed a bunch of good reasons to remove letters of recommendation when evaluating candidates for jobs or scholarships. Today I was at a meeting and thought of one more issue. Letters of recommendation are not merely a noisy communication channel; they’re also a biased channel. The problem is that letter writers are strategic: […]

Raghuram Rajan: “The Third Pillar: How Markets and the State Leave the Community Behind”

A few months ago I receive a copy of the book, “The Third Pillar: How Markets and the State Leave the Community Behind,” by economist Raghuram Rajan. The topic is important and the book is full of interesting thoughts. It’s hard for me to evaluate Rajan’s economics and policy advice, so I’ll leave that to […]

The neurostatistical precursors of noise-magnifying statistical procedures in infancy

David Allison points us to this paper, The neurodevelopmental precursors of altruistic behavior in infancy, by Tobias Grossmann, Manuela Missana, and Kathleen Krol, which states: The tendency to engage in altruistic behavior varies between individuals and has been linked to differences in responding to fearful faces. The current study tests the hypothesis that this link […]

“Abandon / Retire Statistical Significance”: Your chance to sign a petition!

Valentin Amrhein, Sander Greenland, and Blake McShane write: We have a forthcoming comment in Nature arguing that it is time to abandon statistical significance. The comment serves to introduce a new special issue of The American Statistician on “Statistical inference in the 21st century: A world beyond P < 0.05”. It is titled "Retire Statistical […]