In light of our recent discussion of the unwillingness of authors of published papers to acknowledge their mistakes, I thought I’d post the four correction notices that I’ve felt the need to issue.
From 1999, correcting a paper that appeared in 1993:
From 2013, correcting a paper that appeared in 2008:
From 2017, correcting a paper that appeared in 2006:
Finally, there was this note from 2014 correcting a paper from 1996 that had been published in a volume, not a journal. There was no way to issue a formal correction notice for an already-published book, so I posted the correction on the blog. It’s an interesting story—in short, in the paper we said we’d done something that we’d never done, and we made a false statement. In the correction notice, I confirmed with a simple simulation in R that we’d indeed made a mistake.
Discussion
I like how the correction notices are so crisp and un-hedging. We got it wrong, that’s the story!
The only thing I regret is that about these notices is that I don’t explore how the mistakes happened. If I could do it all over, I’d add a sentence to each, on what aspect of our workflow failed so as to enable the error.
Here goes, to be appended to the published correction notices above:
1999: It is not clear to us in general how to avoid this sort of false proof, the problem being that the false statement seemed so natural to us that we did not think to look at it carefully.
2013: It seems likely that we could have avoided this error with a better data-analysis workflow making graphs of the correlations between all pairs of variables, along with a computational workflow that would have made it easier to correct the problem.
2017: We might have caught this problem had the paper included a fully worked example with code.
Not perfect, but you get the idea.
Looks good. Sadly, I think (despite how motherhood and apple pie this type of attitude should be in science) that this is rare to have this clear of an attitude. I suspect the careerization of Big Science and Big Academia has a lot to do with this.
Although I have no proof to that effect. And certainly man has been subject to failings since Adam and Eve chomped. So I may be wrong about things having been more honest pre WW2.
This reminds me of work being done to use set standards for retraction notices. See CREC: https://www.niso.org/standards-committees/crec
Applies to corrections to, with a little thought.
Any kind of scholarly correction/notice could use some deep thinking about what specifically to report and how. This article is a great starting point for some kind of consensus-based development of reporting guideline.
A good place for additions like these would be the pubpeer page for the article in question. It’s a good central clearing house for short comments, and more discoverable than a blog post. Also quite a boss move compared to the way most authors behave on pubpeer.
I’ve been thinking about how poorly the academic world adapted to all the wonderful technologies that the internet has brought. This post is a nice example: a pretty wonderful set of honest, direct corrections, and ideas from the author to improve them still.
And yet, journals still stick these corrections as an awkward separate article, a disconnected addendum. The modern web allows for strikethrough text, links, “Last updated on”, and a long et cetera.
My kingdom for a journal that would dare to publish online like it’s 1998.
Posting three corrections when you said you’d post four is very clever.
Anon:
As I wrote above, the fourth one was for an article in an already-published book; there here was no way to issue a formal correction notice, so I posted the correction on the blog and linked to it here.
The first paper subject to a corrigendum is three pages long, consisting of the following sections: 1. Introduction; 2. The Problem; 3. Theorem; 4. Proof. If the theorem’s wrong and the proof’s wrong, what’s left? Wouldn’t this merit a retraction? Or was retraction just not something one considered 25 years ago?
John:
As I wrote a few years ago, I prefer corrections to retractions. Our correction notice is clear and direct. A retraction would be fine too, but I prefer the precision of a correction, where I can say exactly what went wrong. That paper was a fun example because it was so easy to specify the counterexample!
I asked ChatGPT.com the following question: “How many published papers has Andrew Gelman retracted?” It gave me two simultaneous responses.
————————————————————————————————————–
1. “As of November 24, 2024, there is no publicly available information indicating that Andrew Gelman has had any of his own published papers retracted. Gelman, a professor of statistics and political science at Columbia University, is known for his critical analyses of statistical methods and research practices. He has authored numerous papers and books, but none have been retracted to date.”
2. “Andrew Gelman, a professor of statistics and political science, is known for his critical commentary on research practices but does not appear to have retracted any of his own papers. Discussions on his blog frequently address retractions and corrections in the academic community, but none explicitly involve retractions of his work
Statistical Modeling”
Statistical Modeling
.
If you are looking for specifics about retractions in the field of statistics or cases he has discussed, let me know, and I can assist further!”
————————————————————————————
I did not press the issue further by pointing out that there were retractions mentioned this very day by a contributor to Andrew Gelman’s blog. One can lose faith in AI when such things happen.
In my experience, these llms don’t search the web in real time for such information. It’s always going to be hit or miss with these kinds of questions.
I may need to correct what I just wrote. I used the word “retractions” instead of “corrections.” So, I then asked
“Has Andrew Gelman ever made corrections to his published papers?”
and received this reply
“Yes, Andrew Gelman, a prominent statistician and political scientist, has acknowledged and made corrections to his published work. Gelman is known for his commitment to transparency and rigor in research. He frequently discusses errors, corrections, and the general fallibility of research in his blog and academic writings.
Gelman has openly admitted mistakes in his own work and even encouraged the broader academic community to adopt a culture of addressing errors constructively. This attitude aligns with his broader advocacy for improving scientific methods, replicability, and honesty in research.”
If you’re interested in specific instances, many of these are documented on his blog, Statistical Modeling, Causal Inference, and Social Science, where he writes about such corrections in detail.”
My apologies to Chat GPT for posing the wrong question. So to speak, ask the wrong question and you are likely to get the wrong answer.
I don’t mean to be flippant, but why does asking a LLM count for anything at all? It uses old information and you can do the actual research yourself if you truly care about this. I genuinely do not understand what this contributes to the discourse except that LLMs have limited utility…which we should already know.
I fully endorse the correction/corrigendum route. However, I recently had a PhD student tell me that they were going to cite one of my papers (which has a correction to the authors list, contributed by me) until they were informed by their supervisor not to include any papers with a correction. It is the right thing to do but it is almost certain to penalise you as an early career researcher.
> It is not clear to us in general how to avoid this sort of false proof
There is now computer software (proof assistants such as Coq, Lean, Isabelle, …) which can check mathematical proofs step-by-step. A proof specified and checked in those languages is guaranteed to be error-free and cover all edge cases. The downside is that writing a proof in this way is still really difficult comparable to writing assembly instructions instead of Python code. But it’s not hard to imagine a future where the standard libraries of those languages become powerful enough that even normal mathematicians are able to write proofs in this way.
6, 5, 11, and 18 years. Not sure what to do with that, but interesting…