Stan Liebowitz writes:

Have you ever heard of an article being retracted in economics? I know you have only been doing this for a few years but I suspect that the answer is that none or very few are retracted. No economist would ever deceive another. There is virtually no interest in detecting cheating. And what good would that do if there is no form of punishment? I say this because I think I have found a case in one of our top journals but the editor allowed the authors of the original article to write an anonymous referee report defending themselves and used this report to reject my comment even though an independent referee recommended publication.

My reply: I wonder how this sort of thing will change in the future as journals become less important. My impression is that, on one side, researchers are increasingly citing NBER reports, Arxiv preprints, and the like; while, from the other direction, journals such as Science and Nature are developing the reputations of being “tabloids,” publishing articles that are newsworthy but often empty of content.

Based on your side of the story (which is all I have to go on), I find the behavior of those two economists to be unsettling. The very fact that they wanted to suppress a criticism—that itself makes me suspicious. It would be much more appropriate for them to request space for a rejoinder in the journal.

Some attitudes surprise me. For example, on his blog, journal editor Steven Levitt wrote, “Is it surprising that scientists would try to keep work that disagrees with their findings out of journals? . . . Within the field of economics, academics work behind the scenes constantly trying to undermine each other.” See my discussion here.

To get to your question. I am not an expert on the econ literature and am aware of only two retractions. The first is the paper by Emily Oster, a young economist who engaged in a several-years-long battle with public-health researchers regarding a theory of sex-ratio changes in developing countries. She eventually gave up and admitted that the public-heatlh researchers were right and she was wrong. (But she did not, as far as I know, retract her claims that economists benefit from having a special kind of reasoning.)

The second retraction I know of in econ is Bruno Frey‘s admission that he had self-plagarized (fulfilling “Arrow’s theorem” which states that any unique result can be published up to five different times). This was a bit of a scandal; he was reprimanded by the editor of a journal that had published one of the redundant papers.

In any case, I think we can agree that the number of retractions is all too low. Here’s another story. Many years ago I had a colleague who showed me a manuscript of a paper he had submitted to a top journal in his applied field. I looked over the paper and realized it had a serious flaw. I won’t go into details, but the short version is that he had fit a probability model inappropriately. Nothing wrong with the computation, but the model didn’t fit the data. The parameter estimates didn’t mean what he thought they did. My colleague agreed with me (or, at least, he assented when I gave his criticisms) and he went on to fit a better model. But in the meantime, the original paper was accepted by the journal! I naively thought he’d withdraw the article but, no, he published it essentially as is. I asked him how he could in good conscience do this, and he said that he thought the paper still had good stuff in it and that it was an advance in that field. And, the funny thing is, maybe he was right! Who am I to say?

I was also more peripherally involved in a project with experimental data where the researchers had a before-after comparison with the treatment group and a before-after comparison with the control group. There was a large and statistically-significant improvement in the treatment group—and also a large and stat sig improvement for the controls. The difference in differences was not statistically significant. So what did the researchers do? You guessed it: they just reported the difference for the treatment group. This one bothered me a lot, but I didn’t follow up on it, and I’m not sure if maybe they cleaned it up before publication (I only saw an early version of the report).

Back to the main thread. My impression is that the biggest concerns regarding retractions right now involve three issues:

1. Biased research in medicine, including: (a) drug companies wanting success and manipulating data and analysis to get there; (b) manipulation through selection (commissioning many studies and reporting only the ones you want), and (c) labs where the researchers are under intense pressure to get results, so they manufacture them. An extreme case of (c) is Robert Gallo stealing HIV and pretty much getting away with it.

2. Pure noise corrupting the signal. Here I’m thinking about professional researchers who don’t really do research, but they manage to publish their papers somewhere or another, and those journals and conferences which have nothing to do with science but exist only to make money off of publication fees etc. You might feel you could ignore these because they’re not prestigious journals, but can an outsider really distinguish between the (legit) Journal of Money and Banking and some fake journal out there? Also we hear about researchers in other countries who spend their time swamping legitimate journals with papers that are made up, plagiarized, etc.

3. Errors arising in serious research due to selection bias, confirmation bias, etc. Psychology researchers are particularly worried about this one: the idea is that if you look hard enough you can find confirmation of just about anything. The most famous recent case is psychologist Daryl Bem, who published a paper on ESP which was supported by 9 different experiments! That sounds pretty impressive (and it impressed the journal editors enough that they published it), but it turns out that, yes, if you’re looking for an effect, you can indeed find it, wrongly, in 9 different ways. There’s a growing concern in psychology research that this sort of thing is happening all the time–not just on joke topics such as ESP but in more respected areas of research such as motivation and social psychology.

Liebowitz wrote back:

I was already familiar with Frey’s problem (I was recently told that his contract at Zurich was not renewed). I don’t consider self plagiarism, or even pure plagiarism as serious a “crime” as making stuff up. Even with his brazen behavior I doubt that the same fate would have befallen him if he were in the prime of his career. The cost of letting him go now is much lower, although he was mentioned as a possible European Nobel prize candidate, which I suspect is no longer the case. Interestingly, he is supposed to attend a conference that I am attending, on scholarly publishing of all things, in about a month. I am looking forward to hearing what he has to say. I have trouble believing he will show up, although I am always surprised at the chutzpa that some people have to be able ignore problems that would make me ashamed to show my face. One of the two authors that I criticize about impropriety in piracy research was Frey’s student and one of the self-plagiarized papers was coauthored by the two of them, although the fellow I believe to have made stuff up was too junior to be given credit for the deceit with Frey. Small world.

To which I replied that I agree that self-plagiarism is not as bad as plagiarism, which in turn is not as bad as outright fraud. (In many cases, though, plagiarism does involve fraud, for example a secondhand story presented as factual, or a plagiarized passage that is so badly garbled that it loses its original meaning.)

Also, I agree on the chutzpah thing. I’m stunned that people don’t just admit these offenses when they’re caught.