“In my previous post on the topic, I expressed surprise at the published claim but no skepticism”

Don’t believe everything you read in the tabloids, that’s for sure.

P.S. I googled to see what else was up with this story and found this article which reported that someone claimed that Don Green’s retraction (see above link for details) was the first for political science.

I guess it depends on how you define “retraction” and how you define “political science.” Cos a couple of years ago I published this:

In the paper, “Should the Democrats move to the left on economic policy?” AOAS 2 (2), 536-549 (2008), by Andrew Gelman and Cexun Jeffrey Cai, because of a data coding error on one of the variables, all our analysis of social issues is incorrect. Thus, arguably, all of Section 3 is wrong until proven otherwise. We thank Yang Yang Hu for discovering this error and demonstrating its importance.

Officially this is a correction not a retraction. And, although it’s entirely a political science paper, it was not published in a political science journal. So maybe it doesn’t count. I’d guess there are others, though. I don’t think Aristotle ever retracted his claim that slavery is cool, but give him time, the guy has a lot on his plate.

17 thoughts on ““In my previous post on the topic, I expressed surprise at the published claim but no skepticism”

  1. I will take the liberty of mirroring the comment I made on Monkey Cage here. MS

    What if Green had denied everything and threatened law suits etc. ? Where would we be then?

    Also, was it ever a real option for Andrew to write a blog post a few months back saying something like:

    “Gee. that result is awfully hard to believe. Maybe the data are fabricated.”

    Notice the furious and insulted reaction of many people to the suggestion that their work may run afoul of the “garden of forking paths” critique. Just the suggestion that maybe some data are fabricated can easily lead to a menacing letter from a lawyer.

    The behaviour of Donald Green completely transforms the dynamics of this case but this aspect is very special.

    • Mike:

      Nobody would’ve stopped me from suggesting, at the time, that the data were fabricated. I just didn’t think of that possibility at all, nor did I think of the milder possibility that the data were miscoded (which was the source of my earlier retraction, noted above). A true failure of imagination on my part.

      That said, I did get a lot of pushback when earlier on the Monkey Cage blog I’d expressed skepticism about the claim that subliminal smiley-faces influence political attitudes. No threatened lawsuits but quite a bit of anger; see here.

      • So why didn’t a claim that sounded too good to be true raise Donald Green’s suspicions?

        Is there an incentive here to be extra naive? Why examine too closely the hen that seems to be laying golden eggs?

        • The whole thing about claims that are too good to be true is that people want to believe them. And people who think they are smart are vulnerable to thinking they are really special. That’s why all those rich people left their money with Madoff.

        • Not extra naive — I think Green wanted the effect replicated, and it was. I think part of it is that people aren’t expecting all-out fraud.

  2. Definitely agree with last point regarding this being an impressively fast turnaround. I wonder, had this paper been published 10 years ago, whether it would have ever been retracted at all?

    • Rahul:

      Section 3 was critical to our empirical claims. The paper also had a theoretical argument that was not affected by our data problem. I could’ve retracted the whole paper but it seemed more precise to just state which section was wrong.

  3. A researcher has to be stupid to fake data in political science. One can almost always apply an unsupported or inaccurate statistical analysis or just not report contradictory observations and get a paper. LaCouer’s problem (this is the guy who allegedly faked the data) was he aimed a little too high and it was a little too simplistic a claim. A little less prominent a subject and no one would have followed up.

  4. THIS IS HOW ONE SHOULD RESPOND TO SUCH A CODING ERROR: In the paper, “Should the Democrats move to the left on economic policy?” AOAS 2 (2), 536-549 (2008), by Andrew Gelman and Cexun Jeffrey Cai, because of a data coding error on one of the variables, all our analysis of social issues is incorrect. Thus, arguably, all of Section 3 is wrong until proven otherwise. We thank Yang Yang Hu for discovering this error and demonstrating its importance.

    The usual way (at least in economics), is to argue, “OK, so correcting the coding error reverses my conclusion. But let me now give you ten reasons that my original conclusion is still valid.”

    Andrew, I’m using your fine example in a talk I’m giving on replication next week. Thanks for posting it!

    Bruce McCullough

  5. I like your note in the MC article about how we can back any claim up with a story pretty easily. It’s interesting how much the opposite of the famous John Adams quote about stubbornness of facts is true and still how strongly it is imprinted in debate culture.

    • Elin:

      I think the difference is only in age and career: Lott was further along when his deceptions were alleged. If LaCour chooses to goes the full ideological route, I expect he could play a Lott-like role somewhere.

      Again, Lott has managed to affiliate himself with some legitimate scholars, which suggests that he has something to offer, that they’ll work with him despite his being a laugh line who could produce no documentation for his most famous study. LaCour seems to have talent—it’s not so easy to make up a whole fake study, and he also has an ability to make pretty graphs, which is a useful skill right there.

      I agree, though, that LaCour seems more like a Glass-style constructer of an entire fake world, as compared to Lott who is maybe following more instrumentally focused on advancing a political ideology. Lott may have felt that the results of his undocumented study were essentially true in any case, so that actually conducting the survey would’ve been a mere formality. Along the same lines, he might’ve felt that the people who questioned his study were themselves motivated by ideology, they’re all a bunch of liars, so he’s just fighting fire with fire, don’t bring a knife to a gunfight and all that.

      For LaCour to start making up checkable things on his resume, that’s just out of control. What next—responding to internet comments using a fake name???

Leave a Reply to Seth Spain Cancel reply

Your email address will not be published. Required fields are marked *