A couple days after listening to the segment where This American Life got conned by Mars One, I happened to listen to the This American Life segment on LaCour and Green. LaCour didn’t appear on the show but Green did. Wow, Ira Glass really got scammed. But it was a pretty elaborate con; LaCour not only had data, he had entire fake conversations between canvassers and voters. Or maybe those conversations were real? It wasn’t clear if they were supposed to be part of the research study or something done afterward, just for the show.
Glass sets up the canvassing experiment as something that shouldn’t actually work:
There’s this thing called the backfire effect. It’s been documented in all kinds of studies. It shows that when we’re confronted with evidence disapproving what we believe, generally we just dig in and we believe it more. And the rare times that people do change, it’s slow.
Ira then gives an example of one of the interviews and says:
And even more amazing than the fact that this actually worked is that it lasted. . . . And a study by researchers at UCLA and Columbia University found that a year later, not only did these voters stay convinced, they also convinced others in their own households to switch. Apparently neither of those things ever happens.
And the episode itself was called “The Incredible Rarity of Changing Your Mind.”
What did Don Green think about all this?
Professor Green says he and his colleagues have read 900 papers. And they haven’t seen anything like this result—anyone who’s changed people’s views and it lasted like this.
Ok, this is just unfortunate. If you don’t entertain the possibility that the data were faked—a possibility I did not myself consider when blogging on the paper a year ago, when I described myself as “stunned” at such “huge” effects—then, yeah, you gotta say this is newsworthy.
Now here’s something from the This American Life retraction statement:
As for the canvassers at the Leadership LAB at the Los Angeles LGBT Center, they say they were blindsided by the news of Green’s retraction. “This is a complete and utter shock to us and we’re still trying to figure out which way is up,” said Steve Deline, one of the organizers of the canvassing. “We had no idea Mike was fabricating data.”
Deline told me that the way it worked is that LaCour gave them lists of people he claimed to have signed up for the online survey. Then canvassers did their jobs and went to those houses. This took hundreds of hours. If LaCour was lying, what a waste.
So now I’m confused. The interviews all actually happened? Not all 9,507 of them, right? And if there was really no money to do the survey—apparently LaCour also lied about the funding—then who paid the canvassers? Or did they do it all for free?
Glass concludes the retraction as follows:
“Maybe the thing to convey in your blogpost” Green told me “would be something to the effect that, just because the data don’t exist to demonstrate the effectiveness of this method of changing minds, doesn’t mean the hypothesis is false. And now the real work begins.”
He thinks the canvassers should go out and do the study again, for real this time.
Wait. One. Second.
You have a study that, if it were actually done, would be very expensive (a point Glass made in his original show), to explore a hypothesis that contradicts what’s been “documented in all kinds of studies”; there have been “900 papers” on related topics and none of them report anything like these effects; the only time this effect has been found, it turns out to have been an embarrassing fraud—and Green thinks they should go out and do the study again?
Ulp. There are lots and lots of studies people are interested in doing, and I’m sure this activist group in Los Angeles has a long to-do list. Do you really think they should spend their precious time, money, and human resources to study an idea that is contradicted by an entire 900-paper literature and whose only claim to plausibility was a made-up experiment??
“And now the real work begins,” indeed.
You gotta be kidding.
P.S. In fairness to Don Green, he gave this quote to Glass on a day when he must have been stunned by LaCour’s duplicity. I imagine that now, after several months has passed, he’d no longer think it was a wise use of resources to try to replicate this fake study.