We all make mistakes. What’s important is to engage with our mistakes and learn from them. When we don’t, we’re missing an opportunity to learn.
Here’s an example. A few years ago there was a Harvard study that notoriously claimed that North Carolina was less democratic than North Korea. When this came out, the directors of the study accepted that the North Korea estimate was problematic and they removed it from their dataset. But I don’t think they fully engaged with the error. They lost an opportunity to learn, even though they were admirably open about their process.
Here’s a more recent example. The authors of the influential policy book Nudge came out with a new edition. In the past, they, like many others, had been fooled by junk science on eating behavior, most notably that of Cornell business school professor Brian Wansink. I was curious whassup with that so I searched this press release news article and found some stuff:
They began the [first edition of their] book with the modest example of a school administrator rearranging the display of food at the cafeteria, increasing the likelihood that kids choose a healthier lunch. . . .
In addition to new things in the new version of the book, there are old things from the original version that are gone. That includes the research of former Cornell University professor Brian Wansink, a behavioral scientist who got caught producing shoddy research that fudged numbers and misled the public about his empirical findings. . . . Thaler is cheering on the social scientists probing academic literature to suss out what can be proved and what can’t. “That’s healthy science,” he says.
That’s cool. I like that the author has this attitude: instead of attacking critics as Stasi, he accepts the value of outside criticism.
Just one thing, though. Removing Wansink’s research from the book—that’s a start, but to really do it right you should engage with the error. I don’t have a copy of either edition of the book (and, hey, before you commenters start slamming me about writing a book I haven’t read: first, this is not a book review nor does it purport to be; second, a policy book is supposed to have influence among people who don’t read it. There’s no rule, nor should there be a rule, that I can’t write skeptical things about a book if I haven’t managed to get a copy of it into my hands), but I was able to go on Amazon and take a look at the index.
Here’s the last page of the index of the first edition:
And now the new edition:
Lots of interesting stuff here! But what I want to focus on here are two things:
1. Wansink doesn’t play a large role even in the first edition. He’s only mentioned once, on page 43—that’s it! So let’s not overstate the importance of this story.
2. Wansink doesn’t appear at all in the second edition! That’s the lost opportunity, a chance for the authors to say, “Hey, nudge isn’t perfect; indeed the ideas of nudging have empowered sleazeballs like Wansink, and we got fooled too. Also, beyond this, the core idea of nudging—that small inputs can have large, predictable, and persistent effects—has some deep problems.” Even if they don’t have the space in their book to go into those problems, they could still discuss how they got conned. It’s a great story and fits in well with the larger themes of their book.
Not a gotcha
Connecting Nudge to pizzagate is not a “gotcha.” As I wrote last year after someone pointed out one of my published articles: It’s not that mistakes are a risk of doing science; mistakes are a necessary part of the process.
P.S. The first edition of Nudge mentioned the now-discredited argument that there is no hot hand. Good news is that the Lords realized this was a problem and they excised all mention of the hot hand from their second edition. Bad news is that they did not mention this excision—they just memory-holed the sucker. Another opportunity for learning from mistakes was lost! On the plus side, you’ll probably be hearing these guys on NPR very soon for something or another.
Brian Wansink is still busily nudging. A little “taste” from his blog post on redesigning school cafeterias:
“As I mentioned, our lab studies showed that lunchgoers were 11 percent more likely to take whatever vegetable they saw first compared to whatever they saw third. Well, that’s true when three vegetables are in the middle of the serving line, but here we put them in the front of the line. Nobody scoops up a plate of green beans and then looks for the entrée that goes with it. People pick out the entrée and then the vegetable. They didn’t want to take a veggie until they knew what they were having for a main course.
When the interview got to this point, the producer asked, ‘You’ve been doing eating research for twenty-five years. Sales didn’t increase by 11 percent, they dropped by 30. Why were you so far off?’ I said, ‘Well, if we always knew what we were doing, we wouldn’t call it research. (He seemed amused enough by this answer to not report these missed predictions in his story.)'”
And Wansink is publishing again. Here’s Retractionwatch:
“The articles [by Wansink], in Cureus and the International Journal of Community Medicine and Public Health, appear to use data that are at least a decade old. Wansink’s only coauthor on the papers is Audrey Wansink, a high school student and, evidently, his daughter.”
It’s a shrewd move. Enlist a high school kid, and the Stasi will lose their will to fight.
I am confused, severely.
First I was thinking: wow, you have entrees (appetizers) in school cafeteria, that’s fancy, but little googling tells me that in United States entree means the main dish. Allright…
Then I was even more confused!
In all schools, in all buffets, in hotels, in pizzerias, everywhere where I’m expected to gather my own food on my plate, salads have always been the first thing to be picked. That has never been a problem, I’ve never even thought about it that much – it’s just how it is. I’ve always just scooped up something that looks good and filled up about half of my plate with that.
But in school cafeterias, I’m told, children are so picky or so culinaristic that they are actually planning their meal in such a detail that they can’t possibly choose their salad before seeing what’s for lunch? What?
It’s like I’m reading a description of something that’s happening on another planet.
And I’m not complaining about the children! It’s the research and Wansink’s description of it that seems surreal. I don’t understand the context of it at all. Maybe they are, indeed, working in some context in which that sort of thing IS important, but more than that, it just seems like they’re just fiddling around with random things and the kids, perhaps, get confused… who knows, and seems, according to that quote, that they don’t themselves know.
Reminds me actually of another Wansink quote. He was talking about the endless soup bowl experiment. Some people tried picking up the bowl and Wansink – in an attempt at dry humour – described them as channeling their viking ancestors. Now, everytime me or my girfriend pick up the plate/bowl to eat from eat from it, we have to remark something about being savage viking warriors.
(That description the the vikings is actually really narrow).
Let’s face it, in reality kids have two choices, the greasy pizza or the chicken nuggets, and they can either have an apple or an orange, and a regular milk or a chocolate milk.
I was originally kind of taken aback by the characterization of Wansink as a “sleazeball”. However, if this is accurate, and Wansink is pushing old data in new bottles with his daughter’s name on the label… well, that is certainly sleazy. (My initial reaction was born of reflex, though, and says nothing at all about the accuracy of the label!)
I’m thinkin’ the “sleazeball” label is related to him having been caught faking data. Seems pretty sleazy to me.
Andrew linked to one of his lectures once and I watched much of it. He served a lot of emotional-bias-appealing mocking-big-food jokes but offered only one single lonely slide late in the lecture with what I guess was supposed to be the nutritious data to support his claims. It looked like a hand-drawn graph, kind of like the last dried-up slice of apple at the bottom of the bowl, so pathetic that it was almost like he was mocking the concept of having to bother with data at all when he already has a perfectly good story. The audience…er…ate his story right up…so I guess it is true that if you put the yummy bias-satisfying stuff up front people tend to consume that and ignore the fact that almost nothing nutritious is even being offered.
From that list, I was curious enough to Google “cashew phenomenon”. All the results only proved to me that economists are bad at dinner parties.
“…before you commenters start slamming me about writing a book I haven’t read…”
How about if I slam you about leaving a word out: “…writing a book I haven’t read…”?
Well, okay, maybe “slam” is too strong for just one word. Let’s try “nudge”.
Should be “…writing about a book I haven’t read…”
My original comment had “about” in greaterthan-lessthan symbols and the auto comment screener treated it as an html code.
Is discussing Wansink in the new edition of Nudge really the sensible thing to do? It’s not like a newspaper article where one wants to avoid “stealth edits” online of what was originally printed since it’s a matter of record what was printed at a specific date/time. People who buy the current edition may not care at all about what was wrong with the inferior earlier edition. On the other hand, perhaps there could be an intro discussing just what’s improved in the new edition. The question of how/why the authors got fooled in the first place might be deserving of an entirely new book.
Wonks:
To me, the reason for discussing Wansink in the new edition of Nudge is not to avoid “stealth edits” but rather to teach readers the pitfalls of believing too much in a good story. Consider: the Nudge authors were big fans of Wansink—they described some of his work as masterpieces—then it turns out that the Wansink experiments were based on nothing but hype. This should suggest some wariness regarding other “nudge” claims, no? But, no, the hype machine goes on (as here, for example). By memoryholing the Wansink incident, the nudgelords are avoiding the chance to learn from their mistakes. I think it would be better for them to devote a chapter to how their entire field got conned by Mister Soup Man. It’s not just that there was a flaw in the first edition of the book, it’s that this error reveals a flaw in their whole method of operation.