There’s a motto in poker: Fold or raise, never call.
Ted’s really going all in on this one. The interview was 100% Cuddy with not a single link to any critical remarks. Here’s a partial list of people not interviewed to offer their reactions:
– Dana Carney,
– Eva Ranehill,
– Uri Simonsohn.
In the interview, Cuddy offered up some Daryl Bem-style meta-analysis (which, as someone pointed out, must be one of the few meta-analyses ever presented in which every one of the papers counted as either a success or as mixed evidence; not a single failure in the bunch! It’s good to know that they have a hypothesis that can never fail.), along with this wonderful, wonderful anecdote:
I [Cuddy] have heard from three different labs that have conducted research on “power posing” but who said they feel they cannot submit the work to journals, because they fear backlash after seeing what has happened to me.
They could publish these blockbuster results on Arxiv, no? Or maybe PPNAS? I’m sure PPNAS’s social psychology editor would give these submissions a fair hearing. In all seriousness, I give zero credence to results which have not even been posted anywhere. This is just ridiculous, it’s really a pitiful response in this modern era in which anything can be posted on the web and in which there are thousands of scientific journals.
Also it’s not clear what Cuddy means by “what has happened to me.” What exactly has happened to Cuddy? She was one of three authors on a paper that was published in a psychology journal. She followed this up by promoting this paper in a popular talk and in a book that was reviewed in the New York Times among other places. She did lots and lots of media. At some point people pointed out statistical flaws in this paper. Other people tried to replicate her findings and failed. The first author on the original paper wrote an expose describing lots of problems in the research and disavowing the claims. At what point in this process did something “happen to” Cuddy? Where did she become the victim here?
Is there a rule that if you publish a paper in a peer-reviewed journal and promote it around the world, that other people aren’t allowed to point out flaws in that work? Or is it a rule that the first author of the paper isn’t allowed to take criticism seriously and retract errors? Cos nobody told me about this rule!
Anyway, I can understand why Cuddy keeps on with this; she must feel like there’s no going back now, only forward movement is possible. Sure, her behavior does not follow what I consider good scientific practice.
But there’s a selection bias here. I’ve heard of Cuddy only because of the power pose fiasco; by looking at her case I’m selecting on a scientist with a record of exaggeration. Recall the final sentence of the abstract of that famous paper:
That a person can, by assuming two simple 1-min poses, embody power and instantly become more powerful has real-world, actionable implications.
I bring up that sentence a lot because it’s just stunning, as the paper in question had no measures whatsoever of anyone “becoming more powerful.” This is not a statistical dispute, it’s a simple statement of fact. The abstract concluded with a statement on something not measured in the paper. Even had the experiment been done just as described with none of the forking paths and the leakage of information described in that later report, even if all the statistics had been fine and we could take all those silly p-values seriously, even then, the paper that got all the attention was full of hype from the start. I’m not talking about the Ted talk, or the media interviews, or the book, I’m talking about the article in the purportedly serious journal Psychological Science.
So, sure, given that track record we should not expect Cuddy to suddenly become scrupulous about evidence. She could make that change—it’s still possible—but not reason to expect it.
No, the thing that surprises—or at least interests—me here is the position of Ted. They made a mistake on that Cuddy talk, then have been years of controversy, the Ted team must have had some discussion of what to do about it, and then they doubled down in a “We stand by Matt” sort of way.
What gives? “Ted talk” has become a punchline, and then they go out of their way to endorse Cuddy’s discredited claims??? Perhaps it was a calculation on the part of the Tedsters that they’d already lost the likes of me, but if they were to disown one of their stars, they’d lose their core supporters.
Or maybe it’s a commercial calculation. I remember back in the 1970s that huge numbers of copies were sold of books like “Chariots of the Gods” and “The Bermuda Triangle,” and various scientists in the media, Carl Sagan and people like him, kept loudly denouncing them. I had the impression that the scientists wanted the publishers to stop selling the books, or to label them as fiction (remember, according to the book publishing industry, Charlie Brown, Pogo, and the Bible are all considered nonfiction), but of course the publisher kept dodging the issue because these books were massive money-makers.
I don’t actually know how Ted makes its money—it’s not by selling copies of Amy Cuddy’s books! Maybe they sell tickets to the Ted conferences? Or maybe they get contributions from credulous richies? I can’t imagine. Ummm, here it is, from Wikipedia:
The membership model was shifted in January 2007 to an annual membership fee of $6,000, which includes attendance of the conference, club mailings, networking tools, and conference DVDs. The 2017 conference will be $17,000 per attendee. . . . TED is currently funded by a combination of various revenue streams, including conference attendance fees, corporate sponsorships, foundation support, licensing fees, and book sales. Corporate sponsorships are diverse, provided by companies such as Google, GE, AOL, Goldman Sachs, The Coca-Cola Company; among others. . . . The TED staff consists of about 140 people headquartered in New York City and Vancouver.
So, I guess I can see it. These 140 people are supported by a revenue stream which is based on the quality of the Ted talks. When the Ted team is told of a Ted talk that has spread false information, they have two choices: admit the error or double down on the original claim. Both choices are risky but you gotta decide, you gotta protect the brand. Fold or raise, never call. They decided to raise.
P.S. It also says on that wikipedia site there’s an annual Ted prize of a million bucks. I wonder if they’ll give it to me this year? I’d love to have another million bucks: I’d spend it directly on Stan programmers. Ted should definitely give me the money, as it would demonstrate their open-mindedness; it would be such good publicity for them. I give this in the same spirit as Albert Brooks had in his spiel to the casino boss in Lost in America.