Skip to content
 

The illusion of the illusion of control

Yesterday we discussed the sad and disturbing career of psychology researcher Ellen Langer, who was was famous (to me) for her 1975 article on the illusion of control, “defined as an expectancy of a personal success probability inappropriately higher than the objective probability would warrant.” And then, in her own research, she herself became subject to the inclusion of control, and seemed to have developed a side career of hype and pseudoscience.

Actually, though, things may be even worse.

The illusion of control itself does not seem to be as robust as was implied by Langer in her famous 1975 paper; Michael Schulte-Mecklenbeck points us to this 1995 article by Kuhberger, Perner, Schulte, and Leingruber, who report on “four experiments . . . run in order to replicate the findings of Langer and Nichols et al. None of them was successful.” They continue:

Langer’s effect is not easily replicated. Inspection of the literature shows that most references to the illusion of control in the risky choice paradigm relate to Langer’s original work and that there are no replications of her Study 2 in the social psychology and decision theory literature. This is quite surprising, since the illusion of control is one of the more important instances of cognitive illusions . . . mentioned in most textbooks. . . .

Given the great variation in response patterns and the dearth of replications of Langer’s effect, it is an open question whether the effect does actually exist. The two positive reports in the literature (Langer, 1975, and Nichols et al., 1996) may reflect a reporting bias of positive findings in a sea of unreported failures to replicate.

Kuhberger et al. continue on a moderate note:

Perhaps this is too extreme a view to take and Langer’s effect does exist. However, in that case our results suggest that it is highly volatile. It leads to statistically reliable findings only on occasion, and we do not yet know which combination of factors is responsible for it.

Why did we all believe it?

At this point, the question arises: Why were I (and others) so quick to believe Langer’s claims? Indeed, even after hearing about her other shaky research, why did I not question the illusion of control.

As always, a Why question indicates an anomaly, a gap in our understanding of the world.

In this case, Langer’s claim had a lot going for it:

– It was in the classic book by Kahneman, Slovic, and Tversky, a book of articles that have been highly influential in psychology and elsewhere, a book which I’ve described as the best edited collection since the New Testament.

– The article is itself well written and tells a convincing story.

– The message of the article rings true. Whatever happens with Langer’s particular claims, the general phenomenon of “the illusion of control” happens all the time.

In that way, Langer’s claimed results are similar to various published claims in evolutionary psychology. The general framework is evidently correct—yes, our brains have evolved, and, yes, we overestimate our chances of success in many settings. But, of course, we already knew all that; the purported contribution of experimental research is to understand these phenomena, not merely to remind us of the truth of some very general theories. The devil is, as always . . . well, you know where the devil is. He’s not going anywhere.

17 Comments

  1. Z says:

    “The devil is, as always . . . well, you know where the devil is. He’s not going anywhere.”
    Good writing

  2. G says:

    The Devil is in the sample size of _23_ for the experiments by Kuhberger, Perner, Schulte, and Leingruber?

  3. Rahul says:

    “The article is itself well written and tells a convincing story”

    Andrew:

    This is exactly why I’m wary when someone pushes the utility of “storytelling” in a research setting. The problem with stories is that there are many very tantalizing ones which are unfortunately not true.

    Promoting the storytelling model of doing research increases the risk of science being driven by the most engaging storyteller rather than the actual merits of the underlying data.

    • Andrew says:

      Rahul:

      When Basbøll and I write about the importance of stories, it’s not so much that we’re recommending or promoting a storytelling model of doing research. Rather, we’re saying that, for better or worse, we do learn from stories, so let’s figure out how that works.

      Recall that one of the properties we like about stories is their immutability, or stubbornness of facts. The immutability of the “illusion of control” story allowed these later researchers to understand things better via their attempted replications.

    • george says:

      Rahul: Research is always going to be somewhat adversarial; data can not and does not speak entirely for itself.

      Ideally authors would try to present optimally-lucid ‘stories’ (i.e. arguments) for and against the underlying ideas. Where that’s not done, the reader has to be responsible and come up with these arguments on their own. Nullius in verba, and all that.

      • ConvexPhil says:

        @George:

        Data doesn’t speak for itself, but I think there’s a considerable difference between storytelling around data and formally specifying a model that generates testable predictions in some experimental context. In my experience (primarily as a psych undergrad), ‘modeling’ in psychology tends to consist primarily of storytelling with a poorly articulated link to a statistical test derived from the general linear model. It makes it very difficult to gauge whether a psychological theory has been falsified or not, since it’s often not clear at all what the theory consists of in the first place.

      • george says:

        Rahul, ConvexPhil: Apologies if I was unclear, I was responding to Rahul’s concern over “science being driven by the most engaging storyteller rather than … data”.

        There is no way to completely avoid science being driven in this way, so long as it’s conducted by humans. A bit of objectivity will always feature, and communications skills will always matter.

        I think we agree on the importance of good modeling – it’s part of making a good argument.

    • Fernando says:

      @Rahul

      I agree. The only thing better than p-value <0.01 is good piece of science fiction writing.

      Maybe the science fiction part (consciously or not) tightens our prior around the alternative hypothesis. As a result we are much more likely to interpret a rejection of the null as evidence in favor of the alternative, rather than as evidence in favor of a research artifact (a la Jaynes).

      You could test this experimentally I guess. Prime a group of subjects with great story telling about the real possibility of ESP using certain prior evidence. Then show them Bem's research. Another group receives the same "prior evidence" but dressed in dry / objective scientific language. My hypothesis is that more subjects in the science fiction writing will be convinced by Bem. The difficulty here is to manipulate only the writing style, not the underlying information. Ideally, the subjects would be journal editors.

  4. Rahul says:

    Is it just a freak coincidence that the iffy work was in Kahneman’s book? Or is there a chance this is the start of a trend, the tip of the iceberg?

    I get this uneasy feeling reading Kahneman, and especially the parts of Thinking, Fast and Slow that I’ve sampled.

  5. Jeff says:

    > Whatever happens with Langer’s particular claims, the general phenomenon of “the illusion of control” happens all the time.

    Would you mind elaborating on this? If the general claim is obviously true, then a the failure of particular operationalization to yield significant replication results isn’t much at all. If we’re convinced the effect is real, that’s all we need. No?

  6. Zachariah says:

    Readers might be interested to see this article I co-authored with Francesca Gino and Don Moore called Keeping the Illusion of Control under Control (http://fgino.weebly.com/uploads/4/7/4/7/4747506/ginosharekmoore_obhdp_2011.pdf). We explore some of the subsequent work done on the illusion of control since Langer’s paper and explore some obvious extensions that cast further doubt on the illusion of control.

    Incidentally, the title of my dissertation is ‘The Illusion of the Illusion of Control’ and explores this work even further. Haven’t gotten around to publishing it yet though.

Leave a Reply