Hot hand 1, WSJ 0

link

In a generally good book review on “uncertainty and the limits of human reason,” William Easterly writes:

Failing to process uncertainty correctly, we attach too much importance to too small a number of observations. Basketball teams believe that players suddenly have a “hot hand” after they have made a string of baskets, so you should pass them the ball. Tversky showed that the hot hand was a myth—among many small samples of shooting attempts, there will randomly be some streaks. Instead of a hot hand, there was “regression to the mean”—players fall back down to their average shooting prowess after a streak. Likewise a “cold” player will move back up to his own average.

No no no. The funny thing is:

1. As Miller and Sanjurjo explain, the mistaken belief that there is no hot hand, is itself a result of people “attaching too much importance to too small a number of observations.”

2. This is not news to the Wall Street Journal! Ben Cohen reported on the hot hand over a year ago!

On the plus side, Easterly’s review did not mention himmicanes, power pose, the gay gene, the contagion of obesity, or the well-known non-finding of an increase in the death rate among middle-aged white men.

In all seriousness, the article is fine; it’s just interesting how misconceptions such the hot hand fallacy fallacy can persist and persist and persist.

16 thoughts on “Hot hand 1, WSJ 0

  1. My take-away from Miller and Sanjurjo is: “If you believe that [miss, miss, miss, miss, hit, hit] represents the exact same degree of “hotness” as [hit, hit, hit, hit, hit, hit], then there is such a thing as a hot hand even with iid coin flips.”

    Here is a table of rounded and exact percentages under their interpretation, for sequences of lengths N=2 to 20:

    2 50.0% 1/2
    4 40.5% 17/42
    6 41.6% 129/310
    8 43.3% 769/1778
    10 44.5% 4097/9198
    12 45.5% 20481/45034
    14 46.2% 98305/212966
    16 46.7% 458753/983010
    18 47.1% 2097153/4456414
    20 47.4% 9437185/19922906

    • Peter:

      No, with coin flips there is no hot hand. The hot hand comes because you’re more likely to make the shot when you’re hot; that is, the probabilities are not 50/50 and they change over time.

      Miller and Sanjurjo used coin flips to demonstrate the bias of the Gilovich et al. estimate, but the relevance comes because basketball shooting is not coin flipping.

  2. Check out the persistence of the huge number of Eskimo words for snow. Or the Chinese word for crisis being composed of danger and opportunity. Once embedded in the brain, it never leaves the stockpile of cliched examples to be hauled out by motivational speakers the world over.

  3. To be fair, the study is recent (mid-2015) and currently not yet published. It’s not surprising someone might have missed it , especially as Easterly does not usually work in this area.

    Also, in your previous hot hand post you mention that another way to think about it is that we do not dispute the existence of a “cold hand,” so a “hot hand” is not that outrageous. However, there are many situations where there is asymmetry in performance. For example, a very bad financial trader will soon get a -100% return and lose everything. However, a brilliant trader probably will not beat the market much more often than just by chance.

    • Jack:

      Sure, fair enough that Easterly is not an expert in this area so he’s just repeating something he remembered hearing somewhere. In particular, the existence of a hot hand is not contradicted by regression to the mean. Suppose (to take an extreme, exaggerated example) I make 10 shots in a row, then I only make 8 of my next 10. I am regressing toward the mean, but there’s still a hot hand effect.

  4. I’ll grant that “there is zero evidence of athletes having a hot hand’ is a zombie lie. But let’s not get carried away. Even with the improved stats, the effect is very small. True to a small degree and important in a small number of cases, but let’s try this thought experiment: tell one group of subjects that there is no hot hand, and tell another group that there is, but it is very small. Given people’s very strong tendency to be overconfident when they have highly unreliable information, my guess is that the latter group will predictable overestimate the likelihood of continued success or failure, and perform substantially worse in a prediction task.

    • Rjb:

      I don’t know why you’re so sure the effect is very small. A simple calculation shows that, even if the underlying effect is huge, it will appear small when being estimated using autocorrelations.

    • RJB:

      Small is relative. The point estimate in 13pp in GVT’s data, and 8pp in the Three Point Contest, *on average*, and that is very likely an underestimate. These estimates aren’t small, considering the gap between the median and best 3-point shooter in 10pp.

      The hot hand effect, i.e. the change in a players underlying probability of success, could be ginormous and the tools we have would never detect it.

      For terrible beliefs, it is plausible that some people, especially fans, will get it wrong, but where is the study? Even in GVT’s study the bettors do pretty well predicting, as we show in the new version of our paper.

      • I’ll have to look at the new version. My reading of the overconfidence literature is that people routinely overrely on unreliable information (and underrely on reliable information), which I’ve explained as a pretty reasonable Bayesian error, given that people typically have noisy information about the reliability of their information (“moderated confidence”). I would think these effects are more on the unreliable side of things, but that’s an empirical question.

        As a more general issue, I see the interesting question in psych as “do people generally overestimate the hot hand effect”, not “is there a hot hand effect”? (Of course, for a statistician, the priorities might well be reversed, and kudos for a great paper pointing out the statistical subtleties that were missed in the first go round.

        • thanks RJB.

          For GVTs paper, it was about the cognitive illusion, not about the degree to which beliefs over or under estimate the hot hand. This cognitive illusion result has been interesting to the psych crowd, but it is wrong, and we we found shows that. Anyway, agreed, over/under-reaction to information is what is interesting to the psych crowd. That is a harder question answer in the basketball context. We take investigate the evidence hot hand beliefs in our paper.

          Also, see this recent post (and comments) on Gelman’s blog.

  5. I’m not a stats guy or a sports expert, but isn’t it possible that “feeding” the “hot hand” would still be smart because it might mean that something was awry with the opposing team’s defense that night? Or that that specific player had a favorable matchup against his defender?

Leave a Reply to Joshua B. Millerr Cancel reply

Your email address will not be published. Required fields are marked *