A few years ago we talked about the two modes of pop-microeconomics:
1. People are rational and respond to incentives. Behavior that looks irrational is actually completely rational once you think like an economist.
2. People are irrational and they need economists, with their open minds, to show them how to be rational and efficient.
Argument 1 is associated with “why do they do that?” sorts of puzzles. Why do they charge so much for candy at the movie theater, why are airline ticket prices such a mess, why are people drug addicts, etc. The usual answer is that there’s some rational reason for what seems like silly or self-destructive behavior.
Argument 2 is associated with “we can do better” claims such as why we should fire 80% of public-school teachers or Moneyball-style stories about how some clever entrepreneur has made a zillion dollars by exploiting some inefficiency in the market.
The trick is knowing whether you’re gonna get 1 or 2 above. They’re complete opposites!
I thought of this when rereading this post from a few years ago, where we quoted Jason Collins, who wrote regarding the decades-long complacency of the academic psychology and economics establishment regarding the hot-hand fallacy fallacy:
We have a body of research that suggests that even slight cues in the environment can change our actions. Words associated with old people can slow us down. Images of money can make us selfish. And so on. Yet why haven’t these same researchers been asking why a basketball player would not be influenced by their earlier shots – surely a more salient part of the environment than the word “Florida”? The desire to show one bias allowed them to overlook another.
When writing the post with the above quote, I had been thinking specifically of issues with the hot hand.
Stepping back, I see this as part of the larger picture of researcher degrees of freedom in the fields of social psychology and behavioral economics.
You can apply the “two modes of thinking” idea to the hot hand:
Argument 1 goes like this: Believing in the hot hand sounds silly. But lots of successful players and coaches believe in it. Real money is at stake—this is not cheap talk! So it’s our duty to go beneath the surface and understand why, counterintuitively, belief in the hot hand makes sense, even though it might naively seem like a fallacy. Let’s prove that the pointy-headed professors outsmarted themselves and the blue-collar ordinary-Joe basketball coaches were right all along, following the anti-intellectual mode that was so successfully employed by the Alvin H. Baum Professor of Economics at the University of Chicago (for example, an unnamed academic says something stupid, only to be shot down by regular-guy “Chuck Esposito, a genial, quick-witted and thoroughly sports-fixated man who runs the race and sports book at Caesars Palace in Las Vegas.”)
Argument 2 goes the other way: Everybody thinks there’s a hot hand, but we, the savvy social economists and behavioral economists, know that because of evolution our brains make lots of shortcuts. Red Auerbach might think he’s an expert at basketball, but actually some Cornell professors have collected some data and have proved definitively that everything you thought about basketball was wrong.
Argument 1 is the “Econ 101” idea that when people have money on the line, they tend to make smart decisions, and we should be suspicious of academic theories that claim otherwise. Argument 2 is the “scientist as hero” idea that brilliant academics are making major discoveries every day, as reported to you by Ted, NPR, etc.
In the case of the hot hand, the psychology and economics establishment went with Argument 2. I don’t see any prior reason why they’d pick 1 or 2. In this case I think they just made an honest mistake: a team of researchers did a reasonable-seeming analysis and everyone went from there. Following the evidence—that’s a good idea! Indeed, for decades I believed that the hot hand was a fallacy. I believed in it, I talked about it, I used it as an example in class . . . until Josh Miller came to my office and explained to me how so many people, including me, had gotten it wrong.
So my point here is not to criticize economists and psychologists for getting this wrong. The hot hand is subtle, and it’s easy to get this one wrong. What interests me is how they chose—even if the choice was not made consciously—to follow Argument 2 rather than Argument 1 here. You could say the data led them to Argument 2, and that’s fine, but the same apparent strength of data could’ve led them to Argument 1. These are people who promote flat-out ridiculous models of the Argument 1 form such as the claim that “all deaths are to some extent suicides.” Sometimes they have a hard commitment to Argument 1. This time, though, they went with #2, and this time they were the foolish professors who got lost trying to model the real world.
I’m still working my way though the big picture here of trying to understand how Arguments 1 and 2 coexist, and how the psychologists and economists decide which one to go for in any particular example.
Interestingly enough, in the hot-hand example, after the behavioral economists saw their statistical argument overturned, they didn’t flip over to Argument 1 and extol the savvy of practical basketball coaches. Instead they pretty much tried to minimize their error and try to keep as much of Argument 2 as they could, for example arguing that, ok, maybe there is a hot hand but it’s much less than people think. They seem strongly committed to the idea that basketball players can’t be meaningfully influenced by previous shots, even while also being committed to the idea that words associated with old people can slow us down, images of money can make us selfish, and so on. I’m still chewing on this one.
You use argument 1 or 2 based on whether you think the unfalsifiable construct is beneficial or harmful to the hairless homo economics primates.
Is there a similarity between the hot hand controversy and the somewhat older disagreement, geocentric vs. heliocentric point of contention? With some loose hand waving, there is evidence on both sides of the argument. From https://en.wikipedia.org/wiki/Geocentric_model:
“First, from anywhere on Earth, the Sun appears to revolve around Earth once per day. While the Moon and the planets have their own motions, they also appear to revolve around Earth about once per day. The stars appeared to be fixed on a celestial sphere rotating once each day about an axis through the geographic poles of Earth.
Second, Earth seems to be unmoving from the perspective of an earthbound observer; it feels solid, stable, and stationary.”
Just like the hot hand controversy, people were wrong for a fair amount of time,
“For over a millennium European and Islamic astronomers assumed it was the correct cosmological model.”
To be sure, Copernicus is more well known to the public at large than Joshua Miller and Adam Sanjurjo, although at this time of the year, i.e., March, perhaps that is not true.
The idea of “relativity” is that there is no preferred reference frame, so geocentric vs heliocentric is a matter of preference.
For navigating around the Earth, geocentric is easier to work with, for around the solar system it is heliocentric. If interstellar navigation ever becomes a thing it will probably be Sagittarrius A*, then people will realize the motion of the solar system looks like a vortex (similar to what Descarte theorized).
Also on wikipedia they are still doubting the hot hand:
https://en.wikipedia.org/wiki/Hot_hand
There are settings where human skill plays an important role (e.g., basketball), and there are setting where it does not (e.g., by construction at a roulette wheel). I wonder if people like Tversky had mainly in mind a setting like the latter, but ended up testing the hot hand phenomenon in a setting like the former. Surely it is a fallacy at the roulette wheel, but apparently it’s not on the basketball court…
Maybe there are settings where type 1 economists are right and settings where type 2 are, but hubris might make them believe that the world always behaves the way they think it does.
Part of the “hot hand fallacy” could be philosophical differences in randomness. Coin flips are usually considered random but if you knew everything about the action (i.e. the precise area you strike the coin, the force that you strike it, wind resistance, etc.) you could deterministically know which side it will land on. Similarly, it could be the case that observed shot-making can be accurately modeled by some sort of probability distribution so a scientist may conclude “hot hand fallacy”. But it could be that there is actually a deterministic process going on – just one too complex to model with current data but is intuitive to players.
Is this possible or am I getting something wrong?
Humans are a thin stratum of rationality on top of a large mass of habit. But the habits come from a long, sometimes, thousands of years long process of cultural transmission. (Henrich: Secrets of Our Success, IMO a great book) Henrich makes the case for the Type 1 argument: these habits have a tendency towards good results which operate at a cultural, not an individual rational level. They get can get trapped at suboptimal levels in general and can even be counterproductive when the facts on the ground make them highly suboptimal. Type 2 thinking is about finding those changed circumstances. The ratio of Type 1 to Type 2 thinking is then highly dependent on shifts in the environment which take habits and make them suboptimal. They also depend on *how* suboptimal the circumstances have engendered and on the underlying rate of cutural transmission, which are of course not entrirely independent.
A classic example here is punting on fourth down in football. Coaches learned to do so, and doing so was highly beneficial in low-offense environments. David Romer https://eml.berkeley.edu/~dromer/papers/JPE_April06.pdf came along with a famous paper saying that Coaches were wrong. Over time, coaches have been punting less and less, although still much less than Romer advised. Of course there may be reasons for this which are outside the maximization routine Romer is implementing (like risk aversion on the part of coaches for their jobs.) But the “book” on punting was culturally transmitted and circumstances have definitely changed, as quick strike offenses make field position less important.
See also the dispute over sabermetrics, which is another Type 1 vs. Type 2 thinking battle. Here I think the sabermetricians have much the better argument, as they have explored the conditions under which various strategies are dependent on the offensive era in which they operate, and can argue persuasively that some strategies (bunting) while rational when created, and culturally enforces, became counterproductive when circumstances changed. And since the underlying thought behind the bunt isn’t really rational, but a culturally transmitted “book,” it took a relatively long time for the superior results to come through. But they did, and now the bunt is on its way to becoming moribund. It almost surely would have done so without sabermetrics… so it’s not “scientist as hero,” so much as “scientist advancing the process.”
Jonathan:
Yes, we discussed the 4th down thing recently.
Do people have money on the line on Hot Hand? Is this the moneyball concept? Can teams really profit out of Hot Hand?
My attempt to answer these questions became too elaborate, so I wound up writing an article on my substack which you can find at https://theotherendofthegalaxy.substack.com/p/create-and-harness-the-hot-hand?sd=pf
“ They seem strongly committed to the idea that basketball players can’t be meaningfully influenced by previous shots”
You seem to be using some causal language here that I never understood to be part of the definition of the ‘hot hand’.
In its basic form, isn’t the ‘hot hand’ simply something like ‘a player that has just made a shot is more likely to make the next one than if they had missed.’ Ie no assumptions about some sort of cognitive causal feedback loop?
I’ve always thought of it as just implying that basketball shooting is a nonstationary process and under certain conditions the probability of success can increase and though not necessary, under certain conditions autocorrelation may be positive. Basketball enthusiasts/gamblers probably have internalized their ‘conditions’ where it works, which may or may not be easy to model and test. So even if the original ‘fallacy’ paper didn’t contain a statistical error, it probably just seemed like a ‘strawman’.
The observed effect could be explained by either model. Distinguishing between them might be possible by looking at correctly-aimed-but-blocked shots, if that data is available. People who play basketball and have a good intuitive sense of mechanisms seem to find the causal model more plausible, though I’m not aware of anyone actually surveying them.
Basketball shots are not random draws from a random number generator, they are physical actions taken by apes. “The probability of success” is NOT a physical thing in the world, it is not fixed, and it is not varying, because it does not exist as an object, or a feature, of any physical thing. It’s not like the mass of a coffee cup, it’s not like the thermal energy of the coffee, and it’s not like the frequency distribution of the velocity of the water molecules, which can be verified to have a reliable frequency distribution.
Basketball shots are unique as are the circumstances of the defenders and the positions on the court and etc etc.
To the extent that there is a “probability of success” that probability exists only in the mind of a given observer, and is dependent entirely on the set of facts that observer knows about the scenario at the time of the probability assessment.
People looking for a “hot hand” as if it were objectively visible in the sequence of 0/1 outcomes of recorded shots, or inferrable by some data analysis entirely based on the abstract information of past successes etc are engaging in Jaynes’ mind projection fallacy. The fact is, sometimes a player is “in the zone” and other times they are “having a bad night” and those are physical facts. There is no great coin flip in the sky that determines the outcomes.
DL, I agree with most of your post, but don’t get this: why do you say that probability of success exists only in the mind of an observer? just because we don’t know how to measure the player’s attributes that make a successful shot more or less likely? The physical action taken by the ape must surely depend on his/her state of physical rest, various worries, pain in the wrist, something coach said before the game, etc. These things will matter even without an observer taking notes.
Also these conditions are sure to vary over time. Absence of a hot hand seems as silly a null hypothesis as any other.
In the same vein, note that random draws from a random number generator also exist only in the mind of an observer. (Cf. Jayne’s mind projection fallacy, de Finetti’s PROBABILITY DOES NOT EXIST.)
Thomas:
I like Daniel’s comment! I think Daniel means to say that the basketball shot – or the golfer’s swing or the pitcher’s throw – is not a thing in itself but a sequence of events that occurs in a unique environment every time. Every time Verlander winds up, his physical and mental condition are different, the game situation is different, the temperature and wind are different. All those changes apply to the batter and ump as well. The number of influences is so variable that no probability is possible.
And you say: “Absence of a hot hand seems as silly a null hypothesis as any other.”
+10!! Well said. The HH is a widely observed phenomenon. It’s a commonly experienced phenomenon. Anyone who attempts to execute any activity at a high level knows they have periods of greater fluidity and success, surely because of varying conditions you mention. Performance is likely a function of physical condition (tiredness) and mental clarity (level of ‘background’ mental noise or interference), in addition to overall skill built from years of practice.
Andrew’s characterization of how economists represent their relentless advice to the masses is funny and true, but to my way of thinking it doesn’t capture the mindset that leads to the HHF.
The emergence and temporary unquestioned reign of the HHF is more about the “human cognitive bias” meme that has infiltrated academia. The meme goes that human perception is frail and horrendously distorted, thus we academics need to help humans understand the “right” world. In that memeology, HHF has a lot of company, like Dunning-Kruger, contorted claims about why humans don’t take action on global warming; claims about “privilege” etc. The “words-associated-with-old-people” quote Andrew provides is in a similar vein: we humans are so incredibly stupid!!! we have no idea what’s going on!! We respond stupidly on a massive scale to small cues and don’t even notice it!!!
So here’s what you get:
If humans don’t even recognize how our massive response to words like “wheel chair” affects us, how could we possibly correctly perceive complex phenomena like the HH?? The HH is obviously false, soo…..just a little math here and – yep!! Bingo!! Proved ‘er wrong!!
IOW, the HHF emerged not because the HH is statistically subtle, but because the conclusion was already “known.”
Chipmunk:
Good point regarding the omnipresence of the “human cognitive bias” idea. But I don’t think “why humans don’t take action on global warming” is a good example. Collective action problems are a different thing than cognitive bias.
Andrew:
It seems like there are many examples where collective action was taken successfully in the presence of clear benefit of the action. Banning ozone damaging chemicals is a good example. A better way to think about the climate change issue is that academics/environmentalists overstate the costs and understate the benefits of the policies they endorse, and the public intuitively recognizes that bias.
I’d assert that, to the contrary, taking collective action is one of humanity’s greatest strengths, even when a great deal of sacrifice is involved people will do it if they see the benefit.
Chipmunk:
I agree that people can and do solve collective action problems, but not always, or, at least, sometimes it takes them awhile to get to a solution. Environmental problems are a well-known example where there is difficulty. Climate change is the biggie; other familiar examples are overfishing and draining of aquifers. Anyway, I agree with you that these problems can be complicated; I just don’t think they really fall into the “cognitive bias” category.
Probability of success is what? It’s a judgement about what is likely to happen. When the basketball player takes his Nth shot, that is a sequence of actions which is unique, as chipmunk says. The sequence either succeeds, or it fails, depending on whether the actions result in the ball having initial conditions at the moment of release that take the ball from the hand to the hoop in a way that causes the ball to go through the hoop. There’s nothing but physics involved and at the scale of the basketball there is no need for quantum weirdness… Physics has no probability at this scale it’s just position, and momentum.
Carlos is in a sense correct that even RNGs have probability only in the mind of the observer, but in another sense they have been carefully arranged into an algorithm designed to satisfy the frequentist axioms. So long sequences really do have the expected properties needed for frequentist calculations to come out correct. We can act as if there is probability as frequency built into the system and we make no important errors (cryptographic RNGs even more so).
Ah, I see, you say probability does not exist in general, not just as regards apes shooting basketballs. Sure, in the end events are determined by physical laws, including the roulette, weather phenomena, etc. But you deduce from this that probability “is not fixed, and is not varying”. Why? if probability exists “only” as a mental tool/device, and helps us understand the world, it can be allowed to vary (over time or other variables), why not (isn’t that the whole point?). If we couldn’t talk about it because it’s not a physical object we should close down this blog, the entire field of inferential statistics, but also arts, literature, etc
(esprit d’escalier…)
– I should have said why can’t we allow the phenomena which give rise to a probability assessment to vary…
apologies! i wrote a comment which now I cannot see, and then added a correction…
The comment was something like this: probability is not a physical property but a mental device (agreed), but why would it not be allowed to vary (in fact that’s the point of it)
Daniel L. wrote: “When the basketball player takes his Nth shot, that is a sequence of actions which is unique, as chipmunk says. The sequence either succeeds, or it fails, depending on whether the actions result in the ball having initial conditions at the moment of release that take the ball from the hand to the hoop in a way that causes the ball to go through the hoop.”
Dunno about basketball, but I’m trying to learn backcrosses (club juggling), and I sometimes get fairly long runs, and sometimes have trouble getting to, say, 10 throws (truth in advertising: I’m throwing one backcross every third throw from one hand; a full shower of backcrosses is still a distant dream). It seems to me that a naive probabilistic analysis that assigns an independent probability of success to each throw wouldn’t describe this phenomenon correctly.
That is, after a good throw, the probability of another good throw is way higher than after an iffy throw. Presumably “hot hand” is describing a similar thing. After a good throw, I can remember what my hand/arm felt like as I pulled the club from the place I caught it to behind my back where I then throw it. And repeat that. So once the process gets started, runs can be quite long.
So it seems to me that a basketball player would remember how his/her body felt when s/he made the good shot, and be able to reproduce that better than remembering that feeling at their last practive session.
Thomas, yes of course, Bayesian probability, which is not a property of the physics of the basketball shot, may vary in any way that is appropriate.
I was trying to make clear what is wrong with “probability as frequency, inherent in the object” which is the Frequentist view of statistics. It’s not just a little bit wrong and needing a tweak here and there, it’s a wholly bankrupt form of analysis. It replaces the actual mechanisms of how things actually work, such as neuromuscular control, and position, momentum, and force… With a complete and probably incorrect fiction: a physical mechanism which has the same mathematical description as algorithmic randomness.
If you define the problem as “a sequence of events in which each event is an IID draw from a random generator of success/failure events” then of course the probability doesn’t change, you’ve prevented it by definition. If you allow that the underlying probability can change in a certain way only… Then again it must be definition… In either case you have replaced reality with something fake anyway, might as well if you’re going to lie about reality go whole hog… Include that on special events with some unknown low probability the ball magically becomes a pony if you succeed in the shot. Frequentist has that flavor anyway.
Andrew,
I think it makes sense to consider the possibility that the difficulty of solving a collective action problem is inversely proportional to its collective benefit. If the collective benefit is large and immediate, the problem is easy to solve. If the collective benefit is small and distant, the problem is hard to solve.
The Two Models of Microecon thing is very clear if you’ve read [Inadequate Equilibria](https://equilibriabook.com/toc/). If people with decent visibility on the ground-facts acting on their own behalf without need for co-ordination are doing something weird, they probably know something you don’t. Outside of that case, it’s entirely likely they’re messing up (and perhaps economists can help).
Daniel:
Yes, this is the point. The “people with decent visibility on the ground-facts acting on their own behalf without need for co-ordination” all seem to believe in the hot hand, so this seems like a natural case where economists would follow Argument 1 and start with the assumption that these players know what they’re doing. But in this case, after this one study in 1985 the psychologists and economists all happily jumped to Argument 2 and never looked back. Even after Miller, Sanjurjo, and others pointed out the flaws in that 1985 study, much of the psych and econ establishment stuck with their model that the hot hand doesn’t exist. They seemed much more comfortable assuming that all the players and coaches were wrong, than that some of their academic heroes had made a mistake.
People tend to believe what their peers believe. This applies both to basketball coaches and academics. But mavericks and contrarians, who are wrong most of the time, are valuable because they can get us all out of comfortable complacency when we are wrong. It’s that simple and banal.
Concerning basketball, wasn’t there some study that have researched a very slow adoption of a 3 point shot as a regular tactics despite its statistically provable benefits? Basketball professionals got there eventually, but it have taken them a decade or more. Sorry, no cite
Often times economists’ (or laypeople’s) claim that people are irrational is just a misspecified utility function, so it’s always worth trying to build the best model of behavior. However, when the correct utility function is so clearly quantifiable (people like winning more money when betting on sports) and can clearly demonstrate whether or not the population’s behavior is at odds with that objective, then it’s time to go down path #2. In this case, the ‘clearly demonstrate’ part was a failure, but that’s not to say that the general approach is wrong.
This article is one of the reasons I love your blog.
This seems like the efficient market hypothesis. It can be an useful model even if it isn’t exactly correct.
James:
There are two natural default hypotheses here. One default hypothesis, fitting the efficient market idea, is that the players and coaches know what they’re doing, and the hot hand effect is large and it is real. Another default hypothesis is that shots are independent events and there is no hot hand. What’s interesting to me is that the psychology and economics establishment went with that second default (independent events) rather than the first default (the efficient market).
“… One default hypothesis, fitting the efficient market idea, is that the players and coaches know what they’re doing, ..”
I am not convinced that is the correct analog. There is (was?) a school of investment analysis called technical and practiced by so called technicians. They would look for patterns in stock charts and attempt to discern from them what the stock was going to do next. So they would talk about “head and shoulders patterns” or “resistance levels” and so forth. But reportedly they would happily find such patterns in stock price charts generated randomly. So according to the efficient market guys they were being fooled by the human tendency to see patterns in randomness and did not in fact know what they were doing.
By the way how large is the hot hand effect believed to be? How does the difference between a player having a good day or a bad day compare to the differences in ability among players?
“… how large is the hot hand effect …”
I answer this in the case of NBA free throws at the end of a substack article I just wrote: https://theotherendofthegalaxy.substack.com/p/create-and-harness-the-hot-hand?sd=pf
Part of my conclusion: “… a poor free throw shooter during a hot streak can be as successful as a good shooter at a typical time.”
Also, “I anticipate that comparable field goal analysis will reveal comparable magnitudes.”
I love your concluding remarks comparing the attitude toward priming vs. the hot hand!
Coincidentally I was just writing up some thoughts on the hot hand prompted by a conversation with a past World Series of Poker main event champion who, I was surprised to discover, still thought the hot hand was a fallacy. I’ve linked to this post at the end of my article.
https://theotherendofthegalaxy.substack.com/p/the-hot-hand-in-sports
“Coincidentally I was just writing up some thoughts on the hot hand prompted by a conversation with a past World Series of Poker main event champion who, I was surprised to discover, still thought the hot hand was a fallacy. ..”
Not that surprising to me. Many gamblers believe in lucky streaks (or unlucky streaks) but this is a fallacy when it comes to gambling.
Paul:
Yes, the cards really are dealt randomly (or so I assume!). The mistake is to generalize that to non-random settings such as shooting baskets.
I remember a similar generalization error a few years ago (with follow-up here) when a political pundit made the assumption that campaign polls are a random walk, by analogy to the stock market. Public opinion and the stock market are different! So, even if the stock market can be reasonably approximated in some settings by a random walk, that doesn’t mean that it makes sense to think about polls in that way.
I should have stated more completely, “… still thought the hot hand IN SPORTS was a fallacy.” This despite years of playing basketball and following the NBA.
As for the hot hand in poker, just as with the massive data now available for sports analysis that has more clearly revealed the hot hand’s substantively relevant magnitude, the situation for poker is similar, at least in principle, thanks to televised and streaming poker with hole card cameras, and online games.
We also now have ‘GTO’ (game theoretically optimal) poker engines by which to evaluate the quality of play.
As a result, I wouldn’t be surprised if analysis of sufficient poker data showed similar time variation in the magnitude of (in)accuracy for an individual player’s decisions, giving rise to hot and cold streaks in financial outcomes.
Paul:
Yes, I understood that you were saying that poker players didn’t believe in the hot hand in sports. My guess is they were generalizing from randomness in cards to presumed randomness in basketball shooting.
“…So, even if the stock market can be reasonably approximated in some settings by a random walk, that doesn’t mean that it makes sense to think about polls in that way.”
Well it seems like there are some similarities which can produce effects that look like momentum. New information takes a while to make it to all the voters or market participants. So the reaction may not be instantaneous. And polls and prices may be stale (that is outdated and based on old information). So for example a poll may be based on interviews over the prior 3 days again leading to a delayed reaction to new information. And similarly stock prices are reported based on trades and stocks don’t trade continuously. So again a market average may respond gradually to an event because some of the prices in the average are stale (that is for stocks that haven’t traded since the event).
Of course things happen faster in the market because participants have more incentive to stay current than voters.
James:
Polls tell you how people feel right now, and they can change in predictable ways even without any new information. Respondents to polls are people giving their attitudes; they are not investors.