With the spread of statistics, quantitative measurement, and data analytics in our culture, the sorts of bad ideas that used to be promoted with misleading qualitative evidence are now being promoted with misleading quantitative evidence.
In the old days, we’d have blurry photos of the Loch Ness Monster, the shroud of Turin, and spoon benders on TV. Nowadays it’s noisy sex ratio statistics, p-hacking in studies of intercessory prayer, and statistical studies of ESP and embodied cognition.
Similarly, when it comes to political propaganda, yes, we still have modern frauds in the Protocols genre, but we also have purported statistical evidence in the service of climate-change denial, vaccine denial, election denial, etc.
There’s an obvious connection between statistics used in bad science and statistics used to mislead people for political reasons. Statistics has prestige, so it makes sense that people will try to use statistics as a convincer. Back in the 1970s when those books on ancient astronauts, Noah’s ark, and the Bermuda triangle were popular, statistics wasn’t such a thing. Similarly, science deniers and frauds back then relied more on quantitative pseudo-evidence: stories rather than numbers.
But I also see another link. Tversky and Kahneman coined the wonderful phrase “law of small numbers” to refer to the thesis that “people view a sample randomly drawn from a population as highly representative, that is, similar to the population in all essential characteristics.” We see this a lot; indeed, Kahneman himself has been subject to this fallacy in showing too much trust in noisy psychology experiments. I’m sure I’ve fallen for it too. I agree with Tversky, Kahneman, Gigerenzer, and others that these sorts of cognitive illusions seem to be part of human nature.
And it goes further than that: excessive belief in studies with “causal identification” that reach “statistical significance” can trickle back into a belief that the theories being promoted apply in each individual case. An extreme example of this belief in zero variance came from the authors of a Yale study a few years ago, as discussed here.
OK, here’s the connection. The real world is complicated. Take a generally true theory, gather lots of data, and you’ll see some patterns that appear to contradict the theory. Or, take a completely wrong theory, gather lots of data, and you’ll see some patterns that are consistent with the theory. That’s just the way things go, it’s what Tversky and Kahneman call “the laws of chance.” But . . . if you have the attitude that there is no chance, that your pet theory will show up in every small sample, then the flip side is that you can take any contradiction and use it to sow doubt. Sometimes this is done by people who are genuinely confused; other times it’s done by hacks. But I think there’s lots of overlap. If you’re a hack, you can convince yourself that weak evidence is strong enough. Fooling yourself can be the first step to success.
Agreed. But for this to work it requires more than means and incentive, both of which are clearly present. It also requires that the victims don’t know any better. The lack of quantitative thinking underlies the effectiveness of these deceptions. As humans, we are prone to invent/follow stories, almost by definition subjecting ourselves to these psychological biases (law of small numbers, laws of chance, etc.). The way we combat this is by using our Type 2 thinking – that takes effort and skills, both of which are in short supply.
This problem will not go away easily. For years I told undergraduates that if they would add a simple regression model to their term papers, they would likely get A grades because their instructors would be overly impressed (as well as not knowing enough to be critical – of course, this applies to classes where statistics is not part of the usual curriculum for the course). A student that actually looks up data and attempts to do something with it – when not asked to do so – usually impresses their instructors enough to get a good grade, regardless of the merits of their quantitative work. And, this is at a college level – imagine what happens in high school, junior high, etc. The inability of our thinking to keep up with the complexity of the world is key to the phenomenon in this post.
For those of you that might quote Gigerenzer here, you might argue that our heuristic ability evolves to handle this complexity. I would disagree. I see our heuristic abilities as working well for many things – things that rely on our biological adaptations, such as playing sports, driving a car, and other things that we have lots of experience with and build on our innate abilities to avoid predators, find prey, etc. I just don’t see our innate ability to understand variation or appreciate the nuances of sample size. So, I’m not a fan of heuristics easily overcoming these problems.
Dale Lehman wrote:
” A student that actually looks up data and attempts to do something with it – when not asked to do so – usually impresses their instructors enough to get a good grade, regardless of the merits of their quantitative work.”
While I do not necessarily disagree with his statement, I, at first, wanted to amend his statement to the following:
“A student that actually generates the data and attempts to do something with it – when not asked to do so – usually impresses their instructors enough to get a good grade, regardless of the merits of their quantitative work.”
However, upon language reflection, because “data” to some is plural, not singular and “their” should possibly be the awkward, “his or her,” and “generate” somehow feels like a synonym for “concocting,” and the first “that’ should be a “who,”—so I submit the following:
“Students who actually acquire data and attempt to do something with the data – when not asked to do so – usually impress instructors enough to get a good grade, regardless of the merits of the quantitative work.”
I agree to that – it is what I intended. “Acquire” may simply (!) mean finding an official data source, downloading the data, and presenting some analysis using it.
So I totally agree with this post and the first comment.
It might generalize a bit to acknowledge that a big problem is that the complexity of the world means it’s very very difficult figure many things out.
I guess that the readers of this blog are universally expert at something (probably a few things) compared to the vast human population (or even the US population.)
And yet I would bet a lot that all of us make mistakes evaluating evidence—even in areas where we qualify as expert. The world is so complex and the work to understand is so hard! It’s certainly happened to me.
——now I guess I go off topic a bit:
My observation (story, not data—also certainly not an original thought) is that our first, maybe fundamental heuristic is a form of tribal identity.
And many tribal leaders have learned that to become and remain leaders these days their stories have to have at least a veneer of science and numbers. What matters to them is getting elected, and/or raking in money, and/or having a great title, and/or having influence, etc. Those outcomes are the “facts” that really matter to them.
Fooling oneself is easy, our basic heuristics help with that, and doing something else is usually both difficult and time-consuming.
Hmm. I hope this comment actually adds something…
“our…fundamental heuristic is a form of tribal identity.”
Perhaps. But people also function on the basis of a set of beliefs for which “tribal identity” serves as a simplified short-cut. IOW, identities are built at some level on a set of underlying beliefs, and the identity is merely a shortcut for expressing a complex set of beliefs.
The debate over mandatory vaccinations can be characterized as a “tribal” conflict. But for many people it is a conflict between the belief of some that an individual has the fundamental freedom to choose what goes into their bodies and the belief that the society should be able to force the individual to accept into their bodies what’s best for society.
Andrew is pointing out that data serves often serves as a political tool. The tribal identity argument does the same. Characterizing a given behavior as “tribal identity” behavior is used to demote the behavior to silliness and hide the actual underlying conflict, which is threatening to the person demoting the behavior.
jim –
I agree that much gets swept under the heading of “tribal” in such a way as to begin bringing diminishing returns as to understanding. “Tribes” behave in a vast variety of ways under varying circumstances and there’s much about that process that we know little about (hence why I have less confidence than it seems you do in the science of evolutionary psychology).
That said, re vaccines – the personal bodily autonomy arguments, imo, should not just be taken at face value. The value people place on that as a motivating philosophy can vary across context (e.g., how do many anti-vaz/pro-bodily autonomy advocates feel about the government’s authority to perform ultrasounds in pregnant women seeking an abortion?). And often those contexts interact with identity-orentired cognition (a broader concept into which I place “tribalism” as a sub-set).
That’s not to dismiss the concern across the board – only to say it doesn’t exist as some kind of stand alone value that works independently (on either side of the vaccination issue I would add) of tribalism.
I don’t think this you could meaningfully first sort people based on their views on an abstracted taxonomy of bodily autonomy questions and then quiz them on their views on vaccines land mandates) and find some kind of one to one correlation. Same would be true with any related political overlay – such as abortion.
Jim:
Yes. One way to see this is that, as of two years ago, there was no big difference between liberals and conservatives on vaccine attitudes; indeed, it was only a very small fraction of the population who opposed vaccines.
One way I sometimes think about this is to analogize voters to a compass, which when activated by a large magnet will line up. The vaccine polarization is one example; another was attitudes toward health care reform, which polarized during the 2009-2010 debate.
A big part of this is political actors seeking to use this tribalism or polarization to their political advantage. I see the political logic when leaders of either side encourage their more fanatical or deluded followers, but as a person who has to live in this society I find it very frustrating. For the anti-vax thing, the problem to me isn’t that there are people who value their independence and resist assertions of authority; it’s that these people are encouraged and empowered to mislead millions of people and to interfere with the function of society.
Jim,
I agree with you wholeheartedly that “tribal identity” is an oversimplification. It would have been better if I’d written “tribal identities” since we all have so many. Also, I certainly don’t intend to demean any of us for having them and using them. Heck, I choose to spend time on this blog because I’ve built up respect for Andrew Gelman and the discussions that happen here. It’s clearly a part of a tribal identity for me.
I also agree that saying tribal identity in an argument with someone will be taken as criticism, but I wasn’t criticizing anyone. I think we all construct these identities all the time, and in many cases they are extremely effective heuristics to use! I, for example, trust that Johns Hopkins has great doctors. (I recently saw one and the experience was poor so I switched to another—so I am comfortable with the idea that not all JH doctors are great.)
But I really think you’re missing something when you mention the debate about mandatory vaccinations. In fact I think that’s a perfect example of tribalism in action. Before 2020 very few people complained about many many mandatory vaccinations, and studies seemed to show that the ones who did were not aligned with political parties. There were antivaxxers on all sides.
Now all things Covid-19 are so politicized that it’s hard to have a reasoned, nuanced discussion about any aspect of it.
Dogen,
I think your basic point is that we evolved as social animals. That has good aspects (e.g., ethics), but also the downside you are talking about.
Could be, although that’s not what I was thinking about. I was more circling around the idea that quantitative reasoning is very hard, even for those of us with interest and training and skills. It’s just not possible for us to reason out everything so we use heuristics to help us get through the day.
This discussion reminds me of something I read a long time ago, I think in an Oliver Sacks essay. Apparently certain brain injuries can short-circuit our emotions. It turns out that people who can’t feel emotion are unable to make decisions. Pure reasoning is just not how we operate.
“bad ideas that used to be promoted with misleading qualitative evidence are now being promoted with misleading quantitative evidence.”
Sure, but “bad ideas” based on “quantitative evidence” have their own special history going all the way back to Malthus and have become wildly popular in the last fifty years, expressed most clearly in the “Population Bomb,” Peak Oil I & II, peak [choose your resource] and others.
Jim,
Sure, this has been around forever—indeed, I’m sure there were some German generals back in 1914 using quantitative evidence to back their disastrous decision to start a world war! I guess my feeling is that there’s more of this than there used to be.
Not that I have any quantitative evidence backing up that claim.
Andrew –
I didn’t see your comment here before I started writing mine below. In the future please check in with me before responding to jim.
jim –
> and have become wildly popular in the last fifty years
….
I’m ever skeptical of arguments that seem might be adjacent to the “kids today”/old man yelling at clouds variety.
Consider, for example, the quantitative arguments related to skull/brain size, race/gender and intelligence.
No doubt, there’s probably a positive correlation between availability of the tools of statistical science and the number of bad stats. I’m willing to guess the number of good stats has grown in larger proportion, and that there isn’t some trend of a higher % of bad stats relative to good stats over the last fifty years.
Joshua:
You write, “I’m ever skeptical of arguments that seem might be adjacent to the “kids today”/old man yelling at clouds variety.”
I used to feel that way, but for some reason this sort of argument has become increasingly convincing to me as I have gotten older.
Andrew –
I’m sure it’s just a coincidence.
Thanks for the reference to Tversky and Kahneman’s original use of the phrase “Law of Small Numbers”. In discussions with others I often used this phrase but in the opposite sense to that of T and K, namely:
“small numbers don’t obey the law of large numbers.”
Andrew wrote:
If you’re a hack, you can convince yourself that weak evidence is strong enough. Fooling yourself can be the first step to success.
Well, Feynman is, no doubt, rotating rapidly.
I love it—I wish I had written it. Come to think of it, I probably will write it.
Bob76
“In the old days, we’d have blurry photos of the Loch Ness Monster, the shroud of Turin, and spoon benders on TV. ”
I am sorry to burst your bubble, but there are whole massively popular TV channels in the United States devoted to that mid-20th-century nonsense (maybe not the spoon benders, and the shroud of Turin tends to only interest reactionary Catholics even though when it first appeared the local bishop investigated and determined that it was a fake). Jason Colavito covers some of those channels. Their audience tends to be over 50, Anglo, male, and less universitied. Ancient astronauts, ghosts, and cryptozoology are massively popular in the United States today.
Sean:
I don’t like my bubble to be burst, but I appreciate that you apologized for it!
I have a question. I work with public health data that governments actively use to decide where to allocate resources. If there is more disease in one region, it is given more resources. But in all the analysis I have done for the governments, I have never made confidence intervals. The argument is that the data captures the entire population.
First of all, it may not capture the entire population. There might be measurement errors. Is this reason enough to draw confidence intervals? Why do I feel uncomfortable without confidence intervals? How do you think about uncertainty when you have population data?
Wannabe:
This question comes up from time to time. Here’s the answer.
Thanks for the reply. I have heard the notion of a population as a sample of future populations before. But why should I think of population data that way? Or rather, when? What if I don’t care to make predictions; just about the current state of the world (say, so I can make resource allocation decisions)? And I don’t understand why or when it is useful/necessary to think of data as coming from a probabilistic process.
Suppose I know the height of every individual in a country. I don’t care about height in other countries. And I don’t care about height of past or future generations. All I want to know is whether the mean height of men in this country, at this point in time, is different from the mean height of women. Where is the uncertainty coming from here? Is it even possible to talk about the extent to which the difference in mean height is due to chance? In this setting, mean height of men and women seem deterministic to me, not probabilistic.
Wannabe:
Yes, if you have your data and that’s all you care about, then that’s it. There’s no uncertainty at all. In the problems I’ve seen, there’s always some interest in things outside the observed data. For example, if you want to make resource allocation decisions, you’re likely to be interested in the future. Things can change between when the data have been gathered and when the resources will be allocated. When nothing is changing, then no need to worry about uncertainty. It all depends on your goals.
Ok this helps my brain.
If you have measurement errors, model them.
In physics, these errors are usually fairly exact. If you use a ruler to measure a length, that’s usually +/- 0.5 mm on a mm graduation; there’s no “tail” where there’s a chance the error could be 2 mm.
Probabilities and ratios don’t need errors: if you put 10 red and 15 white balls into a bag, there’s a 40% chance to pick up a red ball at random, no confidence interval needed.
Error comes in when that ratio isn’t known directly, but needs to be sampled: then you’re modelling the error resulting from randomness with confidence intervals.
If you feel uncomfortable with your data because you see sources of error, think about these sources, quantify them, and add them to your data model?
Thanks Mendel.
Here is a formalization of this. They key concept is the expected precision of an explanation. Roughly speaking, if you screw that high enough you get delusional inference. (The delusionality isn’t in the content of the belief, but in the way information is processed).
https://doi.org/10.1016/j.schres.2020.11.048