Dsquared’s comment on this entry mentioned the economist Deirdre McCloskey, whom I googled and found this paraphrase of a quote from Don Boudreaux, “that no one was ever convinced by raw data of the truth of a proposition that he or she did not already hold to be true.”
I wonder what the original form of the quotation was. Or maybe what I really wonder is, what is the point of demarcation for which Boudreaux’s statement is true? As written it is certainly false. I know this because I myself get convinced by raw data of propositions on which I held no prior opinion. Here are just a few recent examples:
– Rich people and poor people differ more in their political preferences in poor states than in rich states
– The NYC police department stopped more minorities than whites, even after controlling for neighborhoods and previous crime rates
– Americans have, on average, about 700 acquaintances each
– and many, many others.
And, as a bonus, here’s an example where an analysis of raw data left me unconvinced of a hypothesis (that a local election was rigged).
I certainly did not “already hold” these propositions to be true–or false. In fact, in many cases (for example, the rich and poor states), the interesting proposition didn’t even come to me until the data whacked me on the head with it.
Different sorts of propositions?
How, then, could Boudreaux (and McCloskey?) say such a thing? There must be different sorts of propositions. The propositions that I study tend to be technical–even the studies of voting and police stops are emprical questions, not value judgments. But for the big social questions, maybe it’s hard for people to be persuaded by data because they already have such strong opinions. One thing I like about emprical science (including social science) is that I can be “convinced by raw data of the truth of a proposition that he or she did not already hold to be true”–and it happens all the time.
These sound more plausible, if not as aphoristic: "No one was ever convinced by raw data of the falsity of a proposition that he or she already held to be true," or alternatively, "No one was ever convinced by raw data of the truth of a proposition that he or she already held to be false." Which amounts to a rejection of naive doctine of falsification, I suppose.
Without having much knowledge on the author I would think that it's more of a comment on how questions are often motivated by pre-existing opinions or ideas rather than a hardline stance on the un-objectivity of statistics or analysis.
I agree with you that we certainly often find things we didn't anticipate or set out to find, that's what makes science so interesting. Many great discoveries have been made by 'accident', with the researcher studying something often quite irrelevant to the find.
I guess the point is to be aware that our pre-existing ideas and opinions will drive what we study and to be ignorant to that would be akin to ignoring a covariate.
Vincenze.
I think it's true that for questions of policy (and maybe law) it's very hard to get people to take data into account. I'm living through this right now, in a political process in the city of Berkeley. I'm on the "Creeks Task Force," a citizen's group appointed by the City Council to come up with a new ordinance regulating building near creeks. Everyone agreed at the outset that we should learn about the science of creek buffers and construction setbacks, so we saw hours of presentations about scientific issues: how wide a riparian buffer do you need in order to support fish, to provide water filtration, yada yada. We also agreed that we need to collect data on existing conditions in Berkeley, to see what we have to work with and what condition the creeks are in. The answers, in a nutshell: (1) the bigger the setback, the better, as far as the riparian corridor is concerned, up to about 150-200 feet where diminishing returns set in, and (2) Berkeley's creeks, although not lifeless, are in serious trouble due to not having a healthy riparian corridor, with about 50% of creekside properties having a structure within 30 feet of the centerline.
You might think that these two facts would lend strong support to the argument that Berkeley needs to at least retain, and maybe even strengthen, the 30-foot setback that has been the rule since 1989, but opponents of government "interference" haven't been influenced by the data in the slightest way.
And of course the same is true at the national level, where the Republican response to a budget surplus was "we should decrease taxes because we don't need the money", and then when the surplus became a deficit the response was "we should decrease taxes to stimulate the economy." When the output is independent of the inputs, it's a good implication that the facts don't matter to the people making the decisions.
Your own experience notwithstanding, I remain convinced that Boudreaux was right.
As long as people don't have an interest in the outcome, they'll accept facts and take them into account. I think another relevant quote here is something I've seen as Sinclair's Law: "If a man's livelihood depends on their not understanding something, you can pretty much bank on their not understanding it."
"No proposition about economic behavior has yet be overturned by econometrics."
McCloskey, Donald N. (1985), The Rhetoric of Economics. University of Wisconsin Press, Madison, WI
p. 182
I like Kieran's version of the aphorism. I think I still disagree with it, but it's no longer knock-down wrong.
Brent: Huh? Given that Boudreaux wrote "no one was ever…", this would imply that any of my examples would represent a counterexample! I'd be curious to know how the original quote was phrased. I assume that Boudreaux was thinking of some particular class of propositions (e.g., claims about macroeconomic policy), not trying to make a general claim about science or social science.
Kieran's formulation is better, but it is still hyperbole. Substitute "rarely" for "never" and it's truer, if less attention grabbing.
McCloskey's point about the absence of null results from econometric and statistical reporting holds, though – publication bias is much stronger than most people think. This is a real aid to dogmatism and a barrier to science.
McCloskey also harps on the importance of practical as opposed to statistical signficance in social science research. And the general reluctance of economists to care about the quantitative as opposed to qualitatitive results.
Phil Price:
"As long as people don't have an interest in the outcome, they'll accept facts and take them into account."
As long as people don't have an interest in the outcome, it's really hard to tell what they accept and take into account.