The ecological fallacy rides again

Robin Varghese points to this paper by James Galbraith and Travis Hale, “State Income Inequality and Presidential Election Turnout and Outcomes.” At a technical level, they offer a new state-level inequality measure that they claim is better than the measures usually used. I haven’t read this part of the paper carefully enough to evaluate these claims. They then run some regressions of state election outcomes on state-level inequality, and find that “high state inequality is negatively correlated with turnout and positively correlated with the Democratic vote share, after controlling for race and other factors.” So far, so good, I suppose (although I recommend that they talk with a statistician or quantitative political scientist about making their results more interpretable–I’d recommend they use the secret weapon, but at the very least they could do things like transform income into percentage of the national median, so they don’t have coefficients like “4.58E-06”).

Noooooooooo……

But then they write:

We [Galbraith and Hale] can, however, infer that the Democratic Party has engaged in campaigns that have resonated with both the elite rich and the comparatively poor.

Well, no, you can’t infer that from aggregate results! To state it in two steps:

1. Just because state-level inequality is correlated with statewide vote for the Democrats, this does not imply that individual rich and poor voters are supporting the Democrats more than the Republicans. To make this claim is to make the ecological fallacy.

2. The Democrats do much better than the Republicans among poor voters, and much worse among the rich voters. (There’s lots of poll data on this; for example, see here.) So, not only are Galbraith and Hale making a logically false inference, they are also reaching a false conclusion.

This example particularly bugs me because it’s something my colleagues and I have worked on (see here, or here for a couple of pretty pictures).

To conclude on a more polite note

I don’t want to be too hard on Galbraith on Hale. The essence of a “fallacy,” after all, is not merely that it’s a mistake, but that it’s a tempting mistake–that it seems right at the time. (Otherwise we wouldn’t have to warn people about these things.) Through the wonders of the www, the paper reached me, I noticed the mistake, I’ll notify the authors, and they can fix this part of the paper and focus it more on the inequality measure itself, which is presumably where their paper started in the first place.

P.S. I certainly don’t see my role in life as policing the web, looking for statistical errors. I just happened to notice this one, and the topic is something we’ve been researching for awhile. (And, of course, maybe I missed something important myself here…)

P.P.S. Galbraith and Hale have revised their paper to remove the implication about individual voters. I agree with them that this is an interesting topic to study–the key is, I think, to be able to work in the individual-level poll data to address questions of interest.

4 thoughts on “The ecological fallacy rides again

  1. I'm a bit confused about the use of the ecological fallacy here.

    It seems we know that on average, a relatively poor person is more likely to vote Democratic if they vote at all. When looking to explain this, the authors argue that this is because "the Democratic Party has engaged in campaigns that have resonated with…the comparatively poor."

    As I understand the ecological fallacy, it would be to infer that all poor people are more likely to vote for Democrats, than all the rich when in fact all we know is that a randomly selected poor person is more likely to prefer the Democrats than the a random rich person.

    How could it arise that relative poverty be correlated with voting democratic but individual poor voters are NOT supporting the Democrats more than the Republicans?

    What am I missing?

  2. OneEyedMan:
    The authors were claiming that (1) states with a lot of income inequality tend to have higher Democratic support, and (2) THEREFORE poor and rich voters must tend to vote Democratic.

    It's true that a _possible_ explanation for (1) could be (in absence of other data) that poor and rich voters vote Democratic, while others vote Republican; then, a state with lots of rich and poor voters would have a lot of Dems, while other states would have lots of 'Pubs. But although that _could_ be the answer, it doesn't _have_ to be the answer…and indeed, it happens not to be the case. There is no state in which rich voters vote heavily Democratic.

    The "ecological fallacy" doesn't have anything to do with "all" people in a given group doing one thing or the other; it's the fallacy that characteristics of individuals can be determined from statistical relationships among groups. A famous example was noted by, um, someone or other, who found (back in the early 1900s, I think) that U.S. states with lots of immigrants also had the highest English literacy levels. Does this mean most immigrants were literate? No, in fact most immigrants were not literate (at least in English)…but the states to which immigrants tended to immigrate happened to be those with high literacy rates.

    –Phil

  3. Dear One -eyed:

    A better way of explaining the fallacy would be to say that it may be the case that lower income voters are likely to support the Democratic party more strongly in high inequality sates.

    As for you argument that the rich do not vote Democratic — They do, actually. But there are two kinds of upper income folks in this country: the entrepreneurial rich (who vote Republican)and the professional rich: doctors, lawyers, writers, artists (who increasingly go Democratic — and who were the core of Obama's support in the primary.

Comments are closed.