Peter Enns shares a new article, which states:
The finding that government policy is “virtually unrelated to the desires of the low- and middle-income citizens” (Gilens 2005:789) is one of the most influential social science results of the last two decades. This article offers a new perspective on this finding. I [Enns] show that the seemingly innocuous decision to restrict analyses to data where different income groups’ policy support differs (i.e., a preference gap exists) introduced Simpson’s paradox, leading to misleading conclusions about whose preferences policy reflects. The same concerns apply to analyses of responsiveness to men and women and to partisan groups. I also present evidence that other common approaches for evaluating policy responsiveness can produce equally misleading conclusions. These findings suggest a need to reconsider conventional wisdom about political influence. The conclusion offers methodological recommendations and discusses implications related to understanding social and economic inequality and support for populist candidates.
Enns writes:
I believe this research overturns Marty Gilens’ argument that “actual government policy does not respond to the preferences of the median voter” and Gilens and Ben Page’s conclusion that, “average citizens have little or no independent influence.” I show that the core result in Gilens’ original research stems from Simpson’s paradox (the short videos in this blog post illustrate this paradox with Gilens’ data).
This all reminds me of something we found when working on the Red State, Blue State project: the political differences between red and blue states (and thus between states controlled by Republicans and Democrats) were much more correlated with the political positions of upper-income voters than with lower-income voters. So, from that perspective, it seemed to us that one could observe a pattern in which the positions of elected officials were better predicted by the positions of upper-income voters in their state, without this being directly a responsiveness issue. It could just be correlational, having to do with the interaction between income and geographic variation in party preferences.
I looked at the paper by Enns and had some questions, to which Enns responded. I’ll give my questions below with his responses interspersed:
Me: You seem to have two different criticisms of the work of Gilens and others:
1. When they talk about upper-income voters, it’s pretty much just the upper 20%, not the truly rich.
Enns:
Yes, this is important for our theoretical expectations and mechanisms of influence.
2. There’s the Simpson’s paradox issue you discuss.
Enns:
Yes.
Me: But, substantively, are the Gilens et al. conclusions so wrong?
In your paper, you write, “The probability of policy change does, however, differ when the affluent support a policy more or less than the low-income group. Specifically, proposed policies are more likely to become law (i.e., α is greater) when the affluent prefer the policy more than other income groups.” This sounds pretty close to the general understanding of Gilens etc.
Enns:
There are two important differences:
(1) If this is a policy advantage, it is a completely different mechanism. Gilens (and Gilens and Page), like most other scholarship, focus on responsiveness, which is captured by beta. The conclusion that policy does not respond to the preferences of the middle or low-income voters no longer receives support. Some groups may have political advantages, but past conclusions about unequal responsiveness or some group’s preferences not mattering need to be revised. I think this distinction holds large implications.
(2) More importantly, I don’t think we can interpret this as an advantage at all. Figure 5 touches on this by plotting policy responsiveness to the affluent. When the affluent overwhelmingly oppose a policy, are they really advantaged when the unpopular policy is more likely to become law if they want the policy more than the poor do (orange dots on left side of Fig 5)? Conversely, when the affluent are extremely supportive of a policy, does it benefit them that the policy is less likely to become law when the poor want the policy more (blue dots on right of Fig 5)? I don’t think the different alpha when the affluent support policies more or less than other income groups maps neatly onto any conception of policy advantage as understood by political scientists. But I’d be eager to get your take on this.
Me: Later, you write, “Headlines based on Gilens and Page’s research, such as ‘Rich people rule!’ (Bartels 2014), ‘The Politics of Always Ignoring What Average Americans Want’ (Coy 2014), and ‘Politicians listen to rich people, not you’ (Prokop 2015), align with their original findings. But we have seen that the data cannot support these conclusions.” But it does seem like rich people rule, no? And, sure, politicians listen to all voters, and non-rich voters have influence (look at the recent NYC mayor election where all the rich people in the world weren’t enough to buy the election for Andrew Cuomo), but don’t you think people have always taken these “Politicians listen to rich people, not you” sorts of claims as relative statements?
Enns:
Here’s a two-minute video by Ezra Klein that I think very much aligns with my interpretation of these articles. I think a lot of other coverage takes the no responsiveness to average citizens claims very literally and I believe this really matters. If we take Gilens and Page at face value, there’s nothing the general public can do – policymakers have consistently completely ignored most voters. By contrast, the results in my article suggest: (a) we do not have evidence that policy makers ignore average citizens or even the lowest income group, and (b) we have strong evidence that policy responds to the general public (Table 4), so more political engagement could, theoretically, have an even greater influence.
Me: Also, I agree that these survey data don’t tell us about the actual rich, but, again, it’s hard to imagine that political influence doesn’t go up a lot when you go from the 90th to the 100th percentile of income or wealth.
Enns:
I totally agree about the different political influence on this spectrum, but we should no longer conclude that a study of the 90th income percentile can tell us about the super rich. In other work, Page and his co-authors have made this exact point, writing: “For systematic evidence on the policy preferences of really wealthy Americans—such as the top 1 percent or the top one tenth of 1 percent of wealth-holders—it is necessary to design special surveys that explicitly target those groups” (Page, Bartels, and Seawright 2013, 52).
Me: So, I’m not knocking your methodological point, and it does help make sense of objections raised by Bob Erikson and others regarding the strong claims made by Bartels etc. from those regressions; I just don’t know that the political conclusions are changed so much?
Enns:
If we take the narrowest conclusion of my findings, I think we have to update our understanding of who policymakers respond to (i.e., the interpretation of beta). I think this is important, and as Figure 4d shows, reverses existing conclusions about negative responsiveness to women.
I think this also changes the study of political influence. Different alphas would be a different type of policy advantage that has not been theorized and social scientists need to study and understand. I think it is absolutely implausible that policymakers pay attention to the difference in group preferences, enacting policies when their affluent constituents support a policy more than their middle-income constituents, and not enacting policies when their affluent constituents support the policy less (regardless of the actual total amount of affluent support). Policymakers don’t have this level of data and it seems theoretically absurd. If policymakers were considering group opinions, wouldn’t they look for policies with broad support across groups instead of the difference between groups? Something else must account for alpha.
Ultimately, I hope three things change in the field based on this article.
1. Scholars no longer conclude that there is near-zero responsiveness to the economic majority and to women.
2. Scholars find new ways to study responsiveness, so current methodological misinterpretations no longer happen.
3. Scholars start to theorize and study alpha. If this is a policy advantage, what accounts for it? If it’s not a policy advantage (because unpopular policies are more likely to pass when the affluent prefer them more and popular policies are less likely to pass when the affluent prefer them less), what does alpha represent?
I think these would be large in important changes for the field, but it’s okay if they aren’t.
I’ll admit – I haven’t read the Enns article (though I skimmed it) and I am not at all familiar with this literature. But I have some immediate skeptical reactions that are not assuaged by casually reading the piece or the posted dialog above. Who are the policy makers – elected or appointed people? How is the difference between policy views and actual policies addressed – e.g., one question concerns whether you favor cutting Medicare expenses, but I don’t know of any policy proposal that is simply expressed as cutting Medicare or not – rather, the policies are a complex matter of overall Medicare spending, and who and how that money is spent? Are policy-makers’ views measured by stated opinions or actions taken (since these too often differ)? Most actual policy is also complex with effects buried in the details – so the actual policy enacted may be quite different than the headline portrayal of that policy. Further, there is the issue of stated policies vs actual policies: politicians and appointed officials lie (too often), so is the responsiveness to different parts of the income distribution measured in relation to the stated policy views or the actual proposed policies? Then, the actual impact of policies is rarely clear cut – even policies designed to help a particular group may have indirect consequences that hurt that group. For example, the recent federal tax cuts have differential impacts on different parts of the income distribution, and there are knock-on effects regarding what government services will be provided given the reduced tax collections. It is widely claimed that these tax reductions favor the rich more than the poor, but it is far from clear whether the poor benefit or not once these further impacts are considered.
I’m sure that researchers have addressed some of these concerns. Given that I have not read any of this literature, consider this an uninformed opinion. But this myriad of measurement issues seems at odds with the clearly stated issue of whether policy-makers care about views of the poor, median voters, or the rich.
+1. We can sidestep some of these issues and get a better understanding of what the intentions were with a given policy, and who benefits, from looking at historical documents, not doing statistical analyses of survey data.
Policy benefits the uber-rich and large corporations. See e.g. Gabriel Kolko’s The Triumph of Conservatism, among other works.
“But it does seem like rich people rule, no?”
Boy, you really don’t want to take the scientific conclusion seriously, do you?
Total:
What do you mean? What aspect of the scientific conclusion do you think I’m not taking seriously?
I mean that you continued to insist multiple times to Enns that the conclusion has to be right, even if the stats are wrong.
“Sure, the science is wrong, but I just *know* that the conclusion is right” is vibes and you would rightly be critical of someone who did that on another issue.
Total:
A key part of Enns’s paper is that the data aren’t there to address the question of the influence of rich people. Those analyses just address the top 20% or top 10%, which. The statement, ” it does seem like rich people rule, no?”, is not at all in contradiction with what Enns wrote.
To put it another way, you write that I “continued to insist multiple times to Enns that the conclusion has to be right, even if the stats are wrong.” What is the “conclusion” that I was insisting on, and what are the “stats” that say I’m wrong?
Enns is saying that study underpinning the conclusion that rich peope have more influence is fatally flawed. Absent that conclusion, what does a good scientist say?
The issue is that’s not what the allegedly flawed study (I haven’t thought hard about it) is about. The flawed study claims that poor people have NEARLY ZERO influence, and Enns’ argument calls that into question. It does not contradict either that people with higher incomes have more influence, and it does not say anything about the ultrawealthy.
Whether it is “scientific” is exactly the question under discussion in this blog all the time. Funny how everything that agrees with you is science and everything that disagrees with you is not.
Irrespective of what Andrew even meant by that and taking your interpretation in the most charitable light: don’t you think it is good to challenge a scientific conclusion with your perceived daily reality, esp when communicating directly with the author of said scientific contribution?
Oh, go on, take my interpretation in an averagely charitable light. Andrew’s the one holding the scientific conclusion even after serious issues with it are raised — and he’s not critiquing Enns’ work, which I would expect, he’s just announcing that he still thinks the conclusion is correct.
This is a classic “I like the conclusion being undercut and so I’m going to stick with it.” Imagine a similar conversation where Andrew was saying that despite the Wakefield article being undercut, he still really thinks that vaccines cause autism. Just challenging a scientific conclusion, after all!
Total:
I don’t see at all how the statement, “it does seem like rich people rule, no?”, is undercut by a study that does not have data on actual rich people. My understanding that “it does seem like rich people rule, no?” was not based on the papers by Gilens, Bartels, etc., which, again, is not about rich people.
Contra your comment, Enns does not offer “a scientific conclusion” that rich people don’t rule or that the preferences of rich people aren’t disproportionally represented in political outcomes. What Enns said is that there’s evidence that the preferences of lower-income voters don’t count as zero. This does not contradict “rich people rule.” By saying “rich people rule,” I’m not saying that lower-income voters have zero influence.
Again I’ll humor you: science is not dogma, as the above article points out. These are fallible people trying to model reality.
I read the above interaction as not contradicting the author but prodding him to explain more on how Enns’ contribution squares vis a vis his (andrew’s) prior. It read as genuinely collaborative in my mind. I encourage you to try to find this collaborative spirit in your own life.
All research designs are an abstraction from a real problem.
Is it the job of the researcher to convince readers that the abstraction they present answers the questions they are posing. You can be unconvinced of the design and its conclusions while still taking the research “seriously.”
It kind of seems like Enn wants to address two things:
1. The specific claims of the original Gilens article
2. The “conventional wisdom about political influence”
What they refute empirically is from (1) the specific claim that the influence of the non-rich is actually very close to 0. Their research still does suggests the wealthy still have a disproportionate influence, and as they point out sheds very little information about the ultra-wealthy. It seems like in their mind, that also refutes (2), but you (and I as well) think the “conventional wisdom about political influence” is in fact that the wealthy have a disproportionate political influence but the bulk of the population do still have a non-zero influence.
This is a really strange quote from them
Uh, yeah? I would interpret that as an advantage…
This literature is adjacent to areas I’ve worked in, so I’ve read many of the key pieces over the years. I have always been troubled by the methodological issues in Gilens, which have been pointed out here, although the claim of elite rule doesn’t strike me as anomolous when I read the daily news. Or to put it more precisely, I think Gilens et al. both overstate and understate elite influence.
Overstate: Conflating the 1% and the 20% is a huge problem. The distinction between policy views and policy outcomes also struck me as very important. And the very large overlap between the two preference sets looms over the whole thing.
Understate: By far the biggest issue here is selection bias in the policy space. What questions get asked of respondents? Some version of the issues that are on the table — Overton if you want. But how are these chosen? My hypothesis, which comes from just staring at the world and not any sort of careful analysis, goes like this: The 1% rule, but they disagree with each other. So: no policy gets on the agenda unless it has non-negligible support from a faction of the 1%. That means politics is about all those topics that are on the agenda but do not have unanimous elite support, which is just about everything that makes the cut. Then we have lots of different influences on the outcome, such as weight of 1% support, primitive (prior to focused influence) 99% support, and the political effectiveness of proponents on each side. (a) There are a lot of fuzzies to pin down here, such as what is non-negligible, is it really the top 1% or some other slice, how should “primitive” be understood in a world suffused with influence, and how do we incorporate other meaningful divisions within the population? (b) So what would be a research program to test these various components? How would we know if all this is just an illusion?
Fortunately or not, political science has never been my business, so I think these questions but never do anything with them.
I think using statistics to prove that rich people rule is akin to using statistics to prove that a differentiable function is continuous. What really makes me wonder is whether the old Marxist creed of petite bourgeoisie generally aligning with haute bourgeoisie on the “most important” political points in periods “free of political turmoil” is true.
Jesse Unruh was a powerful California politician who said that ‘Money is the mother’s milk of politics.’ Politicians spend an inordinate amount of time asking people for campaign contributions (I know this from personal experience, having run (and won) in a hotly contested local election, and I’ve heard it from members of Congress). Normally, you do better asking people who have money when you ask for money, and when you ask people for money you end up talking about what you are going to do about the things that matter to them.
I found a trivially simple data set helpful for understanding this issue. In case it benefits anyone else (or in case I made a mistake):
Policy 1: 20% poor favor, 20% rich favor
Policy 2: 30% poor favor, 70% rich favor
Policy 3: 70% poor favor, 30% rich favor
Policy 4: 80% poor favor, 80% rich favor
Suppose policymakers track the rich and ignore the poor. Let the measure of policy change be beta, which for Policy 1 is 0.2, for Policy 2 is 0.7, for Policy 3 is 0.3, and for Policy 4 is 0.8. Considering only policies with differences in preference (Policies 2 and 3), there’s a strong negative correlation between beta and the poor’s policy preferences, which Gilens would interpret as negative responsiveness, even though the responsiveness by construction is exactly zero.
Suppose policymakers treat the 20% rich and 20% poor equally. Then beta for both Policy 2 and Policy 3 becomes 0.5. Considering only policies with differences in preference again, the correlation between beta and the poor’s policy preference is zero, which Gilens would interpret as zero responsiveness, even though the responsiveness by construction is equal to that of the rich.
Also (although not mentioned in the blog), if the latter example were analyzed from the perspective of responsiveness to the rich, the same numerical result emerges: zero correlation, interpreted as zero responsiveness. That’s probably a general conclusion: if all groups are equally-sized and receive equally positive responsiveness, the Gilens analysis would conclude that all groups receive zero responsiveness.
Thank you!
I think a really good data source for studying these questions are the results of referendums, and their demographic analysis. The most data rich case is probably Switzerland but there are also many ballot measures in the US (just not at the federal level).
In Switzerland, laws are made by parliament but are always subject to referendum. Most laws still pass without a popular vote (except constitutional amendments). Referenda are typically called in the most controversial cases. When laws are overturned by referendum (as they frequently are) and it turns out that the lower income groups were more likely to vote No, that would be evidence that the parliament was acting more in the interest of the wealthy. For example.
I think this is a rich research field and I’m always surprised how little attention political scientists seem to be paying to these data. An aside, referendum results do not always align with what people say in surveys. Voters may say that the rich should be taxed more, for example, but they often vote differently when it comes to actually implementing higher taxes. A paradox in need of an explanation, I think.
Referendums won’t directly address the relationship between elected or appointed officials’ views and those of the rich and poor public, since referendums are often (usually?) the result of efforts by various interest groups. I suppose data on whether those officials support or oppose the referendums might shed some light. If the basic question is whether “government policy” reflects the views of the rich or the poor, then referendums are best viewed as attempts to steer or mandate government policy – they are not government policy itself. If you are suggesting referendums are a good source of data on public opinion, then I can see that value. But I have difficulty seeing how to then link that to government policy since referendums are usually attempts to change government policy.
Referendums are influenced by interest groups with money.
Government policy is influenced by interest groups with money.
The media discourse is influenced by interest groups with money.
Surveys and polls are influenced by interest groups with money.
Elections are influenced by interest groups with money.
Elected officials are influenced by interest groups with money. Ironically, the fact that money matters at every step of the political process makes it hard to precisely prove how much it matters.
How do you disentangle all of this? I don’t think you can. Political science studies will never be physics experiments or double blind trials. I suggest using the available data as best we can.
Regarding referendums, we know how the parliamentary parties voted, we know what they campaigned for, we know what the various interest groups campaigned for and how voters voted. Try his is more information than we would otherwise have. So let’s use it!
Kudos to Peter Enns illustrating this issue so thoroughly (in the article) and accessibly (in the blog), really great stuff! This could be used as a really good example Simpson’s paradox in a methods course.
Some observations:
1) The choice to restrict the analyses to >10 percentage point differences was a major modeling decision in these studies. Just a cursory glance at the underlying distributions as presented by Enns might have already hinted at the potential for a Simpson’s paradox. That this was missed in the original study, I can understand, it may well have seemed like a reasonable decision at the time, and we are all vulnerable to having some blind spots. That it took about 20 years, in which the same methodology was used extensively in other studies and gained so much traction in policy debates, to get someone to point out the problem, I have a less sympathetic view of. That seems like a collective failure of the field to me. My hypotheses is that what happened here is a manifestation of what I would consider a larger underlying problem, where a) lack of recognition, understanding and or willingness to engage with model/theory-dependence and inferential uncertainties that follow from it, and b) the desire to – and incentives for – publishing simple, clean stories that fit with preconceived notions in academia (in this case, that lower incomes/women are not listened to at all in politics), combine into a toxic cocktail that impede any self-correcting mechanisms in the scientific process.
2) To me, this episode itself, but also some of the comments here (e.g., “… the conclusion that rich peope have more influence is fatally flawed”) just highlights that we need to do a better job in explaining how scientific knowledge production works, in particular with respect aformentioned inferenctial uncertainty, and the difference between truth and evidence. Because if we as academics fail to articulate these nuances clearly, it should not be surprising that others will equally fail to understand them and spin their own stories.
3) I haven’t thought too deeply about it nor simulated it myself. But I wonder if this whole Simpsons paradox issue here would come to light earlier if instead of restricting the policy sample, the full sample would have been used, with some kind of full parameterization of preferences of the different income groups (and relevant interaction terms of their differences), and then plotting model-predictions over the underlying preference distribution. Would be useful for future responsiveness research if such an approach could be developed.
4) The issue with using top 10/20% etc. as proxy for the policy preference of what might be considered to “true” economic elites (the ultra-rich, the 1% etc.) is really a major problem in all research on economic inequality. I wonder if whether contemporary methods using NLP for sentiment coding based on data that can be dredged from e.g. social media, interviews, press releases etc. could be effective to better measure their preferences. Of course, large differences in (social) media presence among this group, and as mentioned in the discussion, we shouldn’t automatically expect fully homogenous preferences within this group. But I don’t think those issues necessarily preclude the construction of measures with some reasonable reliability in this way.
I’m confused that if we do not consider the preference-gap subsample, don’t we run into the same issue Gilens pointed out? Namely, even if the share of the poor who favor a policy increases, we still do not know whether this reflects only an increase among the poor or a general increase in support across all income groups.