Is it accurate to say, “Politicians Don’t Actually Care What Voters Want”?

Jonathan Weinstein writes:

This was a New York Times op-ed today, referring to this working paper. I found the pathologies of the paper to be worth an extended commentary, and wrote a possible blog entry, attached. I used to participate years ago in a shared blog at Northwestern, “Leisure of the Theory Class,” but nowadays I don’t have much of a platform for this.

The op-ed in question is by Joshua Kalla and Ethan Porter with title, “Politicians Don’t Actually Care What Voters Want,” and subtitle, “Does that statement sound too cynical? Unfortunately, the evidence supports it.” The working paper, by the same authors, is called, “Correcting Bias in Perceptions of Public Opinion Among American Elected Officials: Results from Two Field Experiments,” and begins:

While concerns about the public’s receptivity to factual information are widespread, muchless attention has been paid to the factual receptivity, or lack thereof, of elected officials. Re-cent survey research has made clear that U.S. legislators and legislative staff systematicallymisperceive their constituents’ opinions on salient public policies. We report results from twofield experiments designed to correct misperceptions of sitting U.S. legislators. The legislators (n=2,346) were invited to access a dashboard of constituent opinion generated using the 2016 Cooperative Congressional Election Study. Here we show that despite extensive outreach ef-forts, only 11% accessed the information. More troubling for democratic norms, legislators who accessed constituent opinion data were no more accurate at perceiving their constituents’ opinions. Our findings underscore the challenges confronting efforts to improve the accuracy of elected officials’ perceptions and suggest that elected officials may be more resistant to factual information than the mass public.

Weinstein’s criticism of the Kalla and Porter article is here, and this is Weinstein’s main point:

The study provided politicians with data on voters’ beliefs, and attempted to measure changes in the politicians’ perception of these beliefs. No significant effects were found. But there are always many possible explanations for null results! The sensational, headlined explanation defies common sense and contradicts other data in the paper itself, while other explanations are both intuitive and supported by the data.

Also:

The authors claim that the study is “well-powered,” suggesting an awareness of the issue, but they do not deal with it adequately, say by displaying confidence intervals and arguing that they prove the effect is small. It is certainly not obvious that a study in which only 55 of 2,346 potential subjects complied with all phases is actually well-powered.

My reaction to all this was, as the social scientists say, overdetermined. That is, the story had a bunch of features that might incline me to take one view or another:

1. Weinstein contacted me directly and said nice things about this blog. +1 for the criticism. A polite email doesn’t matter, but it should.

2. Weinstein’s an economist, Kalla and Porter are political scientists and the topic of the research is politics. My starting point is to assume that economists know more about economics, political scientists know more about politics, sociologists know more about sociology. So +1 for the original paper.

3. On the substance, there’s some work by Lax and Phillips on congruence of political attitudes and legislative positions. The summary of this work is that public opinion does matter to legislators. So +1 for the criticism. On the other hand, public opinion is really hard to estimate. Surveys are noisy, there’s lots of conflicting information out there, and I could well believe that, in many cases, even if legislators would like to follow public opinion, it wouldn’t make sense for them to do much with it. So +1 for the original paper.

4. The sample size of 55, that seems like an issue, and I think we do have to worry about claims of null effects based on not seeing any clear pattern in noisy data. So +1 for the criticism.

5. The paper uses Mister P to estimate state-level opinion. +1 for the paper.

And . . . all the pluses balance out! I don’t know what I’m supposed to think!

Also, I don’t know any of these people—I don’t think that at the time of this writing [July 2019] I’ve ever even met them. None of this is personal. Actually, I think my reactions would be pretty similar even if I did know some of these people. I’m willing to criticize friends’ work and to praise the work of people I dislike or don’t know personally.

Anyway, my point in this digression is not that it’s appropriate to evaluate research claims based on these sorts of indirect arguments, which are really just one step above attitudes of the form, “Don’t trust that guy’s research, he’s from Cornell!”—but rather to recognize that it’s inevitable that we will have some reactions based on meta-data, and I think it’s better to recognize these quasi-Bayesian inferences that we are doing, even if for no better reason than to avoid over-weighting them when drawing our conclusions.

OK, back to the main story . . . With Weinstein’s permission, I sent his criticisms to Kalla and Porter, who replied to Weinstein’s 3-page criticism with a 3-page defense, which makes the following key point:

His criticisms of the paper, however, do not reflect exposure to relevant literature—literature that makes our results less surprising and our methods more defensible . . .

Since Miller and Stokes (1963), scholars have empirically studied whether elected officials know what policies their constituents want. Recent work in political science has found that there are systematic biases in elite perceptions that suggest many state legislators and congressional staffers do not have an accurate assessment of their constituents’ views on several key issues. . . . Hertel-Fernandez, Mildenberger and Stokes (2019) administer surveys on Congressional staff and come to the same conclusion. . . . elected officials substantially misperceive what their constituents want. The polling that does take place in American politics either is frequently devoid of any issue content (horserace polling) or is devised to develop messages to distract and manipulate the mass public, as documented in Druckman and Jacobs (2015). Contrary to Professor Weinstein’s description, our results are far from “bizarre,” given the state of the literature.

Regarding the small sample size and acceptance of the null, Kalla and Porter write:

Even given our limited sample size, we do believe that our study is sufficiently well-powered to demonstrate that this null is normatively and politically meaningful . . . our study was powered for a minimal detectable effect of a 7 percentage point reduction in misperceptions, where the baseline degree of misperception was 18 percentage points in the control condition.

So there you have it. In summary:
Research article
Criticism
Response to criticism

I appreciate the behavior of all the researchers here. Kalla and Porter put their work up on the web for all to read. Weinstein followed up with a thoughtful criticism. Harsh, but thoughtful and detailed, touching on substance as well as method. Kalla and Porter used the criticism as a way to clarify issues in their paper.

What do I now think about the underlying issues? I’m not sure. Some of my answer would have to depend on the details of Kalla and Porter’s design and data, and I haven’t gone through all that in detail.

(To those of you who say that I should not discuss a paper that I’ve not read in full detail, I can only reply that this is a ridiculous position to take. We need to make judgments based on partial information. All. The. Time. And one of the services we provide as this blog is to model such uncertain reactions, to take seriously the problem of what conclusions should be drawn based on the information available to us, processed in available time using available effort.)

But I can offer some more general remarks on the substantive question given in the title of this post. My best take on this, given all the evidence I’ve seen, is that it makes sense for politicians to know where their voters stand on the issues, but that information typically isn’t readily available. At this point, you might ask why politicians don’t do more local polling on issues, and I don’t know—maybe they do—but one issue might be that, when it comes to national issues, you can use national polling and approximately adjust using known characteristics of the district compared to the country, based on geography, demographics, etc. Also, what’s typically relevant is not raw opinion but some sort of average, weighted by likelihood to vote, campaign contributions, and so forth.

I guess what I’m saying is that I don’t see a coherent story here yet. This is not meant as a criticism of Kalla and Porter, who must have a much better sense of the literature than I do, but rather to indicate a difficulty in how we think about the links between public opinion and legislator behavior. I don’t think it’s quite that “Politicians Don’t Actually Care What Voters Want”; it’s more that politicians don’t always have a good sense of what voters want, politicians aren’t always sure what they would do with that information if they had it, and whatever voters think they want is itself inherently unstable and does not always exist independent of framing. As Jacobs and Shapiro wrote, “politicians don’t pander.” They think of public opinion as a tool to get what they want, not as some fixed entity that they have to work around.

These last comments are somewhat independent of whatever was in Kalla and Porter’s study, which doesn’t make that study irrelevant to our thinking; it just implies that further work is needed to connect these experimental results to our larger story.

16 thoughts on “Is it accurate to say, “Politicians Don’t Actually Care What Voters Want”?

  1. “To those of you who say that I should not discuss a paper that I’ve not read in full detail, I can only reply that this is a ridiculous position to take. We need to make judgments based on partial information. All. The. Time. And one of the services we provide as this blog is to model such uncertain reactions, to take seriously the problem of what conclusions should be drawn based on the information available to us, processed in available time using available effort.)”

    This is one of my favorite genres of post on the blog.

  2. Kalla and Porter posted their research to the New York Times Opinion column…
    https://www.nytimes.com/2019/07/11/opinion/politicians-voters.html

    … so I feel like a similar rebuttal is appropriate. Lynn Vavreck and Chris Tausanovitch are both political scientists and can argue from an equal field.
    https://www.nytimes.com/2019/07/17/upshot/politicians-mostly-follow-voters.html

    In research terms, it feels like the words being used; “don’t care” and “what voters want”, are really broad and slippery concepts that allow weird language tricks to happen. I don’t see how instantaneous polling information represents voter choices at election time or choice to donate or decision to write congressmen. To believe a statement like “politicians don’t care”, I’d want to see how politicians did something against the platforms they ran on that got them voted in.

  3. I think there are plenty of politicians that do care about voter opinions. The passage of the Evidence-Based Policy Act was an indication that it is the quality of evidence that can derail decisions and judgment, including voter opinions.

  4. I haven’t read the linked paper which could probably address this, but without reading it the most critical issue to me here seems to be measurement, namely the measurement of “accuracy of perception of the constituent’s opinions by the legislators”. I’d expect this to be very hard if not impossible.

  5. Seems like a better title for the paper would have been”we couldn’t convince busy politicians to open our emails or to trust us over their existing sources”.

  6. I found this passage in the NYT op-ed about the authors’ methodology a bit unsettling:

    We gave each legislator a unique username and password, which allowed us to track exactly who used the site and for how long. Shortly thereafter, we surveyed the legislators who went to the site about their perceptions of their constituents. We then compared the accuracy of those who’d received the district-specific information with those who saw information only about the four census regions. (To unobtrusively measure representatives’ perceptions in the follow-up survey, we asked a colleague of ours to distribute it, with no reference to us or District Pulse.)

    Secret tracking of legislator web use (apparently). Deception-by-omission in the follow-up survey.

    Not outrageous, but a bit unsettling.

  7. The paper has a piranha problem in reverse.

    The authors fault legislators for not responding quickly and decisively to a weak priming stimulus. So the paper could be seen as evidence against the nudge literature.

  8. It’s common to be required to make a judgment with incomplete information, but this also seems like a case where it’s perfectly fine for you not to have an opinion. As it happens, you are a political scientist and have studied this sort of thing, but that makes it all the more useful for you to apply your domain-level expertise after reading the paper. It’s fine for you to model your own process, but it’s also fine that you didn’t come down on own side or another because you didn’t have to make a decision depending on the dispute prior to reading the paper.

  9. My (perhaps cynical) impression is that most politicians’ first priority is usually to get elected or re-elected. What their constituents want is is a lower priority. So the results claimed don’t surprise me at all, and wouldn’t even if the methodology seemed top notch.

    • Yes!

      Our society’s model for campaigning is that politicians will take (or at least be aware of) popular views in order to get more votes. But there are multiple paths to increasing vote share–making your opponent look bad, appealing to party loyalty, making yourself more likable, boosting your name recognition, etc.–and many of these are easier and cheaper and more powerful, especially in combination, than studying constituents’ views. Other assumptions implied by the idea that politicians should care about our views are also oversimplifications: not every voter’s views matter–only the ones who will turn out; not every view of a voter matters–people often vote on one issue; people often don’t vote for the person on the ticket–many vote for the party, not the person. Plus there are factors like the race, sex, age, and favorite football team of the candidate. Besides, incumbency is at least as powerful a determinant of how people vote than political positions. With all these other factors, spending scarce time and money on learning popular opinions is just inefficient.

    • +1 +1

      A common tactic is to use voters fears about a single hot button issue to push through a bunch of stuff that voters don’t like, but aren’t willing to stand on – in effect saying to voters, “I don’t care if you don’t like my position on Issue A B and C, because you wouldn’t dare vote against me about Issue D !!!”

      • +1
        Politicians have to have opinions A, B, C publicly, but if they know that Issue D is the killer issue they don’t really have to care about issue A, just have a prepared opinion to spout. But I’m falling into the trap of tarring everyone with the same brush – I’m sure there are politicians who care, in the same way that I’m sure there are politicians who couldn’t give a hoot as long as they are in power. And caring about an issue – my definition of ‘to care’ might be very different from a politicians….

        • “But I’m falling into the trap of tarring everyone with the same brush”

          That’s OK. For the most part it works. Mostly, I don’t think they care at all about voters opinions. They get elected because their personal opinion happens to align with voters on a single issue or two. IMO when they claim they’re only doing what their constituents want, that’s only a matter of convenience, or just an outright lie.

          It’s not totally clear if or how much they *should* care about voters opinions. If they constantly shift their position to align with popular positions, they get labeled as “spineless” or lacking conviction of one’s views.

          IMO Trump’s election came in part because Hilary frequently shifted her position to accommodate momentary prevailing winds both times she ran, earning herself a reputation of lacking conviction and only wanting power.

        • Once in a while the public outrage over an issue is so great that they have to accommodate it to stay in power. When Obama proposed military engagement to dislodge Assad, the outpouring of opposition was so great he was forced to back down. In the end he ignored the will of the people anyway. He just shifted less public operations to support the Syrian rebellion.

  10. Caveat in advance: haven’t read the paper in detail, but from the description it seems like they’re conflating geographic constituency and people legislators listen to. It’s likely true that legislators don’t have a perfect sense of what their constituents want. Not clear why they would optimize to understanding average constituent opinion. The old story about this (Fenno, I think) is that legislators have concentric circles of influence/support (friends/family/core backers; activists and core co-partisans; co-partisans and reliable voters; everyone else in the constituency).

    Information-flows are going to be biased, and understandably so. Legislators don’t care about the average opinion of their constituency; they care about the opinion of their winning coalition, which is hard to define in studies like this. It’s not always the same as Republicans listen to Republicans. Closer would be Republicans listen to Republican voters with a past history of voting (for them, if they can work that out), skewed toward positions they’re exposed to a lot by activists/insiders/lobbyists/etc.

    This isn’t an uninteresting result, but it’s definitely over-selling.

Leave a Reply

Your email address will not be published. Required fields are marked *