Skip to content

We were unfair to traditional pollsters

A couple days ago, Slate ran an article by David Rothschild and myself, “We Need to Move Beyond Election-Focused Polling,” in which we wrote about various aspects of the future of opinion surveys.

One aspect of this article was misleading. We wrote:

And instead of zeroing in on elections, we should think of polling and public opinion as a more continuous process for understanding policy. The Affordable Care Act, or Obamacare, was hampered by polls showing low public opinion, with Republicans seeing it as vulnerable. But because of expensive traditional polling, the pollsters were limited to asking a single question of overall support. At the same time, every single component (except the individual mandate) had strong bipartisan support. Cheaper and more flexible polling methods will allow researchers to study the nuances of public opinion more regularly, providing better insight into what Americans want politicians to do once they are elected.

I think we got the general point right but we weren’t being fair to traditional polling. David elaborates:

We have should have noted this as a problem of media polls or could have talked about any number of issues that are not lucky enough to have dedicated, detailed polling on their issue by a reputable institution (such as tax reform, immigration, and infrastructure, which all have serious misconceptions with the media’s topline polling). But, Kaiser does great polls and could have included this a rare exception of what can be done with a larger budget, or we hope, cheaper polling methods in the future.

We were alerted to this error by my colleague Bob Shapiro, who points us to this post he wrote with Greg Shaw, “Why can’t the Senate repeal Obamacare? Because its policies are actually popular,” and adds:

It is true, and this is maybe a more important point you could have made, that how the press reported on the Kaiser and other polls may have cited only the overall ACA support (and media polls may have asked only about overall support), but Kaiser has been doing precisely what your article calls for, so they were a bit surprised by your critique of ACA polling.

Fair enough.


  1. Michael says:

    Seems to be a classical issue of low reliability. Single-item measures are notoriously unreliable.

  2. Tom says:

    Speaking of nuance, you’re right that all the individual components are supported except for the only one that deals with costs. Maybe expand opinion polls to trade-offs? Would you rather have pre-existing conditions covered or children on your insurance until 26? Would you rather have $100 per month in your paycheck of cover pre-existing conditions? $1000 per month? etc.

Leave a Reply