Looking at the polls: Time to get down and dirty with the data

Poll aggregation is great, but one thing that we’ve been saying a lot recently (see also here) is that we can also learn a lot by breaking open a survey and looking at the numbers crawling around inside.

Here’s a new example. It comes from Alan Abramowitz, who writes:

Very strange results of new ABC/WP poll for nonwhite voters

There’s something very odd going on here.

See the table below provided by ABC News. [I’ll put the table at the end of the post.—ed.] They show that Clinton leads Trump by 89-2 among African-Americans and by 68-19 among Hispanics. But then they report that she only leads by 69-19 among all nonwhites. That makes no sense. Trump would have to have a huge lead among the other groups of nonwhite voters, mainly Asian-Americans, to produce that overall result among nonwhites.

Let’s assume that nonwhites are 28 percent of likely voters. And let’s assume that blacks are 12 percent, Hispanics are 11 percent and Asian/other are 5 percent.

According to my calculations, among the nonwhite 28 percent of the electorate, they have Clinton leading Trump by 19.3 to 5.3, a net advantage of 14 percentage points. Among the African-American 12 percent of the electorate, they have Clinton leading Trump by 10.7 to 0.2. And among the Hispanic 11 percent of the electorate, they have Clinton leading 7.5 to 2.1. Adding up the numbers for African-Americans and Hispanics, for that combined 23 percent of the electorate they have Clinton leading 18.2 to 2.3 for a lead of 15.9 percentage points. But remember, they only have Clinton leading by a net 14 percentage points among nonwhites. So in order to get to that result, Clinton must be down by a net 1.9 points among the remaining nonwhite voters. That means she would be LOSING to Trump among those other nonwhite voters by a landslide margin, something like 60 to 20!

Now my assumptions about the African-American, Hispanic and other nonwhite shares of the overall nonwhite electorate could be off a little, but probably not by much. And even if you modify those assumptions somewhat, you are still going to be left with the conclusion that Trump is far ahead of Clinton among nonwhites other than African-Americans and Hispanics.

If we flip the results for nonwhites other than African-Americans and Hispanics, giving Clinton a 60-20 lead rather than a 60-20 deficit, which would certainly be more realistic, this would make a noticeable difference in the overall results of the poll, moving the numbers from a 2 point Clinton lead among all likely voters to closer to a 5-6 point overall lead.

I responded:

What do you think happened? Maybe they used different adjustments for toplines and crosstabs?

Abramowitz said that, given the information that was currently available to him, “I have no idea what they did but I can’t come up with any way that these numbers add up.”

Just to be clear: I’m not saying these pollsters did anything wrong. I have no idea. I’ve not seen the raw data either, and I didn’t even go through all of Abramowitz’s comments in detail. My point here is just that, if we want to use and understand polls, sometimes we have to get down and dirty and try to figure out exactly what’s going on. Mike Spagat knows this, David Rothschild knows this, and so should we.

And here’s that table:

image001

13 thoughts on “Looking at the polls: Time to get down and dirty with the data

  1. Isn’t the key here the footnote?
    “* Early and late September polls combined” – which applies only to their “Blacks” and “Hispanics” categories. Looks like they combined the results from two polls for those categories, presumably due to small sample sizes, while all the other numbers are from one poll only, presumably the late September poll. only.

  2. Thinking about it more, Trump has performed well among people who identify as “American” instead of white. Iirc, people who write in non-standard responses to census questions on race are classified as “other”.

Leave a Reply

Your email address will not be published. Required fields are marked *