Why poll numbers keep hopping around by Philip Meyer

Op/Ed from USA Today (10/28/04)

Are the unsteady poll numbers making you queasy? Me, too. But now, a team led by a Columbia University professor has figured out why the Bush-Kerry pre-election polls have jumped around so much.

The polls are trying to capture two moving targets at the same time, and that multiplies the motion. One target is presidential choice. The other, more difficult one, is the composition of the actual electorate — the people who will exercise their right to vote.

While it is a pretty good bet that more than half of the voting-age population will turn out on Nov. 2 (or before, in states that allow early voting), nobody knows for sure who will be in that active half. A study led by Robert Erikson of Columbia’s department of political science analyzed data from Gallup’s daily tracking polls in the 2000 election and found that the tools for predicting voter participation are even more uncertain than the tools for identifying voter choice.

With every fresh poll, people move in and out of the likely-voter group, depending on who is excited by the news of the day. Only now, this close to the election, can we expect the likely-voter group to stop gyrating and settle down.

This is why predicting elections is the least useful application of early pre-election polls. They ought to be used to help us see what coalitions are forming in the electorate, help us understand why the politicians are emphasizing some issues and ducking others. None of that can be done if the measures are unstable.

Years ago, when George Gallup and Louis Harris dominated the national polling scene, pre-election polls focused on the voting-age population and zeroed in on the likely voters only at the last minute.

This year, the likely-voter model kicked in for the USA TODAY/CNN/Gallup Poll with the first reading in January. News media reported both numbers — the likely-voter and the registered-voter choices. But the more volatile likely-voter readings got the most play.

The volatility may be newsworthy, but it’s artificial. Erikson and his colleagues point out that most of the change in support for President Bush “is not change due to voter conversion from one side to the other but rather, simply, changes in group composition.”

Who are the likely voters?

The decision on who belongs in the likely-voter group is made differently by different pollsters. Gallup, which has done this the longest, uses eight questions, including, “How much thought have you given to the upcoming elections, quite a lot or only a little?”

Responses to these questions tend to “reflect transient political interest on the day of the poll,” said Erikson and his co-authors in an article scheduled for a forthcoming issue of Public Opinion Quarterly. That transient interest bobs up and down with the news and creates short-lived effects on the candidate choices. “Observed differences in the preferences of likely and unlikely voters do not even last for three days,” the scholars reported. “They can hardly be expected to carry over to Election Day.”

So what are the polls for?

A likely-voter poll is the right thing to do if all you want is to predict the outcome of the election — but that’s a nonsensical goal weeks before the event. Campaigns change things. That’s why we have them.

It would be far more useful to democracy if polls were used to see how the candidates’ messages were playing to a constant group, such as registered voters or the voting-age population. Whoever is elected will, after all, represent all of us.

Behind all of the campaign hoopla is a determined effort by each party to organize a majority coalition. Polls, properly used and interpreted, could illuminate that process.

For example, a key question in this campaign is the stability of the low-to-middle income groups that have been voting Republican. They are subject to switching because they are tugged by GOP social issues, such as abortion and gay marriage, on one side and by their economic interests, such as minimum wage, health care and Social Security, on the other.

The polls produce that kind of information, but we are blinded to it because the editors and headline writers like to fix on the illusory zigzags in the horse race.

Scoreboard shouldn’t block field of play

Some critics argue that media should stop reporting the horse-race standings. That solution is too extreme. A game wouldn’t be nearly as interesting without a scoreboard. But a game where we can’t see what the players are doing because the scoreboard blocks our view would not be much fun, either.

We are now close enough to the election for the likely-voter models to settle down and start making more sense. Expect the polls to start converging in the next few days, and they will probably be about as accurate as they usually are.

In the 17 presidential elections since 1936, the Gallup poll predicted the winner’s share of the popular vote within two percentage points only six times. But it was within four points in 13 of the 17. It ought to do at least that well this time.

Philip Meyer is a Knight Professor of Journalism at the University of North Carolina at Chapel Hill. He also is a member of USA TODAY’s board of contributors. His next book, The Vanishing Newspaper: Saving Journalism in the Information Age, will be published in November.

4 thoughts on “Why poll numbers keep hopping around by Philip Meyer

  1. Andrew commented:

    I like the phrase, "Scoreboard shouldn't block field of play". That's potentially a classic, along with "A poll is a snapshot, not a forecast".

    A related issue is that most of the voters have already decided who they support. For that 80% (say) of voters, "horserace" coverage will be of most interest. "Issues" coverage will benefit the 20% who still might make up their mind. It's sort of a broadcasting/narrowcasting problem. From this perspective, it's understandable that the news gives horserace coverage. (Setting aside media outlets such as Fox News, the New Republic, the Nation, etc., whose goals are to mold public opinion.)

  2. Andrew commented:

    Yet another issue is that, at least in the short term, pollsters get a benefit from numbers that jump around. The more the numbers jump, the more they're news! There is a perverse incentive–a "moral hazard"–encouraging the use of noisy methodology.

    It's similar to the famous "Places Rated Almanac" ratings of America's top cities. I read somewhere that they change their criteria often (every year?) because, if they didn't, the rankings would never change and they wouldn't make news.

  3. David K. Park commented:

    Once again, they ignore the work of Gelman and King’s “Why are Presidential Election Polls so Variable When Votes are so Predictable.”

Comments are closed.