A Non-random Walk Down Campaign Street

Political campaigns are commonly understood as random walks, during which, at any point in time, the level of support for any party or candidate is equally likely to go up or down. Each shift in the polls is then interpreted as the result of some combination of news and campaign strategies.

A completely different story of campaigns is the mean reversion model in which the elections are determined by fundamental factors of the economy and partisanship; the role of the campaign is to give voters a chance to reach their predetermined positions.

The popularity of the random walk model for polls may be partially explained via analogy to the widespread idea that stock prices reflect all available information, as popularized in Burton Malkiel’s book, A Random Walk Down Wall Street. Once the idea has sunk in that short-term changes in the stock market are inherently unpredictable, it is natural for journalists to think the same of polls. For example, political analyst Nate Silver wrote in 2010:

In races with lots of polling, instead, the most robust assumption is usually that polling is essentially a random walk, i.e., that the polls are about equally likely to move toward one or another candidate, regardless of which way they have moved in the past.

But, as I discussed then in the context of remarks by Silver and Justin Wolfers, polls are not stock markets: for many races, a forecast from the fundamentals gives a pretty good idea about where the polls are going to end up. For example, in the 1988 presidential election campaign, even when Michael Dukakis was up 10 points in the polls, informed experts were pretty sure that George Bush was going to win. Congressional races can have predictable trends too. Political scientists Erikson, Bafumi, and Wlezien have found predictable changes in the generic opinion polls in the year leading up to an election, with different patterns in presidential years and off years. Individual polls are noisy, though, and predictability will generally only be detectable with a long enough series.

Noah Kaplan, David Park, and I have a paper on the topic (to appear in Presidential Studies Quarterly) reviewing the literature and analyzing data from polls during the 2000, 2004, and 2008 elections. We show that, as the campaign progresses, vote preferences become more predictable based on fundamental variables such as political ideology and party identification. This is consistent with a “mean reversion” model in which the campaign serves to solidify latent preferences, but it is not consistent with a random walk model in which a campaign is an accretion of unpredictable shocks.

To many of the readers of this blog, the above is not news. Political scientists have been talking about “the fundamentals” for awhile, to the extent that journalists and other observers sometimes overestimate the importance of the economy in determining the election (for example, here’s a clueless history professor likening the predictability elections to “the law of gravity”). As John Sides explained reasonably, you have to be careful when translating economic numbers into vote predictions.

Still, a bit of old-fashioned random-walk thinking remains in the old-fashioned news media. For example, Michael Kinsley recently wrote:

In 1988, Michael Dukakis, who was ahead in the polls just after the Democratic convention, declared in his acceptance speech: “This election isn’t about ideology. It’s about competence.” Then he proceeded to blow a large lead and lose to George Bush the Elder, who turned out to be a tougher old bird than anyone suspected.

This sort of understanding of campaigns was pretty standard a few decades ago, back when Kinsley was editor of the New Republic, but nowadays we wouldn’t frame Dukakis as having “blown a large lead” but rather that he lost a lead that was effectively unsustainable, given the economic and political conditions of 1988. Nor would we need to characterize Bush Senior as a “tough old bird” for winning this election; it was more about being in the right place at the right time.

To say that Dukakis blew a lead is not quite to buy into a random-walk model, but I think it is close. Given what we know about elections, I think it would be more accurate to say that the 1988 election was Bush’s to lose (and he didn’t).

Anyway, that Kinsley quote is an example of why I think this blog post could be helpful. I’m hoping that, by explicitly stating the random-walk and mean-reversion scenarios, I can make people more aware of the implicit models that underly their stories about campaigns and elections.

17 thoughts on “A Non-random Walk Down Campaign Street

  1. Party identification isn’t fundamental to a person, and in fact you can observe some individual’s (and group’s) party identification changing over the course of a campaign. Am I misunderstanding what you mean by ‘fundamental variable’?

  2. I thought a random walk resulted from a tie series process integrated of order one , which implies at least infinite variance.

    Vote shares etc are bounded [0,1]

    Not same y_t=y_t-1 + e than y_t=e, e~N(a,b)

    • Map [-Infinity,Infinity] to [0,1], for example, with a logistic function.

      Also, keep in mind that there is only a finite amount of time until the election.

      • The first step changes the scale of the measure not the scale of the problem.

        The second point changes the scope of the problem not its nature.

  3. Or do what Nate Silver does. Take a thorough assortment of polls detailing various demographic information about the U.S. voting population, a dash of economic intuition, and an agent-based computational model infused with a cluster of dynamically interacting behavioral rules based on said polls. Stir, mix, simulate a couple thousand times, and voila! Some kind of an electoral result with some degree of statistical significance should simply pop up among a number of other probable outcomes. Hopefully it stands out. How is this actually accomplished? Don’t ask me; I’m an amateur, but I definitely don’t buy the random walk story! Social science is young. The New Turks are on their way!

    • That’s not what Nate Silver does. His model is just a multi-level poll smoother based on a random walk prior. He also has a model seperate model where he regresses past election results on an average of economic indicators that he flashes forward to today. Then he averages the two models.

      There’s no agent based anything.

  4. Pingback: Economist's View: Links for 09-28-2012

  5. From a theoretical point of view, the big difference between financial markets and political campaigns is that there isn’t an arbitrage process that reacts back on polling numbers. There’s nothing to push against polls whose predictions are predictably at variance with likely electoral outcomes.

    • Peter:

      It’s not just that. Also, the survey response is a statement of vote preference at a given time. It does not represent any investment or prediction on the part of the individual respondent.

    • Sure is. If there was some predictable story that was going to happen in the future that was going to change a lot of people’s minds, then the campaign that benefits from that would play it *now*. The News media would talk about the inevitability narrative.

  6. There’s a HUGE difference between “is equally likely to go up or down” (your 1st sentence) and Nate Silver’s “are about equally likely to move toward one or another candidate.” One says that there is all noise and no signal, the other says that there is a signal but a lot of noise; or in alternative terminology, one says it’s only accretion of random shocks, the other says that there is a non-random trend confounded with random shocks. The alternative–the mean reversion theory–maintains that the only thing akin to signal is the mean level, and that the mean becomes less confounded with noise through the election.
    As a Bayesian, you’re unlikely to have put the pure noise hypothesis as the null, and the mean reversion theory as the alternative, but if you had, you’d be guilty of some statistical sin.
    I’m interested in how you model “more predictable”–I don’t know how you’d do it except assuming that the variance of the noise decreases through time, but I expect that would be very difficult to demonstrate. (And challenging to model too. But I’m probably stuck conceptually, and you’re approaching it in a completely different manner.)

  7. How does the presence of intrade.com bets on the election results change things, given that those bets seem to be not very different from financial securities?

    • Michael:

      Betting markets can be relevant for election forecasts but I don’t see them doing much to the polls, which represent public opinion snapshots, not forecasts.

      To put it another way, I think it could be reasonable to model intrade.com prices as a random walk, even while modeling public opinion as a mean-reversion process.

  8. Pingback: Choose your default, or your default will choose you (election forecasting edition) « Statistical Modeling, Causal Inference, and Social Science

  9. Even though a confirmed brain geek on all things behavioral – it does appear that verbal behavior (ideology) does suggest some behavioral (voting) events. Maybe.

    The media as a projection of beliefs-ideology seems indicative. Magical thinking does appear to be a characteristic as well. That is, the myth that individual preferences matter is what we demand from the media. So most technical errors seem to derive from trying to fit ideology, which always comes first, to the facts.

    Seems like Silver confuses noise with a (random) pattern. For indeed, suggesting a “random walk” is a regularity of events.

    Bottom line, the goal of all knowledge is predicting the future. Random walk models don’t predict as well as other political models.

Comments are closed.