Comparison of forecasts for the 2010 congressional elections

Yesterday at the sister blog, Nate Silver forecast that the Republicans have a two-thirds chance of regaining the House of Representatives in the upcoming election, with an expected gain of 45 House seats.

Last month, Bafumi, Erikson, and Wlezien released their forecast that gives the Republicans an 80% chance of takeover and an expected gain of 50 seats.

As all the above writers emphasize, these forecasts are full of uncertainty, so I treat the two predictions–a 45-seat swing or a 50-seat swing–as essentially identical at the national level.

And, as regular readers know, as far back as a year ago, the generic Congressional ballot (those questions of the form, “Which party do you plan to vote for in November?”) was also pointing to big Republican gains.

As Bafumi et al. point out, early generic polls are strongly predictive of the election outcome, but they need to be interpreted carefully. The polls move in a generally predictable manner during the year leading up to an election, and so you want to fit a model to the polls when making a forecast, rather than just taking their numbers at face value.

Methods

Having read Nate’s description of his methods and also the Bafumi, Erikson, and Wlezien paper, my impression is that the two forecasting procedures are very similar. Both of them use national-level information to predict the nationwide vote swing, then use district-level information to map that national swing onto a district level. Finally, both methods represent forecasting uncertainty as a probability distribution over the 435 district-level outcomes and then summarize that distribution using simulations.

I’ll go through the steps in order.

1. Forecast national vote share for the two parties from a regression model using the generic ballot and other information including the president’s party, his approval rating, and recent economic performance.

2. Map the estimated national swing to district-by-district swings using the previous election results in each district as a baseline, then correcting for incumbency and uncontested elections.

2a. Nate also looks at local polls and expert district-by-district forecasts from the Cook Report and CQ Politics and, where these polls forecasts differ from the adjusted-uniform-swing model above, he compromises between the different sources of available information. He also throws in other district-level information including data on campaign contributions.

3. Fit the model to previous years’ data and use the errors in those retrospective fit to get an estimate of forecast uncertainty. Using simulation, propagate that uncertainty to get uncertain forecast of elections at the district and national levels.

Step 2 is important and is indeed done by the best political-science forecasters. As Kari and I discuss, to a large extent, local and national swings can be modeled separately, and it is a common mistake for people to look at just one or the other.

The key difference between Nate’s forecast and the others seems to be step 2a. Even if, as I expect, step 2a adds little to the accuracy of the national forecast, I think it’s an excellent idea–after all, the elections are being held within districts. And, as Nate notes, when the local information differs dramatically from the nationally-forecast trend, often something interesting is going on. And these sorts of anomalies should be much more easily found by comparing to forecasts than by looking at polls in isolation.

4 thoughts on “Comparison of forecasts for the 2010 congressional elections

  1. There's a third forecast that merits attention. The one at Stochastic Democracy (SD). On September 6, they predicted that the Republicans will regain control of the House with an 85% probability if the election were held now (75% if the election is held in November). They predict a gain of 48 seats for the Republicans on election night. The team at SD includes Dartmouth student Harry Enten who used the work of Bafumi et al earlier this year to predict the Republicans would gain at least 50 seats in November. SD uses a different approach from both Silver and Bafumi. The assume the true opinion of the voters is a latent variable modeled as a random walk– a hidden Markov model. I find this puzzling because it seems to me they should use a random walk with drift to include the effect Bafumi discusses in his paper. Enten certainly knows about Bafumi's work. Perhaps they do have a drift term, but don't discuss it.

    SD also compensates for the "House Effect" (bias) among pollsters. Rasmussen exhibits a large House effect on the generic ballot (so is claimed by various parties). While Rasmussen has a large House Effect his PIE (Pollster Introduced Error) is among the smallest of the major pollsters. Rasmussen polls provide good information because you can correct for his bias. The difference between the House Effect and PIE is the same as the difference between systematic and random error.

    I think a comparison of Stochastic Democracy with Bafumi et al is even more interesting than a comparison with Silver. Of course a three-way comparison is even better.

    I'd be extremely interested to hear your thoughts on the Andrew. The election is fast approaching and these forecasts are getting a lot of publicity. I think SD approach seems like the best if they have a drift term.

  2. I'll propose an alternative method for predicting the outcome. My method is unproven and unsupported by research. Start by taking the results of the previous midterm (4 years ago) as the base level for each congressional district.

    Correct for incumbency (both then and now).

    Assess the nationwide change in demographic characteristics. You do that by taking each birth year cohort and adjusting their expected vote totals by subtracting the number of people in that age group who will leave the voting population due to death/disability and adding any expected increase in voter participation.

    Then you apply the D/R split to each cohort to arrive at the change in the overall partisan composition of the electorate.

    Then, adjust for the changes in the fundamentals (presidential preference, economic conditions, etc.). My adjustments for these are much smaller than most forecasters.

    I end up with a prediction that says the expected split in the House will be almost exactly what it was in 2006. The Dem advantages in the demographics and incumbency are wiped out by the fundamentals.

Comments are closed.