David Shor writes:

I [Shor] am working on a Bayesian Forecasting model for the Mid-term elections that has two components:

1) A poll aggregation system with pooled and hierarchical house and design effects across every race with polls (Average Standard error for house seat level vote-share ~.055)

2) A Bafumi-style regression that applies national-swing to individual seats. (Average Standard error for house seat level vote-share ~.06)

Since these two estimates are essentially independent, estimates can probably be made more accurate by pooling them together. But If a house effect changes in one draw, that changes estimates in every race. Changes in regression coefficients and National swing have a similar effect. In the face of high and possibly differing seat-to-seat correlations from each method, I’m not sure what the correct way to “blend” these models would be, either for individual or top-line seat estimates.

In the mean-time, I’m just creating variance-weighted averages in excel for seat level estimates and using bayes rule to mix the two seat distributions to get pdfs, which I suspect is sufficient for this particular application. But I’m very curious what the “right” thing to do would be.

My reply:

I’m not quite sure what the right thing to do is here–I’m not following the details of what you’re doing, exactly–but I’ll give you my general advice, which is that it’s usually worth it to create a model for the data and work through the likelihood etc, rather than to create an estimator. If, instead of trying to figure out how to do a weighting, you directly model your data, an appropriate weighting might very well pop our satisfyingly from the posterior distribution. That’s been my experience in various contexts, from elections to radon gas.

Shor clarifies:

The issue is that I have two competing models for these races, one that looks horse races as a state-space model (Random walk with noisy biased observations), and the other being that races follow some sort of year to year random walk like you’ve specified in your house papers.

I suppose they arn’t radically different, and your book would recommend trying to build some greater model that incorporates the two interpretations as special cases or construct some hybrid.

Yes, I think a larger model would make sense. But I understand the short-term goal of having a good weighted average. I suppose that a weighted average, weighting by inverse of forecast variance (which is what would be appropriate if the forecasts were truly independent) would make sense.