My Columbia econ colleague Rajiv Sethi writes:
The first (and possibly last) debate between the two major party nominees for president of the United States is in the books. . . . movements in prediction markets give us a glimpse of what might be on the horizon.
The figure below shows prices for the Harris contract on PredictIt and Polymarket over a twenty-four hour period that encompasses the debate, adjusted to allow for their interpretation as probabilities and to facilitate comparison with statistical models.
The two markets responded in very similar fashion to the debate—they moved in the same direction to roughly the same degree. One hour into the debate, the likelihood of a Harris victory had risen from 50 to 54 on PredictIt and from 47 to 50 on Polymarket. Prices fluctuated around these higher levels thereafter.
Statistical models such as those published by FiveThirtyEight, Silver Bulletin, and the Economist cannot respond to such events instantaneously—it will take several days for the effect of the debate (if any) to make itself felt in horse-race polls, and the models will respond when the polls do.
This relates to something we’ve discussed before, which is how one might consider improving a forecast such as ours at the Economist magazine so as to make use of available information that’s not in the fundamentals-based model and also hasn’t yet made its way into the polls. Such information includes debate performance, political endorsements, and other recent news items as well as potential ticking time bombs such as unpopular positions that are held by a candidate but of which the public is not yet fully aware.
Pointing to the above graph that shows the different prices in the different markets, Sethi continues:
While the markets responded to the debate in similar fashion, the disagreement between them regarding the election outcome has not narrowed. This rasies the question of how such disagreement can be sustained in the face of financial incentives. Couldn’t traders bet against Trump on Polymarket and against Harris on PredictIt, locking in a certain gain of about four percent over two months, or more than twenty-six percent at an annualized rate? And wouldn’t the pursuit of such arbitrage opportunities bring prices across markets into alignment?
There are several obstacles to executing such a strategy. PredictIt is restricted to verified residents of the US who fund accounts with cash, while trading on Polymarket is crypto-based and the exchange does not accept cash deposits from US residents. This leads to market segmentation and limits cross-market arbitrage. In addition, PredictIt has a limit of $850 on position size in any given contract, as well as a punishing fee structure.
This is all super interesting. So much of the discussion I’ve seen of prediction markets is flavored by pro- or anti-market ideology, and it’s refreshing to see these thoughts from Sethi, an economist who studies prediction markets and sees both good and bad things about them without blindly promoting or opposing them in an ideological way.
Sethi also discusses public forecasts that use the fundamentals and the polls:
While arbitrage places limits on the extent to which markets can disagree, there is no such constraint on statistical models. Here the disagreement is substantially greater—the probability of a Trump victory ranges from 45 percent on FiveThirtyEight to 49 percent on the Economist and 62 percent on Silver Bulletin.
Why the striking difference across models that use basically the same ingredients? One reason is a questionable “convention bounce adjustment” in the Silver Bulletin model, without which its disagreement with FiveThirtyEight would be negligible.
But there also seem to be some deep differences in the underlying correlation structure in these models that I find extremely puzzling. For example, according to the Silver Bulletin model, Trump is more likely to win New Hampshire (30 percent) than Harris is to win Arizona (23 percent). The other two models rank these two states very differently, with a Harris victory in Arizona being significantly more likely than a a Trump victory in New Hampshire. Convention bounce adjustments aside, the correlation structure across states in the Silver Bulletin model just doesn’t seem plausible to me.
I have a few thoughts here:
1. A rule of thumb that I calculated a few years ago in my post, Is it meaningful to talk about a probability of “65.7%” that Obama will win the election?, is that a 10 percentage point share in win probability corresponds roughly to a four-tenths of a percentage point swing in expected vote share. So the 5 percentage point swings in those markets correspond to something like a two-tenths of a percentage point swing in opinion, which can crudely be thought of as being roughly equivalent to an implicit model where the ultimate effect of the debate is somewhere between zero and half a percentage point.
2. The rule of thumb gives us a way to roughly calibrate the difference in predictions of different forecasts. A difference between a Trump win probability of 50% in one forecast and 62% in another corresponds to a difference of half a percentage point in predicted national vote share. It doesn’t seem unreasonable for different forecasts to differ by half a percentage point in the vote, given all the judgment calls involved in what polls to include, how to adjust for different polling organizations, how to combine state and national polls, and how you set up the prior or fundamentals-based model.
3. Regarding correlations: I think that Nate Silver’s approach has both the strengths and weaknesses of a highly empirical, non-model-based approach. I’ve never seen a document that describes what he’s done (fair enough, we don’t have such a document for the Economist model either!); my impression based on what I’ve read is that he started with poll aggregation, then applies some sort of weighting, then has an uncertainty model based on uncertainty in state forecasts and uncertain demographic swings. I think that some of the counterintuitive behavior in the tails is coming from the demographically-driven uncertainties and also because, at least when he was working under the Fivethirtyeight banner, he wanted to have wide uncertainties in the national electoral college forecasts, and with the method he was using, the most direct way to do this was to give huge uncertainties for the individual states. The result was weird stuff like the prediction that, if Trump were to win New Jersey, that his probability of winning Alaska would go down. This makes no sense to anyone other than Nate because, if Trump were to have won in New Jersey, that would’ve represented a total collapse of the Democratic ticket, and it’s hard to see how that would’ve played out as a better chance for Biden in Alaska. The point here is not that Nate made a judgment call about New Jersey and Alaska; rather, a 50-state prediction model is a complicated thing. You build your model and fit it to available data, then you have to check its predictions every which way, and when you come across results that don’t make sense, you need to do some mix of calibrating your intuitions (maybe it is reasonable to suppose that Trump winning New Jersey would be paired with Biden winning Alaska?) and figuring out what went wrong with the model (I suspect some high-variance additive error terms that were not causing problems with the headline national forecast but had undesirable properties in the tail). You can figure some of this out by following up and looking at other aspects of the forecast, as I did in the linked post.
So, yeah, I wouldn’t take the correlations of Nate’s forecast that seriously. That said, I wouldn’t take the correlations of our Economist forecast too seriously either! We tried our best, but, again, many moving parts and lots of ways to go wrong. One thing I like about Rajiv’s post is that he’s willing to do the same critical work on the market-based forecasts, not just treating them as a black box.
I’ve noticed that there are notable differences among the betting markets. It’s interesting to get some insight as to the reasons. Still, why do those differences translate, say, to Predictit, consistently, having Haris a larger favorite than.Polymarket?
Also intersting that the Real Clear betting market aggregate has just recently dropped Predictit from their list.
https://www.realclearpolling.com/betting-odds/2024/president
Something higher resolution than hourly would be much better (eg, to see which market is leading). Looks like tick data and orderbooks are available for polymarket:
https://docs.polymarket.com/?python#trades
I couldn’t find docs for the PredictIt API, but it doesn’t look like historical data is offered:
https://www.predictit.org/api/marketdata/markets/7456
Sounds like resolution is whatever is available in these CSVs:
https://danielkovtun.github.io/rpredictit/
Besides the differences already mentioned, I think it is also worth noting that the contracts at PredictIt have different settlement rules than the contracts at Polymarket.
PredictIt: “ The contract that resolves to Yes shall be that which identifies the individual who receives a majority of the votes of the appointed presidential electors when the Electoral College votes are cast in the 2024 United States presidential election.
In the event that no person receives such a majority, all contracts shall resolve to No.…”
Polymarket: “ This market will resolve to “Yes” if Kamala Harris wins the 2024 US Presidential Election. Otherwise, this market will resolve to “No.”
The resolution source for this market is the Associated Press, Fox News, and NBC. This market will resolve once all three sources call the race for the same candidate. If all three sources haven’t called the race for the same candidate by the inauguration date (January 20, 2025) this market will resolve based on who is inaugurated…”
It seems like not only can the contracts resolve differently, but the timing of the payout could be different. Months different?
Also note that the bid ask spread for these contracts at PredictIt is very wide, over 7 cents the times I’ve looked at it.
If somehow you managed to beat the bid ask spread, hedge your BTC/USD risk, and managed the risk in time value of money based on potential differences in the timing of the payouts, there still isn’t a pure arbitrage here because of the difference in settlement rules.
JD –
Do you have any idea why those differences translate into Predictit consistently offering higher odds for a Harris win?
Hmm, so a 269-269 tie where the House elects Trump in a contingent election would count for Trump winning on Poly market but not on PredictIt, but if Trump is called by the media networks in November but has a major health event and drops out and the EC picks Vance instead, it’s the reverse?
> if Trump were to win New Jersey, that his probability of winning Alaska would go down. This makes no sense to anyone other than Nate because, if Trump were to have won in New Jersey, that would’ve represented a total collapse of the Democratic ticket, and it’s hard to see how that would’ve played out as a better chance for Biden in Alaska.
I feel a bit silly for saying something that Andrew obviously knows, but the purpose of Nate’s model, like most presidential models, is to estimate the probability that a major candidate wins. What it is not designed for, nor should be expected to do, is to give out accurate conditional probabilities conditional on highly unlikely events such as Trump winning New Jersey (~1% chance.)
What would actually need to happen for Trump to win New Jersey? The obvious “common sense” answer would be that Trump wins in a landslide with 11+% margin popular vote victory, the largest since Reagan in 1984. But is that truly a sensical outcome?
Any model, when asked to perform tasks outside of its main goal, will eventually break down. And that is okay. If the biggest problem with Nate’s model is that it gives out weird outcomes in extreme 1% scenarios, then I don’t think it is much of a weakness at all.
Fred:
Indeed, the tails of the predictions are not so important, and I think that Nate’s forecast was just fine for its intended purpose.
The question arises as to why those weird probabilities were there in the first place. I don’t think anyone would, on purpose, set up a model where, if Trump wins New Jersey, he’d be less likely to win Alaska. So this made me wonder what was going on. I think what was happening is that Nate, quite reasonably, didn’t want Pr(Biden wins) to be too close to 100%, and he made sure of that by adding these high-variance terms to the states and the demographic factors. If you throw a bunch of things into a multivariate probability distribution, all sorts of unanticipated things can happen, and sometimes they reveal a problem with the model. Similar issues arose with us when setting up the Economist model.
It can be useful to look at these weird predictions because they can reveal deeper issues with how the model works, and this might yield insights when trying to understand other hard-to-explain predictions such as noted by Rajiv in his post.
Yeah. I think given the already super implausible event of Trump winning NJ, a state specific cause is more likely than a massive nationwide landslide.
I actually wouldn’t be surprised if at least one state does something weird this election, because turnout ends up being way different than projected.
Confused:
From the usual Bayesian principles, my best guess if Trump were to win New Jersey is that it would be a mix of both factors: Trump doing much better than expected nationally, and some additional Jersey-specific factor that would additionally help him in that state. Again from standard Bayesian inference, this would imply that Trump would probably do better than expected in Alaska, but not by as much as in New Jersey. It’s not impossible that, winning New Jersey, Trump could do worse than expected in Alaska, but, as Nate might say, I wouldn’t bet on it.
Hmm, yeah, I guess that makes sense: a national swing red combined with a state specific factor. I was thinking “a national swing that large is essentially impossible so it’d have to be state specific”, but yeah, I guess a smaller national swing + a smaller state specific factor is less implausible.
I actually do see a way where Harris winning Alaska and Trump winning New Jersey could be negatively correlated. If the majority of voters are mad at their local governments, that would favor Ds in red states and Rs in blue states. (NY’s R House pickups in 2022, which otherwise wasn’t great for Rs, might be a real world case of this effect… though Florida was also R favorable.)
Confused:
See my comment above. The election outcome can be seen as a sum of many factors. Some of these factors could have negative correlations between states (as in the example you just gave), but I think the most important factors have positive correlation (inducing national swing) with some at essentially zero (specific state effects). Add these all up and you should get positive correlations. Again, negatively-correlated uncertainties are theoretically possible, but for election forecasting I doubt it, and I’d guess that any negative correlations in the tails of the forecast are artifacts of patches that were put in there to widen the tails of the national aggregate forecast.
Good point, a “hate your local government” effect couldn’t be large enough in this polarized era.