Forecasting elections using prediction markets has a theoretical appeal, as people are betting their own money so are motivated to get things right. On the other hand there’s been concern about the thinness of the market, especially early on during the campaign. Thin markets can be easier to manipulate, also when there’s not much action on the bets, the betting odds can be noisy. There’s also a concern that bettors just follow the news, in which case the betting odds are just a kind of noisy news aggregator.
Ultimately the question of the accuracy of betting odds compared to fundamentals-based and polls-based forecasts is an empirical question. Or, to put it another way, the results of empirical analysis will inform our theoretical understanding.
A quick summary of my understanding of past empirical work on election prediction markets:
1. For major races, markets are not perfect but they generally give reasonable results.
2. Markets fail at edge cases, consistently giving unrealistically high probabilities of extremely unlikely events.
3. It’s difficult-to-impossible to compare different forecasting approaches because the uncertainties in the outcomes of different races in a national election are highly correlated; in this sense, a national election is giving you only one data point for evaluation.
The best thing I’ve read on the topic is this article by Rajiv Sethi et al., “Models, Markets, and Prediction Performance.”
What happened in 2022?
The recent election featured some strong pre-election hype on markets, along with the usual poll-based forecasts. This time the polls were very accurate on average, while the markets were a bit off, predicting a Republican wave that did not happen. I’d be inclined to attribute this to bettors following the conventional wisdom that there would be strong swing toward the out-party, which ultimately is a “fundamentals”-based argument that made a lot of sense a year ago or even six months ago but not so much in the current political environment with a highly partisan Supreme Court.
But I wanted to know what the experts thought, so I contacted two colleagues who study elections and prediction markets and sent them the following message:
Here are 4 stories for what happened in 2022:
1. Just bad luck. You can’t evaluate a prediction based on one random data point.
2. Overreaction to 2020. Polls overstated Democratic strength in the past election and bettors, like many journalists, did a mental adjustment and shifted the polls back.
3. Bettors in prediction markets have a distorted view of the world, on average, because they are more likely to consume conservative media sources such as Fox, 4chan, etc.
4. Prediction markets don’t really add information; bettors are regurgitating what they read in the news, and in 2022 the news media pundits were off.
What do you think?
The experts reply
David Rothschild:
– The comment section in PredictIt is dominated by the right, politically. Obviously comments are just a portion of traders, but this likely has some effect on some traders. This is especially important because PredictIt is constrained in trade size per person, so in some markets the price looks a little closer to the average trader than the marginal trader (i.e., very confident person cannot swoop in and correct biased market).
– Markets tend to converge towards polls late in the cycle, so while they provide information early in cycle and when information is breaking, final predictions in elections are heavily influenced by polls.
– Markets proved extremely good, faster than anything else, in incorporating information on Election Night.
Rajiv Sethi:
– Markets actually do very well early in the cycle, for example they had Dobbs beating Lake for AZ GOV in early August. But anecdotes like this aren’t evidence. I also feel that the reaction to the PA debate on social media and the markets was absurd – I must have seen hundreds of tweets saying that the Fetterman staff and family should never have allowed him to debate, etc. But he was perfectly capable of making the decision himself, and made a good call, that most people saw as courageous. But aside from PA I think the markets didn’t do badly, just the roll of the dice made them look bad this cycle.
From market fundamentalism to conspiracy theories
I was thinking about some of the above topics after reading this post by statistician Harry Crane, which ran a few days before the November 2022 elections:
When something doesn’t fit the official narrative: Regulate, legislate, or censor it out of existence. . . . It’s central to the Establishment’s strategy on a wide range of issues. Start looking and you’ll start to notice it just about everywhere. Here I focus on how it applies to elections. . . .
Who’s going to win the 2022 midterms? Specifically, which party will win control of the Senate?
According to the polls and pundits in the media, Democrats have the advantage. Voters are upset about Roe v. Wade. Democrats were 75% to win the Senate a couple weeks ago. Now it’s a toss up according to the forecasting website FiveThirtyEight.com.
But if you look at the prediction markets hosted at PredictIt — where savvy politicos risk real money on their opinions – you’ll see that the Republicans are 90% to win the House and 72% to win the Senate. . . .
So which is more accurate? As you’d probably expect, the markets are. . . .
Crane talked about how the prediction markets were favoring the Republican candidate from Jersey Pennsylvania:
Within a few moments after Fetterman opened his mouth for the first time [in their televised debate], Oz shot up to 65% and stayed near that price for the rest of the debate and the week following . . . FiveThirtyEight has Fetterman leading the entire time. We’ll know in a week or so which was more accurate at predicting the Pennsylvania Senatorial race outcome.
We’ll know in a week, indeed. Seriously, though, for the reasons discussed earlier in this post, we shouldn’t take one year’s failure as a reason to discard markets; we should just recognize that markets are human constructs which, for both theoretical and practical reasons, can have systematic errors.
And then Crane went all-in on the conspiracy theorizing, with an image of “Thought Police” and the following text:
When I gave this example [the Pennsylvania Senate race] in a recent interview about the upcoming election, the reporter was disturbed. The interview concluded shortly thereafter. The article was never written.
These markets pose an existential threat to legacy media . . . controlling the narrative before an election is integral to controlling what happens afterwards. Could this be why the media and current administration are putting extra effort to destroy all credible alternatives to biased polling?
When someone is so committed to an idea that he posits conspiracy theories . . . that’s not good.
I conjecture that some of this represents a theoretical misunderstanding on Crane’s part, a bit of what’s called market fundamentalism, a lack of appreciation for the complexity of information flow and behavior. It’s complicated, because if you read his post, Crane is not saying that he knows that markets are better. He says that markets do better empirically, but as discussed above we don’t really have so many data points to assess that claim. So calling him a “fundamentalist” is a bit too strong. I guess it would be more accurate to say that Crane overstates the evidence in favor of the performance of betting markets, he avoids looking at their problems, and then this puts him in a position of explaining the lack of dominance of markets in election forecasting by positing malevolent forces that suppress markets, rather than considering the alternative explanation that people are concerned about market manipulation (a topic as relevant nowadays as it’s ever been).
You might ask why discuss that post at all. Short answer is no, I wasn’t looking to be trolled, nor was I searching the web for the most extreme thing posted by a statistician that week. Given that tens of millions of Americans believe outlandish conspiracy theories, it’s no surprise that the some statistics professors are open to these ideas. I’d guess you could find quite a few believers in ghosts among the profession too, and even the occasional adherent of the hypothesis that smoking does not cause cancer.
Crane’s post interested me not so much for its conspiracy theorizing as much as for its ideological take on prediction markets. Crane loves prediction markets the way I love that Jamaican beef patty place and the way someone I know loves Knives Out. These are topics we just can’t stop talking about.
But let’s unpack this for a moment.
A prediction market, like any human institution, can be viewed as a practical instantiation of a theoretical ideal. For statisticians and economists, I think the starting point of the appeal of prediction markets comes from the theory. Betting is probability come to life, and betting on many related events induces a multivariate distribution. Real-life betting markets are too thin, too noisy, and have too many biases for this derive-the-distribution idea to really work, but it’s cool in theory. Indeed, even at the theoretical level you can’t be assured of extracting probabilities from markets, given possibilities such as insider trading and feedback. Anyway, seeing a post from someone who is such an extreme prediction-market fan gives us some sense of the appeal of these markets, at least for some segment of the technically-minded population.
Summary
My own views on prediction markets are mixed.
I like that there are election prediction markets and I get the points that Rothschild and Sethi make above about their value, especially when incorporating breaking news.
From the other direction, I would paradoxically say that I like markets to the extent that the bettors are doing little more than efficiently summarizing the news. I wouldn’t be so happy if market players are taking advantage of inside information; or using the markets to manipulate expectations; or, even worse, throwing elections in order to make money. I’m not saying that all these things are being done; I’m just wary of justifications of election markets that claim that bettors are adding information to the system. Efficient aggregation of public information would be enough.
I do like the idea of prediction markets for scientific replication because, why not? For me, it’s not so much about people “putting their money where their mouth is” but rather a way to get some quantification of replication uncertainty, in a world where Harvard professors are flooding the zone with B.S.
At the other extreme, no, I don’t favor the idea of a government-sponsored prediction market on terrorism run by an actual terrorist. In the abstract, I’m in favor of the rehabilitation of convicted criminals, but I have my limits.
Prediction markets are worth thinking about, and we should understand them in different contexts, not just as competition with the polls or as some idealized vision of free markets.