Rajiv Sethi on the interpretation of prediction market data

Rajiv Sethi has some very interesting things to say:

As the election season draws closer, considerable attention will be paid to prices in prediction markets such as Intrade. Contracts for potential presidential nominees are already being scrutinized for early signs of candidate strength. . . .

This interpretation of prices as probabilities is common and will be repeated frequently over the coming months. But what could the “perceived likelihood according to the market” possibly mean?

Prediction market prices contain valuable information about this distribution of beliefs, but there is no basis for the common presumption that the price at last trade represents the beliefs of a hypothetical average trader in any meaningful sense [emphasis added]. In fact, to make full use of market data to make inferences about the distribution of beliefs, one needs to look beyond the price at last trade and examine the entire order book.

Sethi looks at some of the transaction data and continues:

What, then, can one say about the distribution of beliefs in the market? To begin with, there is considerable disagreement about the outcome. Second, this disagreement itself is public information: it persists despite the fact that it is commonly known to exist. . . . the fact of disagreement is not itself considered to be informative, and does not lead to further belief revision. The most likely explanation for this is that traders harbor doubts about the rationality or objectivity of other market participants. . . .

More generally, it is entirely possible that beliefs are distributed in a manner that is highly skewed around the price at last trade. That is, it could be the case that most traders (or the most confident traders) all fall on one side of the order book. In this case the arrival of seemingly minor pieces of information can cause a large swing in the market price.

Sethi’s conclusion:

There is no meaningful sense in which one can interpret the price at last trade as an average or representative belief among the trading population.

This relates to a few points that have come up here on occasion:

1. We’re often in the difficult position of trying to make inferences about marginal (in the economic sense) quantities from aggregate information.

2. Markets are impressive mechanisms for information aggregation but they’re not magic. The information has to come from somewhere, and markets are inherently always living in the phase transition between stability and instability. (It is the stability that makes prices informative and the instability that allows the market to be liquid.)

3. If the stakes in a prediction market are too low, participants have the incentive and ability to manipulate it; if the stakes are too high, you have to worry about point-shaving.

This is not to say that prediction markets are useless, just that they are worth studying seriously in their own right, not to be treated as oracles. By actually looking at and analyzing some data, Sethi goes far beyond my sketchy thoughts in this area.

4 thoughts on “Rajiv Sethi on the interpretation of prediction market data

  1. Since I don't know if Rajiv Sethi has even ask Intrade for trading data, I'm not sure if his conclusions can be realistically evaluated. But since traders in Intrade invest their own money in the markets they trade, it is unlikely that they will trade with the intention of LOSING money. Therefore, as part of the 'knowledge aggregation' their information, joined with the information of other traders, represents a consensus of self-professed experts (traders).

    'Beliefs' fundamentally have no monetary value to traders, nor does belief appear to drive Intrade markets.

  2. The paper by Wolfers & Zitzewitz, "Interpreting Prediction Market Prices as Probabilities", provides sufficient conditions for prices to coincide with mean beliefs exactly. They also do some numerics to show that in many situations prices will be close to mean beliefs, and they discuss how these results hinge on the distribution of beliefs. If I recall correctly though, they don't look at the effect of an uneven distribution of trading resources or what happens when resources are correlated with beliefs.

  3. It's relevant to point out the result of Haile and Tamer (2003, Journal of Political Economy) that, with many markets for the same contract, but only data on winning bids in each market, you can back out the distribution of valuations, and thus beliefs over likelihoods, with some relatively light assumptions.

    If there are multiple markets like Intrade, this could prove a fruitful area of research.

  4. Andrew,

    I saw Rajiv Sethi post and enjoyed and similar others that appear from time to time. But having been CEO of Intrade for circa 10 years when we have listed and expired approx 700,000 individual markets some things seem unequivocal.

    1. As you say "markets are impressive mechanisms for information aggregation but they're not magic" but very few claim they are – we certainly do not, hence they are called prediction markets not something more definitive.

    2. Liquidity matters. You need a certain critical mass of trades/traders to get good predictions.

    3. It is not always simply the wisdom of the crowd but the wisdom in the crowd of the few "seer's" in the crowd that generate the wisdom. Seer = profitable or accurate participant.

Comments are closed.