Suppose you and I agree on a probability estimate…perhaps we both agree there is a 2/3 chance Spain will beat Netherlands in tomorrow’s World Cup. In this case, we could agree on a wager: if Spain beats Netherlands, I pay you $x. If Netherlands beats Spain, you pay me $2x. It is easy to see that my expected loss (or win) is $0, and that the same is true for you. Either of us should be indifferent to taking this bet, and to which side of the bet we are on. We might make this bet just to increase our interest in watching the game, but neither of us would see a money-making opportunity here.
By the way, the relationship between “odds” and the event probability — a 1/3 chance of winning turning into a bet at 2:1 odds — is that if the event probability is p, then a fair bet has odds of (1/p – 1):1.
More interesting, and more relevant to many real-world situations, is the case that we disagree on the probability of an event. If we disagree on the probability, then there should be a bet that we are both happy to make — happy, because each of us thinks we are coming out ahead (in expectation). Consider an event that I think has a 1/3 chance of occurring, but you put the probability at only 1/10. If you offer, say, 5:1 odds — I pay you $1 if the event doesn’t occur, but you pay me $5 if it does — each of us will think this is a good deal. But the same is true at 6:1 odds, or 7:1 odds. I should be willing to accept any odds higher than 2:1, and you should be willing to offer any odds up to 9:1. How should we “split the difference”?
I started pondering this question when I read the details of a wager, or rather a non-wager, that I had previously only heard about in outline: scientists James Annan and Richard Lindzen were unable to agree to terms for a bet about climate change. Lindzen thinks, or claims to think, that the “global temperature anomaly” is likely to be less than 0.2 C twenty years from now, but Annan thinks, or claims to think, it is very likely to be higher. You can imagine a disagreement over the details — since the global temperature anomaly can’t be measured exactly, perhaps you’d want to call off the bet (doing so is called a “push” in betting parlance) if the anomaly is estimated to be, say, between 0.18 and 0.22 C — but surely, given that the probability assessments are so different, there should still be a wager that both sides are eager to make! But in fact, they couldn’t agree on terms.
Chris Hibbert has discussed the issue of agreeing on a bet on his blog, where he mentions that Dan Reeves “argues, convincingly, that the arithmetic mean gives each party the same expectation of gain, and that is what fairness requires.” But Hibbert goes on to say that “the way that bayesians would update their odds is to use the geometric mean of their odds.” I’m not sure of the relevance of this latter statement, when it comes to making a fair bet.
Suppose I think the probability of a given event is a, and you think the probability is b. If the event occurs, you will pay me $x, and if it doesn’t occur, I will pay you $y. We don’t need to know the actual probability in order to figure out how much each of us thinks the bet is worth: I think I will gain ax – (1-a)y, and you think you will gain -bx + (1-b)y. We might say a wager is “reasonable” — the word “fair” is already taken — if I think it’s worth as much to me as you think it is worth to you. Look at it this way: I should be willing to pay up to ax – (1-a)y to participate in this wager, and you should be willing to pay up to -bx + (1-b)y. If those amounts are equal, then we’d each be willing to pay the same amount to participate in this game.
Setting the two terms equal and doing the math, we end up with a reasonable bet if x= y(2-(a+b))/(a+b) or, equivalently, x = y(2/(a+b) – 1). Note that this is the same thing we would get if we agreed that the probability p = (a+b)/2. So, I agree with Dan Reeves and his co-authors: the way to make a reasonable bet is to take the arithmetic mean of the probability estimates.
Thanks Phil! Let's take it further. Say I'm the one who thinks the event is highly unlikely, and you think it's only a little unlikely. Knowing that we'll be taking the mean of our probability estimates to set the odds, I should inflate my probability estimate. That yields more favorable odds for me. And you should shade your probability down for the same reason.
I'm not sure if this maps directly to the Myserson-Satterthwaite Impossibility Theorem, something I've written about here: http://messymatters.com/landlords
I'm not sure why you say of Lindzen that he only "claims" to think that the global anomaly will likely be less than .2 degrees per decade through the end of this century — as if to imply that he actually knows better.
Look, Lindzen has been pretty clear that he thinks the real problem with much of the science behind the climate models is that they don't really capture the natural variability in climate in general and in global temperatures in particular. He firmly believes that that natural variability (in the sense of changes in climate being due to complex factors of which we have little understanding) is quite high, especially over short periods of time. This constrains the sorts of bets he might reasonably be willing to make.
For example, he's going to have little inclination to bet that over the next 20 years that temperatures won't rise by .2 degrees; such a rise would be quite unsurprising given the natural variability he assumes — and yet this is apparently the sort of short term bet that Annan was insisting on making. (Likewise, Lindzen has always been skeptical of the accuracy of global temperatures at the level of hundredths or even tenths of degree, so is unlikely to be willing to make a bet based on the assumption of accuracy in such measurements down to those levels.)
Annan, on the other hand, believes that natural variability is small — at least as any kind of basis for the current changes in global temperature. That assumption of small natural variability is pretty much required by the climate models — if they don't model all of the major factors, how can their predictions be taken seriously? For him, it does probably make sense to make a short term bet, because he thinks he has correctly modeled the forces that will affect temperature over the next twenty years.
Now I think where a bet could be made between the two would be over the long run, where natural variability will play a far lesser role: here's where they really should disagree about likely outcome.
In that context, I'm not sure why Annan refused to take up the long term bet Lindzen offered. It is, of course, uncollectable in their lifetimes, but that shouldn't be an obstacle — if anything, one would think it would make it easier to accept.
Now I don't think that Lindzen's offer is particularly generous: 2 to 1 odds IF temperatures rise at a rate faster than .4 over the next century.
But if temperatures DON'T rise at a rate equal to that clip, then the question arises as to what all the big fuss is over global warming anyway. Certainly if the rise in temperatures is only 2 degrees, it's hard to see the damage that would be done. As best I can make out, the rise in temperatures would have to be about 3-4 degrees at least for significant issues to be raised. Since Lindzen offered that from 2 to 4 degrees would be a no-win for either side, that seems not unreasonable as a way to handle the cases in between.
I guess I find Annan's failure to accept this bet as rather strange, given his own beliefs. Does he not think that temperatures will rise at least 2 degrees? If he does believe that, why would he hesitate to accept the bet? True, he won't actually WIN the bet unless it goes up 4 degrees, but that at least certainly seems to be well within the range of projections he and other believers in global warming are generating, and using as a basis to advocate for legislation.
Wagers are such a mess because of selection effects, not to mention cognitive biases involving uncertainty and loss. I think it's a big mistake when Bayesians (and some others) treat wagering as some sort of foundational principle.
To put it another way: betting, negotiation, etc., are all important topics in their own right, but I think they should be studied as the complex psychological/economic constructs that they are, rather then trying to directly align them with probabilities (except maybe in some clean cases such as horse races where all the bets are laid out in advance).
Daniel, you're right, both bettors "should" game the system. The interesting thing is that they game the system by moving their probability estimates closer to each other…if they know which direction that is, of course. Hmm. Suppose we each write our probability estimates on paper, to be revealed simultaneously; there's a single round (i.e. no chance to change your estimate); and we agree in advance that whoever writes the lower probability will offer the odds determined from the formula discussed above. Let's even assume that we both know that you think the probability is lower than I think it is. How should we determine what to write down? Hey, let's make it explicit, and assume all our cards are on the table: suppose we both know that you think the probability is 0.1 and I think it's 0.4; what numbers should each of us write down? (There might be an obvious answer to this, I'm just writing this problem as it comes to me).
"A little skeptical," I used exactly the same language — "thinks, or claims to think" — to describe both sides of the wager, and I deliberately did not attribute the failure to agree to a wager to one person or the other, so I don't know what you're going on about. (But in fact, my implication of symmetry is misleading: according to the article in reason.com, the wager Lindzen offered is radically inconsistent with his claim that there's a 50% chance that global mean temperature will be lower in 20 years than it is now.) Also, the article is at variance with your comment in several ways. For example, the article says nothing about Lindzen offering a bet with a longer time horizon. Perhaps you are thinking about a different article, or a different bet. Finally, your comment that a long-term bet "is, of course, uncollectable in their lifetimes, but that shouldn't be an obstacle — if anything, one would think it would make it easier to accept" illustrates exactly why such a wager would be meaningless. The whole point of wagering is that you actually have something at stake! If you're going to make an uncollectable bet, there's no incentive to get it right, and the whole exercise is pointless. Anyone who has played poker with no stakes discovers that people will make bets that have nothing to do with their belief in the strength of their hand, because why not?
Andrew, rarely have I disagreed with you more. Do you really think it's pointless to study wagering? I think it's both interesting, and, um, pointful, to study the entire continuum that ranges from how people should bet (given a certain set of beliefs) to how people do bet. The latter involves a lot of psychological and cultural factors that I don't know much about, but I think the former is interesting too. And for me (and many other people) it has at least modest practical importance; for instance, if Netherlands had beaten Spain in today's World Cup final, I would have won 14 bottles of wine worth more than $30 per bottle, in a pool similar to one I wrote about in a Chance Magazine article.
Phil:
You ask: "Do you really think it's pointless to study wagering?"
Read my comment more carefully!
My use of phrases such as "important topics" and "should be studied" are a clue that, no, I don't think it's pointless to study wagering!
P.S. to Phil: I don't know if this helps, but I wasn't trying to criticize your blog. I was making a more general comment about the way in which people (not you) sometimes oversell the idea of betting as a general framework for understanding probability.
Andrew: I think you're right. Odds are like prices. The question "What do you think the odds are that Spain will beat the Netherlands?" is similar to the question "What will you give me for a 2003 Honda Civic?" The answer to the latter will be different for each individual, and for a given individual it may vary from moment to moment based on the complex psychological and economic factors you mention. There is no "true" answer to either question.
I originally found this wonderful blog as I was checking out Karl Popper's propensity interpretation of probability. But isn't that precisely what's at issue in one off bets such as this? If I am to say that I am behaving rationally, I have to believe that there is in fact in the world a 2/3 or 1/3 chance of something happening this one time. But it is not like a roll of a die, where we know in the abstract it's a 1/6 shot. It's more like estimating how many sides there are to the die, i.e. saying something about the propensity of the world to turn out a certain way on this one bet.
Rather than look for the fair wager between the two, I say we introduce a middle man (preferably me). I'll take the long side of the 1:9 bet and the short side of the 1:2 bet. That way both the original players are satisfied (zero expected payoff under each of their personal beliefs, which seems fair) and I realize a nifty profit for my services, regardless of the actual outcome!
Andrew: you're right, I should have said "do you really think it's pointless to study wagering except 'as [a] complex psychological/economic construct'"? If that's what you're saying, then I disagree. I think there is a lot to be learned by asking questions like the ones posed in this post.
I think the "fair bet" interpretation of probability is a good way — maybe the best way — of thinking about the probability of events that will only occur once.
John S, as the paragraph above probably indicates, I think disagree with you, if I understand what you're saying. I think the probability that Spain would beat Netherlands is not nearly as subjective as the price I would pay for a car. There may not be an unarguably right answer to the former (indeed, there probably isn't), but there are certainly wrong answers. 20% would be a wrong answer. So would 80%. Even though the game has now been played and Spain won, I think it is incorrect to say that Spain had a 100% chance of winning the game. Yes, I realize that there are philosophical issues here involving determinism, and what it means for there to be a "probability" for an event that will only happen (or not) a single time. But that doesn't mean that probability doesn't mean anything, or that all probability estimates are equally valid.
J.J., see my comment to John S. I think the concept of "probability" does get a little squishy when we talk about events that only occur once, but I think everyone agrees that some events are more likely than others, some events are very unlikely, etc. As for how to decide which of several probability predictions is more accurate, yeah, that's a tough question. I guess there's not always an answer to it.
I suppose it's worth noting, though, that almost every probability estimate of anything interesting is of a "one-off" event. The probability that it will rain tomorrow, the probability that I'll roll a 7 on this pair of dice that I am rolling right now, the probability that I will die in a car accident today…all of these are _similar_ to things that have come up before, but they're not identical. If we were to restrict "probability" to refer to events that are exactly repeated, then the whole field of probability and statistics would be pretty useless. And once we agree that we're going to tackle events that aren't _perfectly_ repeated, it's pretty arbitrary where we draw the line.
Phil: I never used "pointless" at all. I don't think it's pointless to study wagering without a psychological etc construct. Recall that we analyze football point-spread data in chapter 1 of Bayesian Data Analysis! Betting is great; I just don't think it's a good foundational principle for probability modeling.
Andrew, I guess you threw me when you said " I think they should be studied as the complex psychological/economic constructs that they are, rather then trying to directly align them with probabilities." I thought that meant you don't think it's worthwhile to study wagers without considering them as a "complex psychological/economic construct." Which, I agree, does seem to be contradicted by Chapter 1 of Bayesian Data Analysis.
Just to answer a previous commenter, the simple reason I wasn't interested in Lindzen's counter-offer is that his proposed bet has negative value for me. I would only win if the warming was more than 0.4C in 20 years, and he would win if the warming was under 0.2C. I consider both events to have similar (small) probability, but he demanded 2:1 odds in his favour. Much more likely, the warming will be between these values in which case the bet would be void.
As for people gaming the system and their odds converging, this is a feature not a bug. One aim of betting markets is to find a consensus viewpoint.
I do have some sympathy for Andrew's viewpoint that betting is not necessarily a perfect way to consider probability, but I have also found that it helps to focus the mind (my own, as well as others) on what one really does believe, rather than what one is prepared to say.
James, I agree, wagering — or, really, anything that gives you a non-negligible stake in a quantitative prediction, but wagering is the most common example — is a great way to test yourself about what you really believe. Fairly frequently, someone will make a claim ("this restaurant will be closed in a year," "there's no way the Democrats will hold onto the House", etc.), and I'll convert their statement into a wager that I offer to accept…only to see them refuse to make the wager, much like your experience with Lindzen. They don't really have the courage of their convictions. I must admit, I've had the same failing myself, claiming to be "sure" of something but, when challenged, declining to offer 5:1 odds. In fact, this experience has made me much more careful about my claims. I think it would be healthy for almost anyone who makes predictions — pundits, scientists, pollsters — to be encouraged or forced to wager on them.
Just a small aside from a different field:
This is the EXACT same logic mentioned in "trade theory" of economics….only the buzz words are different: "factor endowments" substitute for internal probabilities assessments, "terms of trade" for the odds of the bet (also split down the middle for 'optimal trade'), and "gains" are gains.
This is entirely wrong. The paper discusses using probability estimates to garner mathematical expectations, then using these expectations to determine if a wager is "fair," or not, with "fair" being the watershed between whether to accept a wager or not.
But Mathematical expectation is NOT the way to asses whether to assume a wager or not, academic opinion to the contrary. Rather it is a proxy for what should be one's criteria in assessing such.
What is NOT discussed in this paper is time horizon of successive wagers (unless we assume a de facto horizon of a single play) which is a necessary input in the decision of whether to accept a given wager or not.
Again, college is the wrong place to learn this stuff, or seek the correct answers from. It is a wager where you risk your money to learn about this stuff — and lose with certainty.
Great post Phil. I was going to make a similar comment to Daniel Reeves: shows how much we think alike. To answer his question, yes I believe it is impossible to have a budget-balanced mechanism that would induce both people to truthfully report their probability.
To address the back and forth between Phil and Andrew Gelman (which seems to be resolved anyway): to me, since we can't look inside anyone's head except our own, the only way to surface subjective probability is through a wager or similar behavior (revealed preference). So, even though it's messy and inevitably tied up with risk preference, psychology, etc., a wager is the best way to think of non-repeated uncertain events, i.e. nearly all events. In short, I agree with Phil but can't tell if I disagree with Andrew.
I'm confused why this appears to be Andrew Gelman's blog but is authored by Phil and hosted under Sam Cook's personal homepage.
thanks,
Dave