The above graph is from our 1993 paper, I guess I should update it sometime . . . anyway, the point is that public opinion used to move a lot during presidential election campaigns. We’re now about 70 days before the election, and even in that period you can see some real movement. For example, in 1988 the polls moved by about 3 percentage points during those last 70 days, and in 1976 they moved about 9 points. At the time of this writing, Harris is polling at about 51.5% of the two-party vote; a shift of even 3 points would make a huge difference.
We talked about this last month (also here): a key difference between different election forecasts is what possibilities they were including in their models for (a) changes in public opinion between now and election day, and (b) systematic polling error.
Can anything be said about the possible directions of (a) and (b)? I’m not sure. The starting point is the so-called fundamentals-based prediction based on the economy, incumbency, and presidential popularity. I think the fundamentals-based prediction is pretty close to where the polls currently are, so that won’t do much for us. I don’t think polls or public opinion are a random walk; it just happens that they’re around where we’d expect them to be based on external conditions. Which is not a huge surprise, given that in our modern age of political polarization, most voters have already “come home” to their parties very early during the campaign.
The next thing is momentum. I don’t really buy “momentum” as a general explanation, but there are specific pieces of information such as fundraising, crowd sizes, etc. Voter enthusiasm can go both ways. As a candidate, you want enthusiastic voters, no question. At the same time, enthusiasm can also show up as a greater rate of response (see this research article, or this shorter version). So Harris supporters being revved up and Trump supporters being demoralized could result in differential nonresponse making the polls look better for the Democrats than they really are. On the other hand, pollsters should be correcting for that! It’s easy enough; you just ask respondents their party ID and who they voted for in the previous election, and adjust accordingly. OK, it’s not trivial, as you still have to decide what to adjust for, but at least it should give more stable polls.
What about systematic polling error? In 2016 and 2020, the Democrats did better in the polls than in the election, and this varied by state. Variation by state is unavoidable; will there still be an average polling error? I don’t know. On one hand, pollsters know what happened in the previous two elections and are trying to do better; on the other hand, they knew about this in 2020 and that didn’t stop them from being off by two percentage points.
Finally, there’s whatever other information is out there that hadn’t made its way into the polls. For example, if the Robert Kennedy Jr. endorsement will affect turnout by bringing anti-vax and conspiracy-minded people to the polls for Trump, or by turning non-conspiracy-minded swing voters away from him, that might not be reflected in current polls but could lead to a shift in the next two months. And then there’s the consistent unpopularity of Trump and Vance: this should already be affecting the trial-heat polls but perhaps could have an impact on undecided voters? There’s also the idea that the Democrats are running a more competent campaign, which could help them as the election grows closer. But I’m not really sure what to make of such arguments, so for now I’ll have to go with our current forecast giving Harris a 60% chance of winning. (For reasons discussed here, I don’t think it makes sense to specify the probability any more precisely than that.)
Again, check out the graphs above to see how much things have changed. Right now, it seems reasonable to expect the polls to stay pretty much the same between now and election day. If they were to move by 3% in either direction, yeah, that’s certainly possible, but it would be considered a big swing. Until recently, though, the saying was that the campaign didn’t even start until Labor Day. And back when we wrote that paper, we found that the fundamentals-based forecasts performed much better than polls-based forecasts. State polls are fine, but this whole obsessing-over-the-state-polls is getting out of control. They’re part of a balanced forecasting approach (see also here) which allows for many sources of uncertainty.
Given that so few states seem to be in play, wouldn’t it make more sense to only look at those states? Looking at the national polls contains a mass of voting preferences that are not likely to make any difference to the outcome and makes it hard for me to gauge the meaning of a 3% swing as being a “big swing.” Also, I recognize the enthusiasm of both the Trump base and the recent Harris excitement, but both party’s candidates leave a lot to be desired – in fact, they are deeply flawed and the ugly campaign is likely to exacerbate that. So, I think it is turnout that is really most important – rather than looking at voter preference, shouldn’t we be looking at likelihood of voting for each candidate’s potential voters?
Dale:
States move together, so there’s a lot of information you can get from the non-swing-states as well. Regarding turnout, yeah, that is an issue; see my 2013 paper with Yair and various posts by Yair such as here and here. I guess he’ll produce another one of these once the current election is over.
“public opinion used to move a lot during presidential election campaigns”
I dunno. The polls for 1960 are practically a straight line. Now, that line is right around the .5 mark, so even tiny fluctuations could make a big difference. The 1956 polls move around a greater range, but consistently above .5, so the larger quantitative changes in the polls would have less (practical) significance. Looking at more recent graphs, 1992 and 1988 show a greater range and dynamism, but 1984 is a relatively flat line consistently above .5.
The takeaway I’m getting from this figure is that there is a lot of diversity in poll trajectories, with no clear pattern over time. Still, a very interesting figure!
Adede:
Agreed. It varied a lot from year to year. Also, the graph has fewer polls in those early years so it’s harder to see what was going on. One general message here is that a discrepancy between final polls and election outcomes is not new: the well-publicized errors in 2016 and 2020 were not large in historical context; they were just more newsworthy because of the closeness of the electoral college vote and also because poll aggregation had given many consumers of the news a false sense of accuracy.
60% is a nice number – what won’t be so nice is the jump if that number ever changes and becomes 50% or 70%.
Carlos:
I’m not quite sure how the Economist has it programmed up, but I think it might be something like “even odds,” “a 3 in 5 chance,” “a 2 in 3 chance,” “a 3 in 4 chance,” “a 4 in 5 chance,” “a 9 in 10 chance,” etc. It’s kind of similar to how you might say that the poll margin is 3 points or 4 points or 5 points or whatever. I have the impression that people have enough sense of polling variation to understand that it does not make much sense to say the poll margin is 3.8 points, but when it comes to percentages, there’s more confusion on that point.
Better to avoid quantitative descriptions all together. “Harris is slightly favored to win” expresses pretty much exactly the same thing without the false promise of numbers.
Total:
I disagree with you on that one. Providing words like that seems to me to just invite the reader to play a guessing game to figure out how they translate into numbers.
See this article from 2020 for more on the general point. I’m not saying that I’m right and you’re wrong here, just that there are difficult questions here!
“Providing words like that seems to me to just invite the reader to play a guessing game to figure out how they translate into numbers.”
I think you’re illustrating my point here about how compulsively people find numbers compelling, and why it’s dangerous to give them something that falsely feeds that compulsion (witness the famous backlash to Nate Silver around the 2016 election: “You said Clinton was going to win!” “I said she had a 70% chance, which is no guarantee.” The numbers presented to the public a false sense of surety, much more than would have a “Clinton seems likely to win but Trump still has a solid chance.”)
But in a larger sense, you can’t control what people do with your conclusions. You can control how you present them — and presenting them in a way that highlights the lack of certainty is more valuable than making a numerical point.
Thanks for the article link.
Ok, so your current forecast gives Harris a 60% chance of winning and the previous one gave Harris a 50% change of winning – and if it changes again it could go back to 50% or increase to 67%.
However, The Economist doesn’t really seem to think that one cannot be more precise than that – or they wouldn’t show more precise predictions in the “How our forecast has changed over time” chart.
As least they did include that chart before requiring registration, I imagine they still do: https://i0.wp.com/www.themainewire.com/wp-content/uploads/2024/06/image_2024-06-17_132300183.png
Thanks for this informative article, and all your prognostication work in general!
Question for you:
The polling underestimation of trump’s vote in 2016 / 2020 was quite significant.
Are there empirical signs we can watch for (WA primary vote, high quality pollsters, etc) to help us temper expectations and sort out if we’re making the same mistake this year?
Thanks
I have trouble accepting the probabilities the Election Forecasters give on the non-swing states. For example 538 gives Trump a 98% chance to win Louisiana! 98% seems good at first glance, but if a betting market gave -5000 odds for Trump to win that state, and they would allow you to bet large sums of money such as a Billionaire betting 500 million to win 10 million on odds that correlate to a 98% chance, I think it would be clear to nearly everyone that Kamala having a 1-50 chance of winning Louisiana is nowhere close to accurate, and that the Billionaire was making a great bet. I honestly believe the odds are much closer to winning the lottery than the odds most forecasters are giving the nominees on these extreme non-swing states!
Billy:
Yes, we discussed this issue a few years ago. Forecasters are focused on having sufficient uncertainty in the national electoral college prediction, and one way to do this is to pump up the uncertainties for individual states and not worry about the probabilities of extreme events.
On the one hand, in 2020 Trump won the Louisiana vote by 1.26M to 0.86M, and yeah, it’s very hard to picture 200,000 Louisiana voters switching from Trump to Harris, or 400,000 of them sitting out the election. So, yeah, maybe Louisiana has been pulled too far towards the mean.
On the other hand, 50:1 is a pretty big gamble. Trump could have a stroke between now and Election Day. He could be caught in bed with a dead girl or a live boy. Eh, actually of his supporters seem to be willing to forgive him anything — I think he really could shoot and kill a man and not lose most of his support, as he once bragged — but if he were caught in bed with a dead girl _and_ a live boy, that might do the trick. Given his state of mental and physical health I don’t think he’s a mortal lock even in Louisiana. I’m not sure where I’d draw the line on taking the Trump side of a bet in Louisiana…yeah, maybe I would offer 50:1, but I don’t think I’d offer 400:1 for example.
Billy, what are _you_ offering? Maybe we can wager.
Given the source of the dead girl/live boy quote, it’s particularly apt here,
There’s still sentencing coming up which could be the 3rd (4th?) black swan of the election. I’m not sure even prison would cost him the state but fleeing the country probably would.
Phil –
You seem to like to make a wager or two, so maybe you can help me to think this through.
If you asked me how I’d place the odds of something happening that would crater Trump’s campaign, (like him getting caught in bed with a dead girl and a live boy…), I’d say it’s likely higher than 100 to 1.
But if you asked me if I’d wager $100 to win $1, I would say no. What’s the point of risking $100 if I’m only going to win $1 by doing so. So there’s something going on here.
Am I fooling myself, somehow, about how I view the odds? Is there some kind of cognitive bias in play, like a negativity bias (the idea of losing $100 somehow gets an added meaning beyond just a purely statistical meaning)?
Joshua:
The standard interpretation of such bets is part of a larger portfolio. For example, suppose you invest $100 in Microsoft stock in the hope or expectation that in a year it will increase in value by 1% over inflation. But there’s some small chance that Microsoft will go out of business and your stock will become valueless. You’re thus effectively making a bet where you’re risking $100 in order to make $1. It’s not a perfect analogy because Pr(Microsoft goes out of business during the next year, in a situation where your money would otherwise have value) has to be a lot less than 1 in 100, but you get the basic idea.
Risking $100 to win $1 might sound silly—it’s “picking pennies in front of a steamroller”—but when you consider that you’re making many such bets over many years, the law of large numbers kicks in, and it’s all about expected monetary value.
As they say in poker, you should evaluate the strategy, not the play.
Joshua,
I’m not sure how to think about these extreme-odds bets either. In practice I am reluctant to offer long odds even if I think they’re fair, but I’m not sure how much of that reluctance is due to fear that I have mis-evaluated the odds, how much is rational due to a nonlinear utility function for money, and how much is irrational.
Suppose we take judgment out of it. I could buy a roulette wheel and run a roulette game, offering the same odds as Vegas: pay 35:1 on a bet on a single number. On a wheel with 0 and 00, a fair payout would be 38:1, so 35:1 offers ample margin as long as the wheel is well-balanced. I would be happy to play the role of the casino in a game, paying 35:1 on bets of $1 or $10. What about $100? Yeah, I think even there, I’d be willing to risk $3500 to win $100…actually happy to do it if the game was going to be played many times. But suppose some big spenders show up and want to wager $1000 each time, would I be willing to risk $35,000 per spin? I think I would want to, but would probably refuse because my wife would be very mad if I lost. And there’s no way in hell I’d do it if the wagers were $10,000 so that I might lose $350K. I think all of that is rational given a reasonable nonlinear utility function for money (and the fact that remaining in my wife’s good graces is so important to me that I would pass up an almost-certain gain of a few thousand dollars rather than run the risk both losing $35K and making my wife really mad). Anyway up to a pretty large amount of money I would be happy to offer 35:1 if I think 38:1 would be truly fair.
But there’s a big difference between a wager like that and a wager like offering 50:1 against Trump losing Louisiana: in the case of roulette I know what the true odds are, whereas in the Trump example I don’t. I’m “sure” Trump has more than a 2% chance of winning Louisiana but I’m not “sure” in the same way I’m sure about the roulette example. This is the kind of phenomenon Andrew has discussed in the “Boxer vs Wrestler” article… which I have to say I don’t think I got anything out of when it comes to practical advice about making wagers, but perhaps I should reread it. In the Trump case, I feel like I should be eager to offer 50:1 because I really think the true odds are more like 200:1 or more…and yet, I don’t really want to offer 50:1. Is this pure irrationality? Maybe.
Andrew and Phil –
Thanks to you both. It’s an interesting question to me. And your answers are helpful.
I hand’t considered the question of repeated plays affecting the decision. And yes, the uncertainty as opposed to a more clear odds scenario is important.
And certainly the “wife” factor isn’t strictly mathematical. That meat cleaver is awfully sharp.
Joshua:
the obvious interpretation of your hypothetical is that if you’re not willing to risk you money on your claim then you don’t really believe it, or at best you don’t believe the odds your claiming.
If you really believe the 100:1 odds about Trump’s behavior, you’d take the (incredibly low) “risk” because the $1 win is a ridiculously sure thing. Why would you *not* bet $100 to win $1 if the odds give you such a massive advantage????
So, yeah, there is an immense “cognitive bias” in play, you’re pulling numbers like 100:1 out of thin air with no reliable basis, your claiming that they’re somehow real, and then your saying that it’s a mystery that you don’t really want to bet those odds. The mystery is solved when you admit you have no serious basis for those numbers and you don’t believe them.
With regards to MSFT, only a fool would risk 100:1 odds in the stock market. The facts are:
1) long term total market returns are about 10% annually so that’s the baseline. In other words, if you only obtain average market returns, you’ll score 10% annually, or $10 per year for every one hundred you bet. To put it another way, you’ll double your money every seven years.
2) The ten year return on MSFT is 24.54% annually (my schwab account shows $45.43 opening price in the first week of the ten year chart, vs. a closing price today of $412.25 and my annual return calculator yields 0.2454) so you’re betting $100 to make $25 – 25x higher potential return than Andrew suggests. Andrew is a great statistician but he doesn’t follow markets and neither he nor phil know what’s actually going on in them. I think that’s a safe claim. So, to follow the “double your money” metric, if you project historical MSFT returns forward, you’d double you money every **four** years with MSFT (actually be slightly short of that, at $93 returns for every $100 invested) .
OK? Average market returns are double you rmoney every seven years. MSFT market returns double your money every four years. This is not at all controversial. Its a simple forward projection of past returns. Another way of saying that is that projecting past returns forward, if your money is invested in the “total market” you’d make four times your money every 28 years ($100 > $1600). If your money is invested in MSFT, you’d make *seven* times your money every 28 years ($100 > $128,000).
The math is not open to question. The only reasnonable question is whether the market overall and MSFT will continue to generate historical returns. And that, of course, depends on the election in the US.
commenter who shirks accountability –
So, yeah, there is an immense “cognitive bias” in play, you’re pulling numbers like 100:1 out of thin air with no reliable basis, your claiming that they’re somehow real, and then your saying that it’s a mystery that you don’t really want to bet those odds.
That was my question – when I asked if I would be “fooling myself.” Yes, there is a conventional wisdom, that if you’re not willing to bet it then you don’t REALLY believe it.
But I’m not sure I buy that. I think first, there are likely cognitive biases in play, like a negativity bias. It’s not like year kinds of biases aren’t well established.
But I think there’s more in play, related to an irrationality about loss, particularly in a non-repeating scenario where it FEELS like the probabilities are different even if mathematically they aren’t.
I think about how climate “skeptics” don’t short the market, and neither do AI doomers. Can I just assume that means they don’t believe what they say they believe? . I dunno, maybe. But I don’t think it’s as easy to mind-probe as you apparently think it is. I think people are complicated.
“…it’s not like THOSE kinds of biases…” (not year)
I think a “loss aversion” bias is more of what I was thinking about rather than a “negativity bias.”
Joshua:
“Yes, there is a conventional wisdom, that if you’re not willing to bet it then you don’t REALLY believe it.”
The “conventional wisdom” is the appropriate view.
If you want to tear your Trump claim appart, there are several components.
1) the first component is the odds you provide, which are at best wildly speculative.
2) the second compoenent is the assumed level of damage from those odds, which is wildly speculative^(wildly speculative).
3) there may be many others, I just can’t think of them off the top of my head.
How do you know that any claim will move the election at all, or even opposite to your expectations? At this point, I confess I have come to agree with Andrew, there’s nothing any Democrat could say that would convince me to vote against Trump, so the swing lies with a decreasingly small proportion of the population – i.e., a decreaingly releveant proportion of the population.
What if Trump said to vote against Trump? Like he *is* a very effective candidate. If he wins this one then he can’t run again in 2028. Republicans can raise more money if he is also in that race.
The biggest thing is the global temperature record though: https://www.drroyspencer.com/wp-content/uploads/UAH_LT_1979_thru_July_2024_v6_20x9-scaled.jpg
That has risen so much during Biden’s term, its hard to believe it could rise again over the next 4 years. And if it drops that would be essentially unprecedented (JFK is the only *maybe* exception since 1880) for a democrat president. Its really likely we will see a republican based on these global warming numbers.
I am actually reluctant to say any outcome is less likely than 1-2% unless its contingent.
IE “Trump wins Louisiana” could maybe be 98%. “Trump wins Louisiana, contingent on his still being the Republican candidate” is notably higher (because you’ve excluded the possibilities where death/a major health event replaces Trump with Vance).
Anon,
You say “it’s really likely we will see a Republican president”, would you care to convert that to a probability? “Really likely”, is that 66% for example? If you offer 2:1, I’ll take it, we can make a wager with the proceeds going to support Stan development like last time.
confused’s points is really important, when we’re saying something like “there’s essentially no way Trump loses Louisiana” people are implicitly making that contingent on Trump being still in the race and eligible and etc. He could die of a second assassination attempt, or a simple heart attack or stroke, he could trip on a gangplank to an airplane and fall down the stairs, he could be in a plane crash or a car crash or choke on some steak, he could have a stroke and decide to pull out of the race, evidence could surface that he was involved in crimes his base couldn’t ignore (an example might be child trafficking) … all of those possibilities put together probably add up to more than negligible percent chance. So what you’re trying to quantify when you say what chance he has of not winning Louisiana is some kind of stuff related to health, accidents, assassination etc not primarily if the people in Louisiana will change their mind.
I think in fact the election models are contingent on a lot of factors. Back in June they were assuming Biden would still be the candidate. There is probably an assumption that votes will be more or less accurately counted (if just one state failed to do so, you could see huge swings vs other states that “should” be similar).
I also wonder about other low probability events. There are a bunch of states that are safely Democratic due to one big city with much of that state’s population. If some kind of disruption hitting that one city led to people in that city voting at much lower rates, you could conceivably get one state swinging way out of its expected position. Or a bad federal government response to some state-specific disaster (like a hurricane) hurting the incumbent party in just that one state.
I also wonder how much post-COVID movement between states will change their relative position. Recent polling has shown some super weird stuff (NY barely more blue than NH, VA being like D+3 and thus close to the national margin while NC and GA are basically tied, etc). I have a feeling most of these are outliers, but
I do have a pretty strong prior that the plausible range of outcomes is between maybe Trump +1 and Harris +5 in the popular vote, with a strong bias toward the middle of that range and a real EC nail biter.
I agree with many of the comments here that there are a number of black swan type events that could cause LA to go blue or CA to go red, etc. What I am unsure about is whether the probability of such events is 2%, 1%, 0.1%, or .001%. I am simply not good at estimating such small probabilities for rare events (and I believe this is true for the vast majority of people). The probability of a candidate dying is somewhat manageable since we at least have life tables that should provide an order of magnitude estimate. But, absent relevant historical data or a clear mechanism that could be modeled for such events, what is the probability of an assassination? I think most of the scenarios we would envision for these highly improbable state swings are like that.
Dale, there have been about 1 major mass extinction every 100M years, so whatever the chance of Trump losing it has to be bigger than 1e-8 because that’s kinda like the size of the chance of a massive asteroid strike wiping all human life off the planet. I mean, we can argue about there being less than a year between now and the elections etc, so maybe it’s 1e-9 but it’s way way bigger than 1e-12. And that’s just the mass extinction. It’s gotta be the case if you look at CDC life tables that there’s probably at least 1e-5 chance Trump just dies of natural causes in the next couple months. so lower bound is probably on the order of 1e-3 ish when you put all the weird stuff that could happen together.
Because I have witnessed more presidential elections than most, the following made me bolt upright:
“The above graph is from our 1993 [!!] paper, I guess I should update it sometime . . . anyway, the point is that public opinion used to move a lot during presidential election campaigns.”
A few things, so to speak, have happened since then and the graph itself in no way indicates that Donald Trump would dominate a good bit of the 21st century. Plus, of the 11 graphs, as is admitted, 9 are from Gallup only. While it is a tour de force to be able to ferret out and display such data, given the massive changes in American life and American history since then, I fail to see the relevance. Put another way, every U.S. presidential election can/could be viewed as unique such that the past is not much of a prologue (to hang a statistical hat on?).
Regarding Henry Ford, who may or may not have claimed that “history is bunk (sometimes),”
https://www.thoughtco.com/henry-ford-why-history-is-bunk-172412
“By all accounts, Ford was a difficult, uneducated, and litigious fellow”
Two percentage point error in 2020? Wasn’t it closer to 4 point error in 2020 nationally and even more in wi, IA, oh, mi, Pa, fl?
Erik:
I think it was about 2 percentage points in vote proportion, which would be about 4 percentage points in vote margin. The errors were larger in some states and smaller in others.
Got it.
In the comments to an earlier post, when Joe Biden was still in the race, it was noted that the 538 election forecast and the Economist forecast had very different projections. Now that both forecasts have been recalibrated for a Trump-Harris match-up, they agree on a 60/40 probability for Harris.
So what I don’t get is why pollsters are so focused on just the vote. Why not, while you have thousands of Americans at your disposal, enquire about what people expect the outcome of the election to do? I mean after all, presumably people care about who they will vote for because they care about what each candiate will do, so wouldn’t it be interesting to know what the responses are to questions like: what would be the most improtant outcome fo (candidate X, Y) being elected”
A few hard-core anti-vaxxers are still angry about operation warp speed, Trump was already lobbying heavily for their vote.
https://www.pbs.org/newshour/show/trump-vows-to-defund-schools-requiring-vaccines-for-students-if-hes-reelected
Should read:
While a few hard-core anti-vaxxers are still angry about operation warp speed, Trump was already lobbying heavily for their vote.
https://www.pbs.org/newshour/show/trump-vows-to-defund-schools-requiring-vaccines-for-students-if-hes-reelected
Linking up with Junior will go a long way there
> for now I’ll have to go with our current forecast giving Harris a 60% chance of winning
Down to 50% today.
It seems that The Economist’s probability for Harris’ was yesterday above FiveThirtyEight’s (59%) and today it’s below Nate Silver’s (53%).
60% again. If the intention was to avoid confusing readers with small, meaningless changes in the reported probabilities the result is that they are confused with large, meaningless changes instead.