Election forecasts: The math, the goals, and the incentives (my talk this Friday afternoon at Cornell University)

At the Colloquium for the Center for Applied Mathematics, Fri 18 Sep 3:30pm:

Election forecasts: The math, the goals, and the incentives

Election forecasting has increased in popularity and sophistication over the past few decades and has moved from being a hobby of some political scientists and economists to a major effort in the news media. This is an applied math seminar so we will first discuss several mathematical aspects of election forecasting: the information that goes into the forecasts, the models and assumptions used to combine this information into probabilistic forecasts, the algorithms used to compute these probabilities, and the ways that forecasts can be understood and evaluated. We discuss these in particular reference to the Bayesian forecast that we have prepared with colleagues at the Economist magazine (https://projects.economist.com/us-2020-forecast/president). We then consider some issues of incentives for election forecasters to be over- or under-confident, different goals of election forecasting, and ways in which analysis of polls and votes can interfere with the political process.

I guess Cornell’s become less anti-Bayesian since Feller’s time . . .

P.S. Some of the material in the talk appears in this article with Jessica Hullman and Chris Wlezien. I should also thank Elliott Morris and Merlin Heidemanns, as the Economist model is our joint product.

5 thoughts on “Election forecasts: The math, the goals, and the incentives (my talk this Friday afternoon at Cornell University)

  1. As far as that Feller stuff goes… I still remember (very well) my bafflement 27 years ago (!) when you explained to me that Bayesian statistics was/were “controversial.” At the time I was sitting in on your Bayesian Data Analysis class, or whatever you called it at the time. I had the same feeling Jaynes mentions in that quote in your article: all of this seems so conceptually simple, indeed I would almost ‘obvious’, that I couldn’t figure out what people could object to.

    (Indeed, some of the basic principles of Bayesian statistics _were_ obvious. Before I started your class, I was working with a dataset that had a big problem with small-sample variability. This was the Minnesota radon dataset, and some counties only had two or three random-sample measurements. All of the counties with the highest and lowest mean concentrations were counties with very low sample sizes, and it was obvious that that’s just due to noise. I was trying to figure out what the _actual_ distribution of county means might look like, so I simulated the sampling procedure, using different distributions of the ‘real’ values, and looked for the simulated distribution that looked most like the data. I described this to you at lunch and you said I should take your class. Anyway, the point is that in essence I had been slowly and ineffectively reinventing part of Bayesian statistics: assume an underlying distributional form with unknown parameter values, and a relationship between the true values and the data, and then search for the parameter values for everything that are consistent with the data. I was a looooong way from developing even a small part of the machinery of Bayesian statistics, but I did have the right idea).

    Anyway, the thing that baffled me at the time was that there was supposedly controversy about the stuff you were teaching, which seemed both conceptually and mathematically straightforward…indeed, how can there be controversy about the application of a theorem? It’s a theorem, not a theory!

    Andrew, I’m sure I told you at the time: a year or two after you left for Columbia, I went to a talk at the UC Berkeley stats department. I got to the seminar room a bit early, and was sitting there killing time when a couple of UCB stats professors came in and started chatting. They were talking about a talk one of them had seen previously, evidently about some sort of Bayesian analysis, and the prof who had been to the talk started pooh-poohing it. I remember one thing he said was “Of course he had to assume exchangeability, which makes the whole thing suspect…” or some nonsense like that. At that point you had already pointed out to me (and others) that ordinary regression also assumes exchangeability (implicitly), so I thought about raising that issue, but in the end I kept my mouth shut. At any rate I found it funny.

    • Phil:

      Unrelatedly (except that your email reminded me of this), the above-linked paper cites Yitzhak! The reference actually comes from one of my collaborators, so it’s not like I just stuck in the Yitzhak reference for fun.

Leave a Reply

Your email address will not be published. Required fields are marked *