Skip to content
Archive of entries posted by

Interactive analysis needs theories of inference

Jessica Hullman and I wrote an article that begins, Computer science research has produced increasingly sophisticated software interfaces for interactive and exploratory analysis, optimized for easy pattern finding and data exposure. But assuming that identifying what’s in the data is the end goal of analysis misrepresents strong connections between exploratory and confirmatory analysis and contributes […]

“Election Forecasting: How We Succeeded Brilliantly, Failed Miserably, or Landed Somewhere in Between”

I agreed to give a talk in December for Jared, and this is what I came up with: Election Forecasting: How We Succeeded Brilliantly, Failed Miserably, or Landed Somewhere in Between Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University Several months before the election we worked with The Economist magazine to […]

An odds ratio of 30, which they (sensibly) don’t believe

Florian Wickelmaier and Katharina Naumann write: In a lab course, we came across a study on the influence of “hemispheric activation” on the framing effect in decision making by Todd McElroy and John J. Seta [Brain and Cognition 55 (2004) 572-580, doi:10.1016/j.bandc.2004.04.002]: Two experiments were conducted to determine whether the functional specializations of the left […]

“Model takes many hours to fit and chains don’t converge”: What to do? My advice on first steps.

The above question came up on the Stan forums, and I replied: Hi, just to give some generic advice here, I suggest simulating fake data from your model and then fitting the model and seeing if you can recover the parameters. Since it’s taking a long time to run, I suggest just running your 4 […]

Piranhas in the rain: Why instrumental variables are not as clean as you might have thought

Woke up in my clothes again this morning I don’t know exactly where I am And I should heed my doctor’s warning He does the best with me he can He claims I suffer from delusion But I’m so confident I’m sane It can’t be a statistical illusion So how can you explain Piranhas in […]

Presidents as saviors vs. presidents as being hired to do a job

There’s been a lot of talk about how if Biden is elected president it will be a kind of relief, a return to problem solving and dialing down of tension. This is different from Obama, who so famously inspired all that hope, and it made me think about characterizing other modern presidents in this way: […]

Estimated “house effects” (biases of pre-election surveys from different pollsters) and here’s why you have to be careful not to overinterpret them:

Elliott provides the above estimates from our model. As we’ve discussed, as part of our fitting procedure we estimate various biases, capturing in different ways the fact that surveys are not actually random samples of voters from an “urn.” One of these biases is the “house effect.” In our model, everything’s on the logit scale, […]

Whassup with the dots on our graph?

The above is from our Economist election forecast. Someone pointed to me that our estimate is lower than all the dots in October. Why is that? I can come up with some guesses, but it’s surprising that the line is below all the dots. Merlin replied: That happened a bunch of times before as well. […]

Pre-register post-election analyses?

David Randall writes: I [Randall] have written an article on how we need to (in effect) pre-register the election—preregister the methods we will use to analyze the voting, with an eye to determining if there is voter fraud. I have a horrible feeling we’re headed to civil war, and there’s nothing that can be done […]

Between-state correlations and weird conditional forecasts: the correlation depends on where you are in the distribution

Yup, here’s more on the topic, and this post won’t be the last, either . . . Jed Grabman writes: I was intrigued by the observations you made this summer about FiveThirtyEight’s handling of between-state correlations. I spent quite a bit of time looking into the topic and came to the following conclusions. In order […]

Reference for the claim that you need 16 times as much data to estimate interactions as to estimate main effects

Ian Shrier writes: I read your post on the power of interactions a long time ago and couldn’t remember where I saw it. I just came across it again by chance. Have you ever published this in a journal? The concept comes up often enough and some readers who don’t have methodology expertise feel more […]

Calibration problem in tails of our election forecast

Following up on the last paragraph of this discussion, Elliott looked at the calibration of our state-level election forecasts, fitting our model retroactively to data from the 2008, 2012, and 2016 presidential elections. The plot above shows the point prediction and election outcome for the 50 states in each election, showing in red the states […]

She’s wary of the consensus based transparency checklist, and here’s a paragraph we should’ve added to that zillion-authored paper

Megan Higgs writes: A large collection of authors describes a “consensus-based transparency checklist” in the Dec 2, 2019 Comment in Nature Human Behavior. Hey—I’m one of those 80 authors! Let’s see what Higgs has to say: I [Higgs] have mixed emotions about it — the positive aspects are easy to see, but I also have […]

We are stat professors with the American Statistical Association, and we’re thrilled to talk to you about the statistics behind voting. Ask us anything!

It’s happening at 11am today on Reddit. It’s a real privilege to do this with Mary Gray, who was so nice to me back when I took a class at American University several decades ago.

Fiction as a window into other cultures

tl;dr: more on Updike. In our recent discussion of reviews of John Updike books, John Bullock pointed us to this essay by Claire Lowdon, who begins: In the opening scene of Rabbit, Run (1960), John Updike’s second published novel, the twenty-six-year-old Harry Angstrom – aka Rabbit – joins some children playing basketball around a telephone […]

More on martingale property of probabilistic forecasts and some other issues with our election model

Edward Yu writes: I’m wondering if you’ve seen Nassim Taleb’s article arguing that we should price election forecasts as binary options. You seem to be generally fine with this approach, as when Nate Silver asked your colleague: On the off-chance our respective employers would allow it, which they almost certainly wouldn’t in my case, could […]

Don’t ever change, social psychology! You’re perfect just the way you are

Richard Juster points us to this article, “Vocal characteristics predict infidelity intention and relationship commitment in men but not in women,” where by “men,” they meant 88 male college students, and by “women,” then meant 128 female college students, and by “predict,” they meant not very well. The story was featured in various classy news […]

“Everybody wants to be Jared Diamond”

As the saying goes, “Everybody wants to be Jared Diamond, that’s the problem.” (See also here and here.) The funny thing is, this principle also applies to . . . Jared Diamond himself! See this review by Anand Giridharadas, sent to me by Mark Palko.

Response to a question about a reference in one of our papers

Tushar Sunkum writes: I like this particular study that you did [with Jeff Fagan and Alex Kiss] on racial profiling. However, I believe that you misrepresented one of the sources on the paper. You state, “For example, two surveys with nationwide probability samples, completed in 1999 and in 2002, showed that African-Americans were far more […]

“Stop me if you’ve heard this one before: Ivy League law professor writes a deepthoughts think piece explaining a seemingly irrational behavior that doesn’t actually exist.”

Under the heading, “Stop me if you’ve heard this one before,” Palko writes: An Ivy League law professor writes a deepthoughts think piece explaining a seemingly irrational behavior that doesn’t actually exist. (see here and here) My favorite bit is this, from the Ivy League law professor in question: What’s more, macroeconomists have typically spent […]