Skip to content
Archive of entries posted by

The value of thinking about varying treatment effects: coronavirus example

Yesterday we discussed difficulties with the concept of average treatment effect. Part of designing a study is accounting for uncertainty in effect sizes. Unfortunately there is a tradition in clinical trials of making optimistic assumptions in order to claim high power. Here is an example that came up in March, 2020. A doctor was designing […]

Understanding the “average treatment effect” number

In statistics and econometrics there’s lots of talk about the average treatment effect. I’ve often been skeptical of the focus on the average treatment effect, for the simple reason that, if you’re talking about an average effect, then you’re recognizing the possibility of variation; and if there’s important variation (enough so that we’re talking about […]

Embracing Variation and Accepting Uncertainty (my talk this Wed/Tues at a symposium on communicating uncertainty)

I’ll be speaking (virtually) at this conference in Australia on Wed 1 July (actually Tues 30 June in our time zone here): Embracing Variation and Accepting Uncertainty It is said that your most important collaborator is yourself in 6 months. Perhaps the best way to improve our communication of data uncertainty to others is to […]

Shortest posterior intervals

By default we use central posterior intervals. For example, the central 95% interval is the (2.5%, 97.5%) quantiles. But sometimes the central interval doesn’t seem right. This came up recently with a coronavirus testing example, where the posterior distribution for the parameter of interest was asymmetric so that the central interval is not such a […]

This one quick trick will allow you to become a star forecaster

Jonathan Falk points us to this wonderful post by Dario Perkins. It’s all worth a read, but, following Falk, I want to emphasize this beautiful piece of advice, which is #5 on their list of 10 items: How to get attention: If you want to get famous for making big non-consensus calls, without the danger […]

Validating Bayesian model comparison using fake data

A neuroscience graduate student named James writes in with a question regarding validating Bayesian model comparison using synthetic data: I [James] perform an experiment and collect real data. I want to determine which of 2 candidate models best accounts for the data. I perform (approximate) Bayesian model comparison (e.g., using BIC – not ideal I […]

The two most important formulas in statistics

0.5/sqrt(n) (which in turn is short for sqrt(p*(1-p)/n) 5^2 + 12^2 = 13^2 With an honorable mention to 16.

“Why do the results of immigrant students depend so much on their country of origin and so little on their country of destination?”

Aleks points us to this article from 2011 by Julio Carabaña. Carabaña’s article has three parts. First is a methodological point that much can be learned from a cross-national study that has data at the level of individual students, as compared to the usual “various origins-one destination” design. Second is the empirical claim, based on […]

Roll Over Mercator: Awesome map shows the unreasonable effectiveness of mixture models

I’m not gonna link to all the great xkcd drawings cos if I did, I’d just be linking to xkcd every day, but today’s is just too good to pass by: He could’ve thrown in some Pacific islands and Scandinavia too, but it’s amazing in any case. The relevant statistical point here is how good […]

Election odds update (Biden still undervalued but not by so much)

Last week I wrote about the discrepancy between our election forecast and the betting odds: Suppose I were to lay $1000 on Biden right now. According to Betfair it seems that, if I win, I make a profit of $840. And our model gives Biden an 88% chance of winning. But we’re modeling Biden vs. […]

Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors

Yuling, Aki, and I write: When working with multimodal Bayesian posterior distributions, Markov chain Monte Carlo (MCMC) algorithms can have difficulty moving between modes, and default variational or mode-based approximate inferences will understate posterior uncertainty. And, even if the most important modes can be found, it is difficult to evaluate their relative weights in the […]

Resolving confusions over that study of “Teacher Effects on Student Achievement and Height”

Someone pointed me to this article by Marianne Bitler, Sean Corcoran, Thurston Domina, and Emily Penner, “Teacher Effects on Student Achievement and Height: A Cautionary Tale,” which begins: Estimates of teacher “value-added” suggest teachers vary substantially in their ability to promote student learning. Prompted by this finding, many states and school districts have adopted value-added […]

Hey, this was an unusual media request

This popped up in the inbox: Hi Professor Gelman – my name is ** and I’m a journalist who reports on issues of ** conducted by **. Recently ** announced the department was working with “research groups” to study and analyze the **. To further my reporting on this issue, I am reaching out to […]

No, there is no “tension between getting it fast and getting it right”

When reading Retraction Watch, I came across this quote: “There is always a tension between getting it fast and getting it right,” said Dr. Marcia Angell, another former editor in chief of the New England Journal of Medicine. “I always favored getting it right. But in the current pandemic, that balance may have shifted too […]

Retraction of racial essentialist article that appeared in Psychological Science

Scene 1: It all started for me on 2 Jan when I received this email from Keith Donohue in Fargo, North Dakota: I am a longtime reader, and I am curious about your reaction to an (in press) journal article that I recently came across. . . . The paper is “Declines in Religiosity Predicted […]

Against overly restrictive definitions: No, I don’t think it helps to describe Bayes as “the analysis of subjective
 beliefs” (nor, for that matter, does it help to characterize the statements of Krugman or Mankiw as not being “economics”)

I get frustrated when people use overly restrictive definitions of something they don’t like. Here’s an example of an overly restrictive definition that got me thinking about all this. Larry Wasserman writes (as reported by Deborah Mayo): I wish people were clearer about what Bayes is/is not and what 
frequentist inference is/is not. Bayes is […]

“The Intellectuals and the Masses”

I just read “The Intellectuals and the Masses,” a book from 1992 by the literary critic and English professor John Carey. I really liked the book, and after finishing it I decided to get some further perspective by reading some reviews. I found two excellent reviews online, a negative review in the London Independent by […]

Do we really believe the Democrats have an 88% chance of winning the presidential election?

OK, enough about coronavirus. Time to talk about the election. Dhruv Madeka starts things off with this email: Someone just forwarded me your election model (with Elliott Morris and Merlin Heidemanns) for the Economist. I noticed Biden was already at 84%. I wrote a few years ago about how the time to election factors a […]

“Laplace’s Demon: A Seminar Series about Bayesian Machine Learning at Scale” and my answers to their questions

Here’s the description of the online seminar series: Machine learning is changing the world we live in at a break neck pace. From image recognition and generation, to the deployment of recommender systems, it seems to be breaking new ground constantly and influencing almost every aspect of our lives. In ths seminar series we ask […]

“Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe”

Seth Flaxman writes: Our work on non-pharmaceutical interventions in 11 European countries (originally Imperial report 13) is now published in Nature, Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Of note for your readers: 1) Nature has an open peer review process, so you can see the (pre-publication) peer review here. 2) Between […]