Skip to content
Archive of posts filed under the Multilevel Modeling category.

Hey, people are doing the multiverse!

Elio Campitelli writes: I’ve just saw this image in a paper discussing the weight of evidence for a “hiatus” in the global warming signal and immediately thought of the garden of forking paths. From the paper: Tree representation of choices to represent and test pause-periods. The ‘pause’ is defined as either no-trend or a slow-trend. […]

“MRP is the Carmelo Anthony of election forecasting methods”? So we’re doing trash talking now??

What’s the deal with Nate Silver calling MRP “the Carmelo Anthony of forecasting methods”? Someone sent this to me: and I was like, wtf? I don’t say wtf very often—at least, not on the blog—but this just seemed weird. For one thing, Nate and I did a project together once using MRP: this was our […]

Scandal! Mister P appears in British tabloid.

Tim Morris points us to this news article: And here’s the kicker: Mister P. Not quite as cool as the time I was mentioned in Private Eye, but it’s still pretty satisfying. My next goal: Getting a mention in Sports Illustrated. (More on this soon.) In all seriousness, it’s so cool when methods that my […]

Continuing discussion of status threat and presidential elections, with discussion of challenge of causal inference from survey data

Last year we reported on an article by sociologist Steve Morgan, criticizing a published paper by political scientist Diana Mutz. A couple months later we updated with Mutz’s response to Morgan’s critique. Finally, Morgan has published a reply to Mutz’s response to Morgan’s comments on Mutz’s paper. Here’s a passage that is of methodological interest: […]

R-squared for multilevel models

Brandon Sherman writes: I just was just having a discussion with someone about multilevel models, and the following topic came up. Imagine we’re building a multilevel model to predict SAT scores using many students. First we fit a model on students only, then students in classrooms, then students in classrooms within district, the previous case […]

A question about the piranha problem as it applies to A/B testing

Wicaksono Wijono writes: While listening to your seminar about the piranha problem a couple weeks back, I kept thinking about a similar work situation but in the opposite direction. I’d be extremely grateful if you share your thoughts. So the piranha problem is stated as “There can be some large and predictable effects on behavior, […]

Research topic on the geography of partisan prejudice (more generally, county-level estimates using MRP)

1. An estimate of the geography of partisan prejudice My colleagues David Rothschild and Tobi Konitzer recently published this MRP analysis, “The Geography of Partisan Prejudice: A guide to the most—and least—politically open-minded counties in America,” written up by Amanda Ripley, Rekha Tenjarla, and Angela He. Ripley et al. write: In general, the most politically […]

What’s a good default prior for regression coefficients? A default Edlin factor of 1/2?

The punch line “Your readers are my target audience. I really want to convince them that it makes sense to divide regression coefficients by 2 and their standard errors by sqrt(2). Of course, additional prior information should be used whenever available.” The background It started with an email from Erik van Zwet, who wrote: In […]

Understanding how Anova relates to regression

Analysis of variance (Anova) models are a special case of multilevel regression models, but Anova, the procedure, has something extra: structure on the regression coefficients. As I put it in the rejoinder for my 2005 discussion paper: ANOVA is more important than ever because we are fitting models with many parameters, and these parameters can […]

Mister P for surveys in epidemiology — using Stan!

Jon Zelner points us to this new article in the American Journal of Epidemiology, “Multilevel Regression and Poststratification: A Modelling Approach to Estimating Population Quantities From Highly Selected Survey Samples,” by Marnie Downes, Lyle Gurrin, Dallas English, Jane Pirkis, Dianne Currier, Matthew Spittal, and John Carlin, which begins: Large-scale population health studies face increasing difficulties […]

Estimating treatment effects on rates of rare events using precursor data: Going further with hierarchical models.

Someone points to my paper with Gary King from 1998, Estimating the probability of events that have never occurred: When is your vote decisive?, and writes: In my area of early childhood intervention, there are certain outcomes which are rare. Things like premature birth, confirmed cases of child-maltreatment, SIDS, etc. They are rare enough that […]

“Light Privilege? Skin Tone Stratification in Health among African Americans”

Kevin Lewis points us to this article by Taylor Hargrove, which states: Although skin color represents a particularly salient dimension of race, its consequences for health remains unclear. The author uses four waves of panel data from the Coronary Artery Risk Development in Young Adults study and random-intercept multilevel models to address three research questions […]

“Do you have any recommendations for useful priors when datasets are small?”

Someone who wishes to remain anonymous writes: I just read your paper with Daniel Simpson and Michael Betancourt, The Prior Can Often Only Be Understood in the Context of the Likelihood, and I find it refreshing to read that “the practical utility of a prior distribution within a given analysis then depends critically on both […]

“Using 26,000 diary entries to show ovulatory changes in sexual desire and behavior”

Kevin Lewis points us to this research paper by Ruben Arslan, Katharina Schilling, Tanja Gerlach, and Lars Penke, which begins: Previous research reported ovulatory changes in women’s appearance, mate preferences, extra- and in-pair sexual desire, and behavior, but has been criticized for small sample sizes, inappropriate designs, and undisclosed flexibility in analyses. Examples of such […]

Fitting multilevel models when the number of groups is small

Matthew Poes writes: I have a question that I think you have answered for me before. There is an argument to be made that HLM should not be performed if a sample is too small (too small level 2 and too small level 1 units). Lot’s of papers written with guidelines on what those should […]

Of multiple comparisons and multilevel models

Kleber Neves writes: I’ve been a long-time reader of your blog, eventually becoming more involved with the “replication crisis” and such (currently, I work with the Brazilian Reproducibility Initiative). Anyway, as I’m now going deeper into statistics, I feel like I still lack some foundational intuitions (I was trained as a half computer scientist/half experimental […]

Principal Stratification on a Latent Variable (fitting a multilevel model using Stan)

Adam Sales points to this article with John Pane on principal stratification on a latent variable, and writes: Besides the fact that the paper uses Stan, and it’s about principal stratification, which you just blogged about, I thought you might like it because of its central methodological contribution. We had been trying to use computer […]

“Objective: Generate evidence for the comparative effectiveness for each pairwise comparison of depression treatments for a set of outcomes of interest.”

Mark Tuttle points us to this project by Martijn Schuemie and Patrick Ryan: Large-Scale Population-Level Evidence Generation Objective: Generate evidence for the comparative effectiveness for each pairwise comparison of depression treatments for a set of outcomes of interest. Rationale: In current practice, most comparative effectiveness questions are answered individually in a study per question. This […]

MRP (multilevel regression and poststratification; Mister P): Clearing up misunderstandings about

Someone pointed me to this thread where I noticed some issues I’d like to clear up: David Shor: “MRP itself is like, a 2009-era methodology.” Nope. The first paper on MRP was from 1997. And, even then, the component pieces were not new: we were just basically combining two existing ideas from survey sampling: regression […]

Using multilevel modeling to improve analysis of multiple comparisons

Justin Chumbley writes: I have mused on drafting a simple paper inspired by your paper “Why we (usually) don’t have to worry about multiple comparisons”. The initial idea is simply to revisit frequentist “weak FWER” or “omnibus tests” (which assume the null everywhere), connecting it to a Bayesian perspective. To do this, I focus on […]