Skip to content
Archive of posts filed under the Multilevel Modeling category.

Smoothness, or lack thereof, in MRP estimates over time

Matthew Loop writes: I’m taking my first crack at MRP. We are estimating the probability of an event over 30 years, adjusting for sampling stratum using a multilevel model with varying intercepts for stratum. When we fit the model, the marginal predicted probability vs. year is a smooth function, since the mean of the varying […]

Fitting big multilevel regressions in Stan?

Joe Hoover writes: I am a social psychology PhD student, and I have some questions about applying MrP to estimation problems involving very large datasets or many sub-national units. I use MrP to obtain sub-national estimates for low-level geographic units (e.g. counties) derived from large data (e.g. 300k-1 million+). In addition to being large, my […]

Causal inference and within/between person comparisons

There’s a meta-principle of mathematics that goes as follows. Any system of logic can be written in various different ways that are mathematically equivalent but can have different real-world implications, for two reasons: first, because different formulations can be more directly applied in different settings or are just more understandable by different people; second, because […]

Postdoctoral research position on survey research with us at Columbia School of Social Work

Here it is: The Center on Poverty and Social Policy at the Columbia University School of Social Work, the Columbia Population Research Center, and the Institute for Social and Economic Research and Policy are seeking a postdoctoral scholar with a PhD in statistics, economics, political science, public policy, demography, psychology, social work, sociology, or a […]

“Deep Origins” and spatial correlations

Morgan Kelly writes: Back in 2013 you had a column in Chance magazine on the Ashraf-Galor “Out of Africa” paper which claims that genetic diversity determines modern income. That paper is part of a much large literature in economics on Persistence or “Deep Origins” that shows how medieval pogroms prefigure Nazi support, adoption of the […]

“Some call it MRP, some Mister P, but the full name is . . .”

Jim Savage points us to this explainer, How do pollsters predict UK general election results?, by John Burn-Murdoch of the Financial Times. It’s bittersweet seeing my method described by some person I’ve never met. Little baby MRP is all grown up! Being explained by the Financial Times—that’s about as good as being in the Guardian […]

My talk at Yale this Thursday

It’s the Quantitative Research Methods Workshop, 12:00-1:15 p.m. in Room A002 at ISPS, 77 Prospect Street Slamming the sham: A Bayesian model for adaptive adjustment with noisy control data Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University It is not always clear how to adjust for control data in causal inference, […]

Why do a within-person rather than a between-person experiment?

Zach Horne writes: A student of mine was presenting at the annual meeting of the Law and Society Association. She sent me this note after she gave her talk: I presented some research at LSA which used a within subject design. I got attacked during the Q&A session for using a within subjects design and […]

The dropout rate in his survey is over 60%. What should he do? I suggest MRP.

Alon Honig writes: I work for a cpg company that conducts longitudinal surveys for analysis of customer behavior. In particular they wanted to know how people are interacting with our product. Unfortunately the designers of these surveys put so many questions (100+) that the dropout rate (those that did not complete the survey) was over […]

Software for multilevel conjoint analysis in marketing

Someone writes: The CBC-HB and HB-Reg programs produced by Sawtooth Software are quite popular among marketing researchers and, essentially, introduced hierarchical Bayes to the marketing research community. They have been around for nearly 20 years. More recent versions I don’t have offer jackknifing, and there have been other enhancements. I’m not sure how well-known Sawtooth […]

“Causal Processes in Psychology Are Heterogeneous”

Niall Bolger sends along this article he wrote with Katherine Zee, Maya Rossignac-Milon, and Ran Hassin, which begins: All experimenters know that human and animal subjects do not respond uniformly to experimental treatments. Yet theories and findings in experimental psychology either ignore this causal effect heterogeneity or treat it as uninteresting error. This is the […]

We’re hiring an econ postdoc!

It’s for hierarchical modeling for policy analysis in Stan. We’re really excited about this project. Will share more details soon, but wanted to get this out right away.

Challenge of A/B testing in the presence of network and spillover effects

Gaurav Sood writes: There is a fun problem that I recently discovered: Say that you are building a news recommender that lists which relevant news items in each person’s news feed. Say that your first version of the news recommender is a rules-based system that uses signals like how many people in your network have […]

All the names for hierarchical and multilevel modeling

The title Data Analysis Using Regression and Multilevel/Hierarchical Models hints at the problem, which is that there are a lot of names for models with hierarchical structure. Ways of saying “hierarchical model” hierarchical model a multilevel model with a single nested hierarchy (note my nod to Quine’s “Two Dogmas” with circular references) multilevel model a […]

Multilevel structured (regression) and post-stratification

My enemies are all too familiar. They’re the ones who used to call me friend – Jawbreaker Well I am back from Australia where I gave a whole pile of talks and drank more coffee than is probably a good idea. So I’m pretty jetlagged and I’m supposed to be writing my tenure packet, so […]

What can be learned from this study?

James Coyne writes: A recent article co-authored by a leading mindfulness researcher claims to address the problems that plague meditation research, namely, underpowered studies; lack of or meaningful control groups; and an exclusive reliance on subjective self-report measures, rather than measures of the biological substrate that could establish possible mechanisms. The article claims adequate sample […]

As always, I think the best solution is not for researchers to just report on some preregistered claim, but rather for them to display the entire multiverse of possible relevant results.

I happened to receive these two emails in the same day. Russ Lyons pointed to this news article by Jocelyn Kaiser, “Major medical journals don’t follow their own rules for reporting results from clinical trials,” and Kevin Lewis pointed to this research article by Kevin Murphy and Herman Aguinis, “HARKing: How Badly Can Cherry-Picking and […]

“Beyond ‘Treatment Versus Control’: How Bayesian Analysis Makes Factorial Experiments Feasible in Education Research”

Daniel Kassler, Ira Nichols-Barrer, and Mariel Finucane write: Researchers often wish to test a large set of related interventions or approaches to implementation. A factorial experiment accomplishes this by examining not only basic treatment–control comparisons but also the effects of multiple implementation “factors” such as different dosages or implementation strategies and the interactions between these […]

Multilevel Bayesian analyses of the growth mindset experiment

Jared Murray, one of the coauthors of the Growth Mindset study we discussed yesterday, writes: Here are some pointers to details about the multilevel Bayesian modeling we did in the Nature paper, and some notes about ongoing & future work. We did a Bayesian analysis not dissimilar to the one you wished for! In section […]

“Study finds ‘Growth Mindset’ intervention taking less than an hour raises grades for ninth graders”

I received this press release in the mail: Study finds ‘Growth Mindset’ intervention taking less than an hour raises grades for ninth graders Intervention is first to show national applicability, breaks new methodological ground – Study finds low-cost, online growth mindset program taking less than an hour can improve ninth graders’ academic achievement – The […]