Skip to content
Archive of posts filed under the Multilevel Modeling category.

My talk at Yale this Thursday

It’s the Quantitative Research Methods Workshop, 12:00-1:15 p.m. in Room A002 at ISPS, 77 Prospect Street Slamming the sham: A Bayesian model for adaptive adjustment with noisy control data Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University It is not always clear how to adjust for control data in causal inference, […]

Why do a within-person rather than a between-person experiment?

Zach Horne writes: A student of mine was presenting at the annual meeting of the Law and Society Association. She sent me this note after she gave her talk: I presented some research at LSA which used a within subject design. I got attacked during the Q&A session for using a within subjects design and […]

The dropout rate in his survey is over 60%. What should he do? I suggest MRP.

Alon Honig writes: I work for a cpg company that conducts longitudinal surveys for analysis of customer behavior. In particular they wanted to know how people are interacting with our product. Unfortunately the designers of these surveys put so many questions (100+) that the dropout rate (those that did not complete the survey) was over […]

Software for multilevel conjoint analysis in marketing

Someone writes: The CBC-HB and HB-Reg programs produced by Sawtooth Software are quite popular among marketing researchers and, essentially, introduced hierarchical Bayes to the marketing research community. They have been around for nearly 20 years. More recent versions I don’t have offer jackknifing, and there have been other enhancements. I’m not sure how well-known Sawtooth […]

“Causal Processes in Psychology Are Heterogeneous”

Niall Bolger sends along this article he wrote with Katherine Zee, Maya Rossignac-Milon, and Ran Hassin, which begins: All experimenters know that human and animal subjects do not respond uniformly to experimental treatments. Yet theories and findings in experimental psychology either ignore this causal effect heterogeneity or treat it as uninteresting error. This is the […]

We’re hiring an econ postdoc!

It’s for hierarchical modeling for policy analysis in Stan. We’re really excited about this project. Will share more details soon, but wanted to get this out right away.

Challenge of A/B testing in the presence of network and spillover effects

Gaurav Sood writes: There is a fun problem that I recently discovered: Say that you are building a news recommender that lists which relevant news items in each person’s news feed. Say that your first version of the news recommender is a rules-based system that uses signals like how many people in your network have […]

All the names for hierarchical and multilevel modeling

The title Data Analysis Using Regression and Multilevel/Hierarchical Models hints at the problem, which is that there are a lot of names for models with hierarchical structure. Ways of saying “hierarchical model” hierarchical model a multilevel model with a single nested hierarchy (note my nod to Quine’s “Two Dogmas” with circular references) multilevel model a […]

Multilevel structured (regression) and post-stratification

My enemies are all too familiar. They’re the ones who used to call me friend – Jawbreaker Well I am back from Australia where I gave a whole pile of talks and drank more coffee than is probably a good idea. So I’m pretty jetlagged and I’m supposed to be writing my tenure packet, so […]

What can be learned from this study?

James Coyne writes: A recent article co-authored by a leading mindfulness researcher claims to address the problems that plague meditation research, namely, underpowered studies; lack of or meaningful control groups; and an exclusive reliance on subjective self-report measures, rather than measures of the biological substrate that could establish possible mechanisms. The article claims adequate sample […]

As always, I think the best solution is not for researchers to just report on some preregistered claim, but rather for them to display the entire multiverse of possible relevant results.

I happened to receive these two emails in the same day. Russ Lyons pointed to this news article by Jocelyn Kaiser, “Major medical journals don’t follow their own rules for reporting results from clinical trials,” and Kevin Lewis pointed to this research article by Kevin Murphy and Herman Aguinis, “HARKing: How Badly Can Cherry-Picking and […]

“Beyond ‘Treatment Versus Control’: How Bayesian Analysis Makes Factorial Experiments Feasible in Education Research”

Daniel Kassler, Ira Nichols-Barrer, and Mariel Finucane write: Researchers often wish to test a large set of related interventions or approaches to implementation. A factorial experiment accomplishes this by examining not only basic treatment–control comparisons but also the effects of multiple implementation “factors” such as different dosages or implementation strategies and the interactions between these […]

Multilevel Bayesian analyses of the growth mindset experiment

Jared Murray, one of the coauthors of the Growth Mindset study we discussed yesterday, writes: Here are some pointers to details about the multilevel Bayesian modeling we did in the Nature paper, and some notes about ongoing & future work. We did a Bayesian analysis not dissimilar to the one you wished for! In section […]

“Study finds ‘Growth Mindset’ intervention taking less than an hour raises grades for ninth graders”

I received this press release in the mail: Study finds ‘Growth Mindset’ intervention taking less than an hour raises grades for ninth graders Intervention is first to show national applicability, breaks new methodological ground – Study finds low-cost, online growth mindset program taking less than an hour can improve ninth graders’ academic achievement – The […]

Allowing intercepts and slopes to vary in a logistic regression: how does this change the ROC curve?

Jonathan Hughes writes: I am an engineering doctoral student. As part of my dissertation I’m proposing a mode of adaptation for a predictive system to individual subgroup specific streams of data which come each from a specific subgroup of a mixture population distribution. As part of the proposal presentation someone referenced your work and believed […]

The garden of forking paths

Bert Gunter points us to this editorial: So, researchers using these data to answer questions about the effects of technology [screen time on adolescents] need to make several decisions. Depending on the complexity of the data set, variables can be statistically analysed in trillions of ways. This makes almost any pattern of results possible. As […]

I don’t have a clever title but this is an interesting paper

Why do we, as a discipline, have so little understanding of the methods we have created and promote? Our primary tool for gaining understanding is mathematics, which has obvious appeal: most of us trained in math and there is no better form of information than a theorem that establishes a useful fact about a method. […]

The Economist does Mister P

Elliott Morris points us to this magazine article, “If everyone had voted, Hillary Clinton would probably be president,” which reports: Close observers of America know that the rules of its democracy often favour Republicans. But the party’s biggest advantage may be one that is rarely discussed: turnout is just 60%, low for a rich country. […]

Causal inference with time-varying mediators

Adan Becerra writes to Tyler VanderWeele: I have a question about your paper “Mediation analysis for a survival outcome with time-varying exposures, mediators, and confounders” that I was hoping that you could help my colleague (Julia Ward) and me with. We are currently using Medicare claims data to evaluate the following general mediation among dialysis […]

The garden of 603,979,752 forking paths

Amy Orben and Andrew Przybylski write: The widespread use of digital technologies by young people has spurred speculation that their regular use negatively impacts psychological well-being. Current empirical evidence supporting this idea is largely based on secondary analyses of large-scale social datasets. Though these datasets provide a valuable resource for highly powered investigations, their many […]