Skip to content
Archive of posts filed under the Multilevel Modeling category.

Using multilevel modeling to improve analysis of multiple comparisons

Justin Chumbley writes: I have mused on drafting a simple paper inspired by your paper “Why we (usually) don’t have to worry about multiple comparisons”. The initial idea is simply to revisit frequentist “weak FWER” or “omnibus tests” (which assume the null everywhere), connecting it to a Bayesian perspective. To do this, I focus on […]

Comparing racism from different eras: If only Tucker Carlson had been around in the 1950s he could’ve been a New York Intellectual.

TV commentator Carlson in 2018 recently raised a stir by saying that immigration makes the United States “poorer, and dirtier, and more divided,” which reminded me of this rant from literary critic Alfred Kazin in 1957: Kazin put it in his diary and Carlson broadcast it on TV, so not quite the same thing. But […]

Classifying yin and yang using MRI

Zad Chow writes: I wanted to pass along this study I found a while back that aimed to see whether there was any possible signal in an ancient Chinese theory of depression that classifies major depressive disorder into “yin” and “yang” subtypes. The authors write the following, The “Yin and Yang” theory is a fundamental […]

Should we be concerned about MRP estimates being used in later analyses? Maybe. I recommend checking using fake-data simulation.

Someone sent in a question (see below). I asked if I could post the question and my reply on blog, and the person responded: Absolutely, but please withhold my name because this is becoming a touchy issue within my department. The boldface was in the original. I get this a lot. There seems to be […]

Multilevel models for multiple comparisons! Varying treatment effects!

Mark White writes: I have a question regarding using multilevel models for multiple comparisons, per your 2012 paper and many blog posts. I am in a situation where I do randomized experiments, and I have a lot of additional demographic information about people, as well. For the moment, let us just assume that all of […]

“Economic predictions with big data” using partial pooling

Tom Daula points us to this post, “Economic Predictions with Big Data: The Illusion of Sparsity,” by Domenico Giannone, Michele Lenza, and Giorgio Primiceri, and writes: The paper wants to distinguish between variable selection (sparse models) and shrinkage/regularization (dense models) for forecasting with Big Data. “We then conduct Bayesian inference on these two crucial parameters—model […]

2018: How did people actually vote? (The real story, not the exit polls.)

Following up on the post that we linked to last week, here’s Yair’s analysis, using Mister P, of how everyone voted. Like Yair, I think these results are much better than what you’ll see from exit polls, partly because the analysis is more sophisticated (MRP gives you state-by-state estimates in each demographic group), partly because […]

Hey! Here’s what to do when you have two or more surveys on the same population!

This problem comes up a lot: We have multiple surveys of the same population and we want a single inference. The usual approach, applied carefully by news organizations such as Real Clear Politics and Five Thirty Eight, and applied sloppily by various attention-seeking pundits every two or four years, is “poll aggregation”: you take the […]

2018: Who actually voted? (The real story, not the exit polls.)

Continuing from our earlier discussion . . . Yair posted some results from his MRP analysis of voter turnout: 1. The 2018 electorate was younger than in 2014, though not as young as exit polls suggest. 2. The 2018 electorate was also more diverse, with African American and Latinx communities surpassing their share of votes […]

“What Happened Next Tuesday: A New Way To Understand Election Results”

Yair just published a long post explaining (a) how he and his colleagues use Mister P and the voter file to get fine-grained geographic and demographic estimates of voter turnout and vote preference, and (b) why this makes a difference. The relevant research paper is here. As Yair says in his above-linked post, he and […]

MRP (or RPP) with non-census variables

It seems to be Mister P week here on the blog . . . A question came in, someone was doing MRP on a political survey and wanted to adjust for political ideology, which is a variable that they can’t get poststratification data for. Here’s what I recommended: If a survey selects on a non-census […]

Can we do better than using averaged measurements?

Angus Reynolds writes: Recently a PhD student at my University came to me for some feedback on a paper he is writing about the state of research methods in the Fear Extinction field. Basically you give someone an electric shock repeatedly while they stare at neutral stimuli and then you see what happens when you […]

Multilevel models with group-level predictors

Kari Lock Morgan writes: I’m writing now though with a multilevel modeling question that has been nagging me for quite some time now. In your book with Jennifer Hill, you include a group-level predictor (for example, 12.15 on page 266), but then end up fitting this as an individual-level predictor with lmer. How can this […]

Perhaps you could try a big scatterplot with one dot per dataset?

Joe Nadeau writes: We are studying variation in both means and variances in metabolic conditions. We have access to nearly 200 datasets that involve a range of metabolic traits and vary in sample size, mean effects, and variance. Some traits differ in mean but not variance, others in variance but not mean, still others in […]

Cool postdoc position in Arizona on forestry forecasting using tree ring models!

Margaret Evans sends in this cool job ad: Two-Year Post Doctoral Fellowship in Forest Ecological Forecasting, Data Assimilation A post-doctoral fellowship is available in the Laboratory of Tree-Ring Research (University of Arizona) to work on an NSF Macrosystems Biology-funded project assimilating together tree-ring and forest inventory data to analyze patterns and drivers of forest productivity […]

You’ve got data on 35 countries, but it’s really just N=3 groups.

Jon Baron points to a recent article, “Societal inequalities amplify gender gaps in math,” by Thomas Breda, Elyès Jouini, and Clotilde Napp (supplementary materials here), and writes: A particular issue bothers me whenever I read studies like this, which use nations as the unit of analysis and then make some inference from correlations across nations. […]

Multilevel data collection and analysis for weight training (with R code)

[image of cat lifting weights] A graduate student who wishes to remain anonymous writes: I was wondering if you could answer an elementary question which came to mind after reading your article with Carlin on retrospective power analysis. Consider the field of exercise science, and in particular studies on people who lift weights. (I sometimes […]

The hot hand—in darts!

Roland Langrock writes: Since on your blog you’ve regularly been discussing hot hand literature – which we closely followed – I’m writing to share with you a new working paper we wrote on a potential hot hand pattern in professional darts. We use state-space models in which a continuous-valued latent “hotness” variable, modeled as an […]

“Dynamically Rescaled Hamiltonian Monte Carlo for Bayesian Hierarchical Models”

Aki points us to this paper by Tore Selland Kleppe, which begins: Dynamically rescaled Hamiltonian Monte Carlo (DRHMC) is introduced as a computationally fast and easily implemented method for performing full Bayesian analysis in hierarchical statistical models. The method relies on introducing a modified parameterisation so that the re-parameterised target distribution has close to constant […]

When anyone claims 80% power, I’m skeptical.

A policy analyst writes: I saw you speak at ** on Bayesian methods. . . . I had been asked to consult on a large national evaluation of . . . [details removed to preserve anonymity] . . . and had suggested treading carefully around the use of Bayesian statistics in this study (basing it […]