Political scientist and Stan user Josh Alley points to this new paper on using hierarchical (multilevel) models to estimate heterogeneous effects.
I like it. The only thing I’d add is an emphasis that group-level predictors can be important. Jennifer and I discuss this in our multilevel modeling book too. For example, when doing MRP with a multilevel model for the 50 states, we also include indicators for region of the country and state-level predictors such as Republican vote share in the previous election. If you don’t do that, you partially pool small states to the national mean, which is a mistake.
The error for people to avoid is to naively think that just because you’ve fit a multilevel model, that you’ve now “controlled for” the groups and so group-level predictors are superfluous. That would be a mistake. Group-level predictors are important; indeed, one of the advantages of multilevel modeling is to facilitate the inclusion of group-level predictors.
“Group-level predictors are important; indeed, one of the advantages of multilevel modeling is to facilitate the inclusion of group-level predictors.”
OLS models can have group-level predictors, no?
Jp:
With least squares regression you can include group indicators and group-level predictors, but not both.
Completely agree! Perhaps you will find our paper with Guido on a related subject interesting: https://academic.oup.com/restud/advance-article/doi/10.1093/restud/rdad089/7262519
I think this is a good example because too often the econometrics perspective is to compare unregularized models (fixed effects) with the same model with regularization (random effects), rather than considering that regularization can allow using an otherwise more flexible model, as can come up in your case.
Andrew, have you any thoughts on other correlates than Republican vote share to include? I’ve used other things sometimes such as vaccination percentage, and also thought of things like median income, population density, %age evangelical and so on. The issue I find is that so often, all these things are incredibly highly correlated with one another, so I then worry if that might interfere or in some way make things go wrong in the model!
Jamie:
My favorite example is an MRP analysis of attitudes toward gun control in U.S. states. Two natural state-level predictors to consider are %Republican vote in previous election and %rural. We’d expect these two variables to predict opposition to gun control and also to be positively correlated with each other. But what about Vermont, one of the most Democratic-voting states and one of the most rural states in the country? The prediction for Vermont will depend crucially on the coefficients of those two state-level predictors. In this case, I think the right thing to do is to include both predictors, even if they are highly correlated, and use moderately strong priors to keep both their coefficients positive.