I would very much be interested in this!

]]>I said it would be automatic, but not that it would be easy!

Seriously, the automaticity of the inferences is all conditional on the model, and you’re right that we can’t grab a model off the shelf for this problem. My recommendation would be something like your Option 2 but with some changes for computational efficiency. Just to start, I’d treat political party as a continuous predictor rather than as a factor, and I’d treat male/female as a linear predictor, not a factor with two levels. I might also want to use informative priors for the group-level variance parameters.

This question comes up a lot. To start with, I’d want to go through one example from beginning to end, which I haven’t done yet.

]]>I still haven’t quite figured out how to do this “automatically” in a Bayesian multilevel model. Imagine I have a treatment or control condition (X) and then a bunch of covariates (Z): gender (2 levels), education (5 levels), race (4 levels), political party (3 levels), income (5 levels).

Option 1: We could make a grouping variable of every combination, for 600 unique groups. The rstanarm::stan_lmer code would be: y ~ x + (1 + x | group). We could look at the random slope coefficient of x and see which groups we are confident there is an effect vs. not, or do specific contrasts in comparing groups. However: (a) This doesn’t lead itself to easily looking at marginal comparisons, such as men, black men, white women democrats, high income people, and (b) the groups become very sparse—some might have only 1 person in the group, making the calculation of a slope for that group impossible.

Option 2: Crossed random factors design, which would be y ~ x + (1 + x | gender) + (1 + x | education) + (1 + x | race) + (1 + x | political_party) + (1 + x | income). However: (a) this doesn’t lend itself to finding more granular combinations, such as upper class white women or lower income college educated men, and (b) there are so few levels within groups that it becomes difficult to estimate a variance. Such as, how do we estimate a variance of the intercept and slope when there is just two gender groups? I get divergent transitions in rstanarm doing this.

]]>