Dave Judkins wrote:

If the user sets up a fully parallel structure for each outcome, both in terms of fixed effects and random effects, it seems to me that there is no possibility of precision gain. (Precision for the relationship of a single putative causal to any of several outcome variables.) One of my colleagues swears differently. He follows the same setup of a multivariate HLM model as Hauck and Street in their online paper. (Equation #9 here.) In what I understand of your work, there is some borrowing of strength across outcomes where you assume that there is a single latent trait underlying the set of multiple outcomes and you relate the putative causal agent to that single latent outcome variable. You also estimate outcome-specific effects by shrinking them to the effect on the latent variable. Now I can see that this would yield precision improvements. But all I see from the Hauck and Street approach is that one is able to study correlations that would otherwise be inaccessible.

This matters because we have a mixed set of binary and continuous outcomes. Trying to fit them into a unified multivariate multi-level model is a pain. I would like to make sure that there is really some benefit to be had.

Chapter 19 of your book with Carlin, Stern and Rubin seems to be richer on “how” than on “why bother”.

My reply: To answer the last question first: If you don’t see why it’s worth bothering after the first 18 chapters, then you probably already have a method that is reasonable for your problems, in which case I wouldn’t try to talk you into switching.

Getting to the main point: I’m not sure, I haven’t looked into it carefully. My intuition is that when everything is balanced, the relative positions of the estimates of the different groups won’t change, but the numerical estimates will change (because of partial pooling) and statistical significance of comparisons will change (in general, making the intervals more sensible). In econometrics the questions are usually framed in terms of how they change the estimate of some particular “beta” of interest, perhaps a group-level predictor or a non-varying indivdual-level predictor. I could see that the in a balanced design, multilevel modeling might not change beta.hat, it might only change the standard error.

The easiest way to check is with simulated data, I think. Either simulating from a model or subsampling data from a survey.

Dave replied:

Since sending you this question, my colleague alerted me to the econometric literature on seemingly unrelated regression that started with Zellner in 1962. There are situations where running parallel models without shared parameters can increase power. This was more clearly articulated in a December 1982 JASA article by J.A. Binkley. I see now that this literature is related to C19 of GCSR, but that you all also do not mention it.

My comment on Chapter 19 was too laconic. What I meant is that this chapter says how to model multiple outcomes simultaneously but doesn’t articulate the advantages of doing so (as opposed to using Bayesian methods to model them separately). You and Jennifer treat this slightly in your GH section 21.8, but this section also does not explicitly address power improvement, does it?

So the question that I thought would be interesting concerns the advantages of simultaneous modeling of related outcomes as opposed to a series of separate models.