Partha Lahiri writes, in reference to my 2006 paper:
I am interested in finding out a good prior for the regression coefficients and variance components in a multi-level setting. For concreteness, let’s say we have a model like the following:
Level 1: Y_ijk | theta_ij ~(ind) N( theta_ij, sigma^2)
Level 2: theta_ij| mu_i ~(ind) N( mu_i, tau^2)
Level 3: mu_i|beta, ~(ind) N( x_i’beta, delta^2)One possibility is to assume independent uniform proper priors with large length for beta and the standard deviations sigma, tau and delta. Since all the distributions are proper, the posterior will be proper. I need to check if anyone talked about propriety of posterior if we make these proper uniform to improper uniform (e.g., delta is uniform in (0,\infty)) in order to avoid insensitivity of the results (in the lines of your section 2.2).
Another possibility is to use improper uniform for beta (usually it does not cause problem with impropriety of posterior – but need to check) and independent half Cauchy on the standard deviation (need an appropriate scale for half Cauchy).
My reply:
If you have enough groups you can get away with just about anything, but in the real world you won’t have enough groups so I think it’s best for you to use an informative prior. I recommend weakly or strongly informative priors on sigma, tau, and delta. Maybe I’d call these sigma_y, sigma_theta, and sigma_mu, actually. For sigma_y you’ll have a lot of data so maybe you don’t even need to bother, but you can do it just for completeness. For sigma_theta and, especially, sigma_mu, you’ll want something real.
Here’s a start: make sure your data y and predictors x are all dimensionless or on a unit scale so that differences of 1 are large. Then put half-normal(0,1) priors on the 3 scale parameters (sigma_y, sigma_theta, sigma_mu) and independent normal(0,1) priors on the betas.
Aki would probably recommend t_7 and he’s probably right, but recently I’ve been just using normals. Of course these priors might not make sense for your problem. They’re the default: use them as an anchor or starting-off point.
Or t_3 as in “Projection predictive variable selection using Stan+R” https://arxiv.org/abs/1508.02502
A clarifying question, Andrew. You suggest putting half-normal(0,1) priors on the 3 scale parameters (sigma_y, sigma_theta, sigma_mu). These are all variance parameters and, so, need to be positive. Do you mean to suggest taking just the POSITIVE draws from an N(0,1) distribution?
…And a comment: It’s seems to me that in most hierarchical model discussions, priors for these variance parameters don’t get as much love and attention as the priors on the means (e.g., priors on x_i’beta, mu_i, or theta_ij). I think it’s because it’s harder for researchers to ground the inverse gamma distribution (which is often used as the prior on these variance terms) in real-world terms. However, these priors are terribly important. They regulate how much “shrinkage” occurs within one’s hierarchical model. I dislike leaving that to ad hoc rules of thumb (e.g., IG(.0001,.0001) which have been tossed around.
MNL:
Yes, by half-normal I mean normal constrained to be positive. We would not actually take draws from this distribution; rather, we would include it in the model and then take draws from the posterior, conditional on the data.
And, yes, I agree that informative priors on the variance parameters are important. In the olden days I’d just use flat priors but that leads to overestimation of these variances, also by using flat priors we’re denying ourself an important opportunity to include real information into our models!
P.S. It’s such a relief to respond to this comment after dealing with all the comments about Hillary Clinton and Ben Carson!