Daniel Kramer writes:

In your book, Data Analysis Using Regression and Multilevel…, you state that in situations when little group-level variation is observed, multilevel models reduce to classical regression with no group indicators. Does this essentially mean that with zero group-variation, predicted coefficients, deviance, and AIC would be the same to estimates obtained with classical regression? I ask because I have been asked by an editor to adopt a multimodel inference approach (Burnham and Anderson 2002) in my analysis. Typically a small set of candidate models are ranked using an information theoretic criterion and model averaging may be used to derive coefficient estimates or predictions. Thus, would it be appropriate to compare single-level and multi-level models derived from the same data set using AIC? I am skeptical since the deviance for the null models are different. Of course, there may be no reason to compare single and multi-level models if there is no cost (i.e. reduced model fit) in adopting a multi-level framework as long as the case can be made that the data are hierarchical. The only cost you mention in your book is the added complexity.

My reply: If you’re fitting multilevel models, perhaps better to use DIC than AIC. DIC has problems too (search this blog for DIC for discussion of this point), but with AIC you’ll definitely run into trouble counting parameters. (BIC has other, more serious problems: it’s not a measure of predictive error the way that AIC and DIC are.) More generally, I can see the appeal of presenting and averaging over several models, even if I’ve rarely ended up doing that myself. Indeed, I’d prefer to let everything of interest vary by group.

P.S. What’s the deal with the 256-character limit on titles, anyway?