Richard Gunton writes,
I’m reading your book “Data Analysis Using Regression and Multilevel/Hierarchical Models” and trying to understand how multi-level techniques relate to traditional statistical methods for factorial datasets (with reference to Quinn and Keough (2002), “Experimental Design and Data Analysis for Biologists”).
I wonder if you can briefly answer a specific question: The application of simple ANOVA to 2-factor datasets with no replication has traditionally been recommended for analyzing randomized complete block designs, but I understand that this entails no pooling of the groups defined by the “blocks” and is therefore highly unsatisfactory as a means of dealing with blocks that are meant to represent replication. Is this correct? (Section 23.1 on p503-5 suggests this but I don’t think you ever refer to traditional RCB analysis using 2-way ANOVA.)
Furthermore, analyses described as “nested ANOVA” are traditionally recommended for designs where different treatments are applied to whole groups of individuals (e.g. Quinn & Keough; also Mick Crawley in “The R Book” (2007)). I understand from your section 23.3 that this is also inferior to using a multi-level model with treatments as a group-level predictor, but perhaps less serious an error than the use of traditional single-level models where groups represent sampling units (rather than treatments) – as in the use of 2-way ANOVA for RCB designs. Could you confirm this?
My probably-disappointing reply: I think if you treat each row of the Anova table as a multilevel factor you’ll be fine. When a factor has only two or three levels (only 1 or 2 degrees of freedom), the default multilevel model (with flat prior distribution on the group-level standard deviation) won’t do enough partial pooling, so if you really want the right answer you’d want to use an informative prior distribution (possibly through a hierarchical model on the variance parameters, as in the Section 6 of my Bayesian Analysis paper).
Also, here are my short paper and my long paper on Anova and multilevel models.
Presumably with a larger number of groups (randomized blocks), the benefits of a multi-level model compared to a simple 2-factor ANOVA would increase?
Your longer paper on the importance of ANOVA is helpful, thanks. I understand that traditional ANOVAs, even when correctly applied, are still generally inferior to the hierarchical regression approach because they don't shrink coefficient estimates towards the grand mean.
Richard