In response to my remark in this post, “So, what to do? I don’t know, exactly! There’s no Platonically correct prior for (mu, tau). All I can say for sure is that the prior we are currently using is too wide,” Erik van Zwet writes:
The distribution of tau across all the meta-analyses in Cochrane with a binary outcome has been estimated by Turner et al..
They estimated the distribution of log(tau^2) as normal with mean -2.56 and standard deviation 1.74.
I’ve estimated the distribution of mu across Cochrane as a generalized t-distribution with mean=0, scale=0.52 and 3.4 degrees of freedom.
These estimated priors usually don’t make a very big difference compared to flat priors. That’s just because the signal-to-noise ratio of most meta-analyses is reasonably good. For most meta-analyses, finding an honest set of reliable studies seems to be a much bigger problem than sampling error.
> For most meta-analyses, finding an honest set of reliable studies seems to be a much bigger problem than sampling error.
Much bigger – recall the editorial from the former BMJ editor. A percent of those studies in that Cochrane analysis will have been low quality or even made up.
So recalling Rubin effect size estimate view of meta-analysis – that prior will widen the pooled confidence interval somewhat when what is need is a large shift in it. Further discussion here on quality versus biological driven heterogeneity https://statmodeling.stat.columbia.edu/2017/11/01/missed-fixed-effects-plural/.
Now, Sander would likely suggest informative bias priors if possible.