We welcome public contributions to the Discussion of the manuscript the manuscript Rank-normalization, folding, and localization: An improved R-hat for assessing convergence of MCMC by A. Vehtari, A. Gelman, D. Simpson, B. Carpenter and P. C. Bürkner, which will be featured as a Discussion Paper in the June 2021 issue of the journal. You can find the manuscript in the Advance publication section of the journal website. The contributions should be no more than two pages in length, using the BA latex style and should be submitted to the journal using the Electronic Journal Management System (EJMS) submission page, before May 10th, 2021. An announcement for the public Webinar presentation will follow. As a reminder, all BA Discussion Webinars can be viewed on the ISBA YouTube channel.
I’m very happy with this article. It takes the basic principle from our 1992 paper on R-hat and improves it in various ways. I also recommend you read the followup paper by Ben Lambert and Aki Vehtari on R*, which is a multivariate mixing statistic that uses nonparametric clustering, again, following the basic idea of comparing individual to mixed chains, but in a more general way.
Afterwards I think we should have had included in the title not just R-hat, but also effective sample size (ESS) and Monte Carlo standard error (MCSE) as the paper says a lot about those, too. So you are welcome to join the discussion also on ESS and MCSE! I’m copying here a thread I just tweeted emphasising the other contributions of the paper.
R-hat is mainly multi chain convergence diagnostic. Multi-chain is crucial if you assume that the chains are not mixing well (e.g. due to strong multimodality or weaknesses of algorithm). R-hat can be also used to provide safe multi-chain ESS and MCSE estimates.
There are many versions of R-hat. The mainline Versions 1-4 fail if the distribution has non-finite mean or variance (very thick tailed finite variance cases are also problematic in finite sample case). Version 5 presented in the paper fixes this by using rank-normalization.
In addition of rank-normalized Rhat, the paper presents 1) details of practical and robust multi-chain ESS and MCSE computations, 2) local ESS plots, 3) Bulk- and Tail-ESS as a diagnostic summary, 4) MCMC rank plots, 5) and discusses the limitations of Rhat.
Although the title mentions only Rhat, we emphasize that in the end MCSE for a quantity of interest is what matters. As MCSE needs to be interpreted in the context of that quantity and domain knowledge, Rhat+ESS are useful scale free summaries for multi-chain mixing efficiency