This post is from Bob.
I’ve been thinking a lot about scores (gradients of the log density function) and how they can be used for convergence monitoring. We know that the expected value of the score is zero. Stein generalized this with Stein operators. In the monomial case, the Stein operators give you functions in increasing degrees, all of which have zero expectation in the posterior. Here theta is the variable being sampled and S is the score function, so that S(theta) is the gradient of the target log density evaluated at theta.
Order 0: S(theta)
Order 1: 1 + theta .* S(theta)
Order 2: 2 * theta + theta^2 .* S(theta)
This leads to a natural test for convergence of first, second, and third moments. Just compute Monte Carlo estimates of these quantities and see if they’re zero. We’d want to standardize for standard deviation to make the result scale-free like R-hat. To develop some intuitions, in a standard normal distribution p(theta) = normal(theta | 0, I), we have S(theta) = -theta, and thus S(theta) converges to zero at the same rate as our variable theta converges to its true value; the order 1 test is 1 – theta^2, which we know has expectation zero because theta^2 has a ChiSquared(1) distribution with expectation of 1). The order 1 case corresponds to equipartition in physics and the form D + theta’ * S(theta) also naturally has zero expectation as shown in the viral theorem in physics in the 1870s.
Diving into this a bit more led me back to Jackson Gorham and Lester Mackey’s work on Stein’s method. They haven’t been sitting still since introducing the basic idea, which kernelizes the idea above. Mackey et al. have produced an absolutely wonderful summary of this body of work in two forms. The first is a dense, 41-slide deck with all the key definitions and results. I’d suggest at least skimming this first.
Lester Mackey. April 2026. Stein’s Method, Learning, and Inference.. GitHub.
Mackey along with Chris Oates and Qiang Liu, who have also worked heavily in this area, put together a definitive monograph. They’ve presented a great deal of difficult material in a way that I can digest (though it’s going to be rough going if you’re not well versed in sampling and how MCMC is traditionally measured and evaluated).
Qiang Liu, Lester Mackey, Chris Oates. March 2026. Probabilistic Inference and Learning with Stein’s Method. arXiv.
In particular, they go over Stein variational inference, which seems to me like it would be the ideal way to perform quasi Monte Carlo-like inference for statistical models if we could only get a robust version to scale. The idea’s to initialize a bunch of points, then use optimization to minimize a kernelized Stein discrepancy of the empirical distribution of those points to the true distribution.
Bob:
You were telling me about this . . . it looks like the method presented in Section 2 of our 2006 paper, Output Assessment for Monte Carlo Simulations via the Score Statistic?
I vaguely remember that when we wrote this paper, that it didn’t seem so immediately useful. This might have been because we didn’t have autodiff so the computation would’ve taken more programming effort. Also, though, I don’t know that the method was detecting problems in settings where traditional R-hat failed. I don’t recall the details, though.
If you look up that paper of ours, it only has 31 citations, so I don’t think it’s caught on as an applied tool. I guess it could be that the method can be very effective and it just hasn’t been automatically implemented. I guess we could start by throwing it at the examples in posteriordb, no?
Yes, this is what I was telling you about. And I also reminded you that you’d published a paper on it. I asked you if you didn’t follow it up at the time because gradients were too hard to think about in general. Beyond first order, you use something that has a Chi-squared distribution and then use Chi-squared test statistics.
There are a couple of things that are nice about the simple Stein operators based on monomials that I list above. Unlike R-hat, they don’t need a multi-chain structure. You can use them to measure SMC, importance sampling, diffusions or normalizing flows, whatever. This means they can work on many short chains liked nested R-hat without requiring nesting. It should be able to diagnose convergence with many chains faster than R-hat because of the result that R-hat is roughly sqrt(1 + num_chains / total_ESS). You see that in the experiments I ran where R-hat converges to 1 at the same rate no matter how many chains are used, though obviously we’re reducing error a lot faster with 64 chains than with 4. Also, they should be more stable because they’re measuring against a fixed target.
Gorham and Mackey then take it much further. They introduce kernels, which turn out to be powerful enough that they get necessary and sufficient conditions for convergence! As you know, R-hat converging to 1 is necessary but not sufficient. If R-hat >> 1, you know you haven’t converged, but we can get R-hat < 1.01 in cases where you're not exploring the target. The simple Stein operator targets I laid out have that same property as R-hat. For instance, E[S(theta)] will still tend to zero for draws from normal(0, 1), even if the target density is normal(0, 10) due to symmetry. R-hat also has this problem---it doesn't even use the target density, so anything that looks like a bunch of sample from the same distribution will cause R-hat converging to 1. We see that in practice with false positive convergence diagnostics (e.g., R-hat < 1.01) when exploring only part of a distribution (e.g., due to divergences in areas of high curvature).
I mentioned this to Reuben Cohn-Gordon and he pointed out that Jakob Robnik and Uroš Seljak used the order 1 test statistic to diagnose when to switch from unadjusted to adjusted sampling in a recent paper. In more than one dimension, the function 1 + theta * S(theta) having expectation equal to a zero vector is what physicists call the “generalized equipartition theorem,” whereas D + theta.transpose() * S(Theta) having zero expectation is what they call the “virial theorem,” both from 100+ years ago. Stein’s method gives you a way of creating this (apply the Stein operator to a first-order monomial) and many more.
Edit: Here’s the link to the Robnik and Seljak paper, Faster parallel MCMC: Metropolis adjustment is best served warm, in which they argue for unadjusted methods during warmup.
Bob:
As I recall, one difficulty with the score statistic was that we could only easily evaluate it for parameters in the model, not for derived quantities such as the log posterior. In any case, I’m interested to see how it works for the examples in posteriordb.
Good point—I hadn’t listed that in the pros and cons column yet. If I’m understanding Gorham and Mackey correctly, it doesn’t have this problem. It uses one kernel evaluation as a standing for bounding error across a wide class of expectations.
The other thing is that the score statistic gives us an expectation but not a sd, so it seems that we would still need a within-between comparison to know how close to zero it should be.