When I first started working on posterior predictive checking back in 1988, it was as a device for determining equivalent degrees of freedom for a chi-squared test for a model with constrained parameters–in that case, positivity restrictions in an image reconstruction problem; see here for background.

The idea is that the distribution of the test statistic depends on the true parameter vector–there is no simple pivotal quantity as would exist under a linear model with no constraints–but you can work out the distribution conditional on the true parameter vector, and then you can average this distribution over the posterior for the parameters.

You can think of this as Bayesian–the marginal posterior distribution of the test statistic–but at the time I was thinking of it more as a generalization of the existing “plug-in” approach that would use a point estimate of the parameter vector. From that perspective, posterior averaging is a technique for getting a distribution with better frequency properties than you’d get from the maximum likelihood estimate, and that’s because in an image reconstruction problem with positivity constraints and lots and lots of pixels, the maximum likelihood estimate will almost certainly be on the boundary of parameter space. So just about any sort of averaging should get you closer to the true parameter value.

As noted above, I started working in this area in 1988. Around 1991 I got my thoughts organized enough to write a paper and give a talk on the topic, and . . . it got a generally hostile reception! The Bayesians didn’t like it because they didn’t like chi-squared tests or frequentist hypothesis testing more generally. They wanted me to do Bayes factors, which even then I realized were generally a bad idea (for more on the topic, see chapters 6 and 7 of BDA3 (it was all in chapter 6 of the first two editions) or this article from 1995). The non-Bayesians didn’t like it because it was Bayesian, and because I just presented the method, without trying to justify it based on minimax properties or whatever.

Xiao-Li, Hal, and I finally published a version of the article a few years later, and I resigned myself to the fact that posterior predictive checking was going to remain in the Bayesian world. Predictive checking was a hard sell to the Bayesians, but, after a few decades of exposure to BDA and lots of applied examples, they started to get used to the idea. I gave up on trying to promote the idea outside the Bayesian community.

But . . . it turns out that posterior predictive checking has been taken up for non-Bayesian uses! Aki pointed me to this R package called “performance” that “provides posterior predictive check methods for a variety of frequentist models.” Here’s their vignette on checking model assumption – linear models,” and here’s the article, performance: An R Package for Assessment, Comparison and Testing of Statistical Models, by Daniel Lüdecke, Mattan Ben-Shachar, Indrajeet Patil, Philip Waggoner, and Dominique Makowski. It came out in 2021 and has already been cited over 2500 times.

So it looks like people really are using posterior predictive checks for non-Bayesian models. And it took less than 40 years for it to happen!