Anirban Bhattacharya, Debdeep Pati, Natesh Pillai, and David Dunson write:

Penalized regression methods, such as L1 regularization, are routinely used in high-dimensional applications, and there is a rich literature on optimality properties under sparsity assumptions. In the Bayesian paradigm, sparsity is routinely induced through two-component mixture priors having a probability mass at zero, but such priors encounter daunting computational problems in high dimensions. This has motivated an amazing variety of continuous shrinkage priors, which can be expressed as global-local scale mixtures of Gaussians, facilitating computation. In sharp contrast to the corresponding frequentist literature, very little is known about the properties of such priors. Focusing on a broad class of shrinkage priors, we provide precise results on prior and posterior concentration. Interestingly, we demonstrate that most commonly used shrinkage priors, including the Bayesian Lasso, are suboptimal in high-dimensional settings. A new class of Dirichlet Laplace (DL) priors are proposed, which are optimal and lead to efficient posterior computation exploiting results from normalized random measure theory. Finite sample performance of Dirichlet Laplace priors relative to alternatives is assessed in simulations.

I have just a few comments (along with my immediate reaction that the tables in the articles should be replaced by graphs. Really, does anyone care that a certain squared error is “493.03”???).

First, I’m happy to see research on theoretical Bayesian statistics looking at the limit of increasing complexity of the model, what I sometimes call “infill asymptotics” (fixed sample, build more and more model, as opposed to “sprawl asymptotics” where you keep adding more and more data to a fixed model). And I’m happy to see connections to lasso (see here and here for my earlier thoughts on lasso and its popularity).

Also, they talk a lot about how to implement the models using Gibbs sampling, but now that we have Stan, this is much less of an issue. What matters are the models and what they do, and that is indeed most of what’s in the paper at hand.

Basically I think this stuff is great. In the old days I used to be upset when other people made progress on ideas in which I had persistent but unformed thoughts (just as others were perhaps upset when I would publish articles that happened to coincide with their unformed ideas). But now as I get older, I am just happy to see that progress is being made.