I couldn’t resist quoting Andrew’s statistical lexicon in my abstract! It’d be great to see you in person if you’re nearby, but otherwise, the talks will go out via MS Teams (links below).
Prior and Posterior Probability Checks in Stan or How to Write a Stan Model to Simulate Data and Why You Should
Mitzi Morris, Columbia University, Statistics Department
A Stan model infers the model parameters given the data. The data-generating program reverses the computation: given parameters, it outputs the data according to the specified distributions. Often simulated datasets are created by one-off scripts in R or Python. Writing a data-generating program using Stan allows for more systematic exploration of the consequences of choice of prior and difficult data regimes.
11am, Tuesday, 4 June in Room 3A at DEAMS, University of Trieste, and available also from MS TEAMS at the following link: MS Teams link for talk
GIST: Gibbs self-tuning for locally adaptive Hamiltonian Monte Carlo
Bob Carpenter, Flatiron Institute, Center for Computational Mathematics
I will present a novel and flexible framework for localized tuning of Metropolis samplers, including Hamiltonian Monte Carlo (HMC). In the Gibbs self tuning (GIST) framework, an algorithm’s tuning parameters are Gibbs sampled each iteration conditioned on the current position and momentum. For adaptively sampling path lengths, I will show that randomized integration time Hamiltonian Monte Carlo, the no-U-turn sampler, and the apogee-to-apogee path sampler all fit within this unified framework as special cases. I’ll provide two examples. One is a multinomial form of randomized bidirectional HMC with a 100% acceptance rate. The second is a much simpler alternative to the no-U-turn sampler for locally adapting path lengths. In all of these samplers, correctness depends on simulating the Hamiltonian dynamics forward and backward randomly in time. The key to making local tuning practical is randomization (aka uncertainty), which as Andrew Gelman likes to say, “greases the wheels of commerce.” I will conclude with a discussion of the opportunity this framework presents for adapting HMC’s step size and mass matrix.
Joint work with Nawaf Bou-Rabee (Rutgers, Camden), Milo Marsden (Stanford), Edward Roualdes (Cal State, Chico), Chirag Modi (Flatiron Institute), and Gilad Turok (Flatiron Institute)
4pm, Monday, 3 June in room 1A (‘Aula Conferenze Bruno de Finetti ‘), at Department of Business, Economics, Mathematics, and Statistics (DEAMS), University of Trieste, Via Valerio 4/1, 34127, Trieste. Available via MS Teams at the following link: MS Teams link for talk
This is great, thanks for letting us know Bob. Looking forward to attending both your talk and Mitzi’s.
From a practical point of view, are any of these samplers better than latest NUTS (as in Stan) at this stage?
Thanks for asking. The centrality of this question is why we’ve moved to evaluating against NUTS rather than a simple HMC baseline.
So far, the delayed rejection models we’ve been working on will get the right answer in multiscale distributions like the funnel where NUTS or vanilla HMC fail due to a fixed step size. And by fail, I mean produced biased estimates after any practical amount of compute. Our (with Chirag Modi and Gilad Turok) NeurIPS submission this year extends delayed rejection HMC to delayed rejection GHMC, which is largely competitive with NUTS across a range of problems. We’ll be putting it out on arXiv soon.
But those aren’t what I’m going to be talking about. GIST is much simpler than all of this conceptually. GIST includes NUTS and the apogee-to-apogee sampler as special cases. So it’s more of a framework than a particular sampler. I did use it to generate a novel U-turn based sampler. It’s competitive with NUTS, but much much simpler and easier to understand theoretically. By competitive, I mean there are some problems where GIST is more efficient and some where NUTS is, but in almost all cases we tried from posteriordb, the variance among runs for a given sampling algorithm is an order of magnitude larger than the difference in means between different algorithms. So you wouldn’t be able to tell in practice without a huge number of evaluations.
What we’re really optimistic about is that GIST opens up a general tool for adding tuning. We’re working on adapting step size the same way as NUTS adapts steps, so that we get a near 100% acceptance rate (unlike with our delayed rejection systems).
Hello, Thank you! The talks seem very interesting. Is this Trieste time or New-York time?
It’s Trieste time. Here’s the link to Bob’s talk: https://deams.units.it/it/eventi/279974
I can’t find a link to Mitzi’s.
Yes, all Trieste time. We’ll be there in person. Italy’s one of my favorite places to visit.
The MS Teams links for the talks are included for both talks along with details of where they’re at in Trieste.