PPL Workshop this Friday (29 July 2022)

The Computational Abstractions for Probabilistic and Differentiable Programming (CAPP) Workshop is happening this Friday, 29 July 2022. It’s organized out of Cambridge and is running on UK time.

Program and Zoom link

The schedule of speakers is online and the free Zoom link is available under the Location tab of their web site:

Despite the title, the talks sound very practical. The speaker lineup includes developers from BUGS, JAGS, Stan, JAX, and Turing.jl.

My talk

I (Bob) will be talking about our plan to introduce Python-like comprehensions into the Stan language to deal with constant matrix constructions like Gaussian Process covariance matrices. This is motivated by our shift in underlying matrix data structure for autodiff in Stan—what the Stan developers have been calling “mat-var” in meetings and pull requests because it’s an autodiff variable with a matrix value and matrix adjoint rather than our original approach using a matrix of scalar autodiff variables.

P.S. This is a big reduction in memory pressure and time for larger-scale matrix ops for Stan. The really clever part is how the C++ devs (Tadej Ciglarič, Steve Bronder, Rok Češnovar, et al.) have managed to abstract the mat-var construct in a way that’s backward compatible with our existing autodiff with simpler code. If you like C++ and functional template meta programming in C++, I urge you to follow along. Steve’s about to drop all the dev-facing doc for all of this—it’s currently in a massive PR undergoing final revisions.

8 thoughts on “PPL Workshop this Friday (29 July 2022)

  1. Bob:

    This sounds really cool—thanks for sharing. And it reminds me that I want to write that informal spec showing what I’m envisioning, regarding a more compact syntax for some Stan models.

  2. “our plan to introduce Python-like comprehensions into the Stan language”

    Great idea, but I feel compelled to point out that list comprehensions are a feature that Python borrowed (in 2000) from Haskell, which had them since the Haskell 1.0 Report in 1990. Haskell in turn borrowed them from Miranda, which was created around 1985. Seems odd to me to associate them with latecomer Python.

  3. This is motivated by our shift in underlying matrix data structure for autodiff in Stan—what the Stan developers have been calling “mat-var” in meetings and pull requests because it’s an autodiff variable with a matrix value and matrix adjoint rather than our original approach using a matrix of scalar autodiff variables.

    The memory debug functional calls it now Struct-of-Arrays (SoA) which I was told to be more commonly used.

    The new data structure provides 25%-40% reduction of sampling time for several posteriors that I tested. The models included GLMs with many predictors, GPs with basis functions, and GPs with covariance matrices. These all had vector or array parameters with more than tens of elements.

    I’m excited to see further speedups from constant matrix constructions for GPs and GLMMs.

  4. The way the previous post title and the next post title are juxtaposed at the top of this blog entry make the job sound a lot more interesting than it probably is.

Leave a Reply

Your email address will not be published. Required fields are marked *