Here’s a cool thread from Bob Carpenter on the Stan forums, “A new continuation-based autodiff by refactoring,” on the inner workings of Stan. Enjoy.
P.S. Just for laffs, see this post from 2010 which is the earliest mention of autodiff I could find on the blog.
Good to see that Greenspun’s Tenth Law is still relevant ;-)
Although Greenspun is often referred to in a snarky way, this was not intended to be snark more like a constant of the universe like death, taxes, the second law of thermodynamics, and the inevitable reinvention of ideas from Common Lisp in other contexts.