Software release strategies

Scheduled release strategy

Stan’s moved to a scheduled release strategy where we’ll simply release whatever we have every three months. The Stan 2.20 release just went out last week. So you can expect Stan 2.21 in three months. Our core releases include the math library, the language compiler, and CmdStan.

That requires us to keep these repositories in synch and working with each other. We’ve had this always-ready-to-release discipline in place since roughly Stan 2.0, when we started picking up many more developrs and working on many issues in parallel. Letting the continous integration builds fail to be deliverable is a recipe for disaster when there are two dozen programmers working on a code base.

What we used to do is decide to do a release when we had an important bug fix or new feature. Inevitably, Daniel and I would have a meeting to do triage and then add a few just-one-more-thing issues to the release. It would take months to do a release just to get all the last-minute fixes in. Now we just release on the scheduled date. We figure that’ll get features to users faster net. It’s also good for developers to see their recent commits released when they can still remember what they did.

Down to the wire for 2.20

For this release (2.20), I rebuilt the base C++ classes for our compiled models and Mitzi updated the way Stan and CmdStan were built and tested so that we could separately compile a model and all the service code.

The good news is that compiling a Stan model in CmdStan on my 2012 notebook is now just under 7 seconds. It used to be close to 30 seconds. This really tested our resolve for this release, as the final tweaks to the makefiles happened on the morning of the release. Mitzi, who was building things got up early and Sebastian, who was reviewing the build pull request, stayed up late in Switzerland to get it in. Otherwise, we were willing to put things off three months just to stick to our process.

So thanks for all the hard work. Hopefully, these compile times will percolate through to RStan and PyStan (the other interfaces get the speedup for free by depending on CmdStan).

7 thoughts on “Software release strategies

  1. Bob:

    Compilation in 7 seconds—wow! Once this gets into rstan, this should help brms to become more of a workhorse for applied statistical modeling.

    The continuing progress of Stan shows the benefit of open source + lots of begging for money from government, foundations, and businesses. Money + distributed cooperation + open-source + real applications: that’s a great combination.

  2. BTW Bob, can you clarify exactly how the release strategy works? Is it that on the date in question, the latest version that passes all its tests is released automatically? Or does someone actually select a particular version, test it, and tag it as release? or maybe the last version on a particular branch that passes its tests, or what?

    I’m particularly imagining that someone might add some features, not have a thorough test case for the features yet, and then a release occurs because this version passes its tests, but it’s actually not all that high quality because adequacy of the tests hasn’t been established.

    I imagine you guys have thought of this, so I just wondered what strategy you had to handle it?

  3. > Is it that on the date in question, the latest version that passes all its tests is released automatically?

    Yup. There’s a cutoff time and all the PRs that have passed tests and code review get released.

    > I’m particularly imagining that someone might add some features, not have a thorough test case for the features yet, and then a release occurs because this version passes its tests, but it’s actually not all that high quality because adequacy of the tests hasn’t been established.

    That’s what code review is for. Making sure the tests are sufficient is the biggest part. But it’s software, so mistakes happen. The combinatorics defeat us all in the end.

  4. Cool! Great job to the whole crew. I use brms frequently for work, so this update will be awesome. It’s amazing all of these tools are open source. They have really transformed the way I analyze data.

    I wish the entire Stan, rstarnarm, brms team (I know Paul Burkner is now part of the Stan team) could get some sort of applied statistics award. I did not have any real exposure to Bayesian statistics in my applied statistics education, but over the last few years, through McElreath lectures, brms and rstanarm vignettes, this blog, BDA3, etc., I have taught myself Bayesian modeling. I now use Bayesian models exclusively for my work. I actually do not think that would have been possible without the very accessible and easy to use packages like brms, which of course depend on Stan.

    So thanks!

Leave a Reply to Wayne Cancel reply

Your email address will not be published. Required fields are marked *