This post is from Bob
I’ll let Thomas Wiecki, who is one of the core PyMC devs and one of the partners at PyMC Labs, speak for himself here:
- Thomas Wiecki. 2026. Alchemize: Transpile PyMC to Rust for 3-7x speed-up. PyMC Discourse.
If you haven’t seen what people are doing with agentic AI, this is a good example. I’m really happy that Thomas and PyMC Labs are sharing their thoughts and initial tries at things like this as I think it has the potential to benefit everyone working on modeling.
If you want to see the basis of the agent’s instructions, check out the “skill” for PyMC that Chris Fonnesbeck wrote.
We’ve already batted this around a bit in email with Thomas, so I can summarize some talking points:
LLM-based chatbots are really good at translating. Compiling (or more technically correct, transpiling) a statistical model down to a language like Rust or C++ or JAX is a kind of translation.
You can start from PyMC’s execution trace, but you can also start with a model description. You could also start with something like Stan code.
The biggest bottleneck to deploying Bayesian models in my opinion is the inherent variance and unreliability of MCMC-based inference. Our workflow proposals are all about making sure this doesn’t go wrong. Wiecki’s point here is that we can have the bots go through the workflow. Iterating until the gradients and log densities match is a good example, but this could be extended to more parts of workflow.
The skills feel a lot like writing a textbook for a bot. I have no idea how hard or easy this is or how much it improves over the baseline. Jeremy Magland built a RAG-like helper for Stan that compressed the Stan Reference Manual down to 1K tokens for context (like a skill) and allowed it to search and import from the Stan User’s Guide, but never measured how much it improved over the baseline. It really feels like it should also have the Stan Functions Reference, BDA3, Regression and Other Stories, and the Bayesian Workflow book, as well.
Hopefully we’ll asymptote at writing a textbook sized set of skills and not have to write one per target model (that is, something like the Stan User’s Guide, Reference Manual, Functions Reference).
I’m curious as to whether it will eventually be able to make writing hard models easier. I’m thinking of efforts like epinow2, which involves a very large chunk of Stan code.
As the foundation models and chatbot tuning changes, there’s going to be an issue of regression testing and tuning for whatever the latest models are.
P.S. This effort explains how Thomas was able to create the huge posteriordb pull requests for PyMC (#320 and #319)!
P.P.S. The latest thing Claude (Opus 4.7) did that impressed me was generate the ess(MatrixXd, vector function in summary.hpp. This function estimates effective sample size Stan style (discounting for R-hat > 1) on a ragged array of Markov chains. We have to generalize all the posterior analysis tools to deal with the new asynchronous parallel sampler). I had Stan’s ESS function and all the other functions I’d written for the ragged structures to give it as a guide. It’s very easy to code review that it matches Stan’s implementation for the new data structure. I only had to tweak the output a little bit for style.
https://github.com/JirkaKlimes/jit-implementation
If you think “the inherent variance and unreliability of MCMC-based inference” is a problem, then why on *earth* are you advocating for the use of LLMs in research? The solution to variance and unreliability is HUMAN UNDERSTANDING. You’re cooperating in destroying the very enterprise of science (which is a *human cultural* artifact, NOT an a-priori ‘way of viewing the world’) that you supposedly care about.
I know I can’t convince you that you’re wrong. But you’re wrong.
Llms:
I think if you just had a few more capital letters in your comment, it would be more convincing.
More seriously, I think the plan is not to turn on the LLM and walk away but rather use the LLM to build code that can be run, altered, and checked.
To see what we’re talking about, forget about the LLM for a moment and consider my current coding strategy, which is to cobble together code from different sources. For example, I might look for a Stan program in the Stan Case Studies that is kinda close to what I want, then look up some matrix transformations that I need in the Stan User’s Guide, then google a question which might send me to a two-year-old discussion in the Stan Forums, then I’d do some experimentation on my own. And when I finally have a model that looks reasonable and compiles without error, I’ll simulate some fake data and see if the fit can recover the true parameter values. Maybe then I’d alter the model, do some simulation-based calibration, all sorts of things, and then even after all that, if I’m using it in a real application I’ll check the fit to data and beware of convergence problems and statistical artifacts.
According to Bob, and I have no reason to doubt him, the LLM can assist in many of the above steps. I don’t think this destroys the enterprise of science, any more than the photocopier destroyed the enterprise of science because then you didn’t have to take notes by hand.
Actually, the photocopier is not a bad analogy, because we do indeed lose a lot by switching from rewriting-by-hand to photocopying. I think photocopying has still been worth it, but, yes, there are tradeoffs.
It’s really converting code to code, so not really any understanding of the relevant modeling or statistical problem is sacrificed. The only knowledge that’s lost is an understanding of the low level target programming language, which most practitioners aren’t interested in. Of course, I wouldn’t call this approach reliable; even if it’s verified on some inputs, it’s impossible to verify in general, and LLMs often generate incorrect or wildly nonperformant code. It seems like a really lazy way to build something that’s kind of like JAX, which may or may not work.
I’m not sure what you think I’m wrong about.
I use LLMs in research coupled with human understanding. See the code I linked that it generated. That was an interactive process that resulted in it actually writing the final code, but it wasn’t “vibe coded.” They are also amazing tutors and able to do things like translate measure theoretic notation to undergraduate applied math notation.
I don’t see how using a tool to help me write code is destroying science any more than calculators or spreadsheets destroyed people’s ability to do advanced mathematics. If you go back to Plato, his worry was that writing would undercut people’s memories and abilities to think.
Bob:
Maybe the title of the post should be that the LLM would “use” Stan/PyMC, etc., rather than “replace” them, in that the LLM is making use of all the code that people have written. It makes sense to say that Python has largely replaced Perl and awk, or that Julia has replaced Matlab for many users, but, the way you describe things in your post above, it seems more like use than replacement.
I should have been more specific. It’s getting rid of the Stan/Python back end transpiler. They keep the language and feed their LLM a PyMC program. Next step is getting PyMC out of the loop and just telling it about the model. But this is also an interesting and much more controllable and testable intermediate step.
Preface:
I’m a SWE, and LLMs have nothing short of revolutionized my way of working, so I’m very bullish.
Now, this looks like an interesting experiment and a great testament to how capable these models have gotten. I say experiment because it strikes me as a bit odd to push a non-deterministic agent that far into the inner loop.
In my experience you want them as far out as possible, or out of it entirely once they’ve done their job.
For Alchemize specifically: instead of translating every model to Rust on demand, you could use the LLM to write a generic PyMC to Rust backend once. Better for determinism and for token spend.
(I see Wiecki noted on the discourse that the per model approach was chosen over the rust backend for development speed. Fair point, though I’d be curious whether the months estimate still holds with current-gen LLMs, or whether it’s closer to weeks, if not shorter; especially since plenty of reference draws already exist to compare against which can keep the agent in line.)
And on the “next step” of getting PyMC out of the loop:
my instinct would be to just publish a skill file itself and let people point their favorite agent harness at it (Claude Code, Codex, pi.dev etc.), as those already run an LLM in a loop anyway?
One thing I don’t understand here is whether/how this is superior to bots with internet access (& where the relevant docs are online). Presumably this approach is more efficient or slightly more accurate? Or is it just easier to package and transport?
It’s automatically verified against a reference implementation, just a slower one
I don’t think you need skills to do the coding – coding agents are good at it already!
The best application of skills here is at the level of the _workflow_
I’ve done some experimentation a while back here https://github.com/jg-you/bayesian-workflow
By no mean polished.
Did some testing of fully autonomous agents (opus 4.6 in the Claude Code harness) in 3 conditions: Unaided, Agent + Skill, Agent + Bayesian Workflow paper.
Agent + Skill was the best at recovering a generative model from data alone, though not by a wide margin. They are all great!
I’m an industry DS uses a Bayesian model with MCMC a couple times a year for various projects. Whenever I work on models like this, the biggest overhead is all the boilerplate one has to write to perform inference, make sure the sampling has worked, iterate on the model, perform model checking, etc. The packages to automate these tasks are often not available or difficult to get working on our internal systems, so I feel like I have to re-invent the wheel every time I do it.
I recently used an LLM to build a model in NumPyro and was blown away how easy it was, no specialized skills for PPLs needed. All of the messy overhead was taken care of for me, and the model was able to handle basic debugging and write much of the model checking code on packages that were easily available for me. I certainly can’t trust it to do everything without review, I was involved throughout the process, but the speedup is fantastic.