Here is the draft version on the dining room table.

https://github.com/bioinfonm/pystan_musings/blob/master/beginners_excercise/network_of_models.jpg

This was the first thing Matt Hoffman and I got set to work on when we started working with Andrew in 2010/11. We pretty quickly decided it would be impractical to explore automatically given the difficulty of model comparison and the combinatorial explosion of potential models. I don’t see how it could help in a practical setting.

What Andrew dreams of is that some kind of IDE would solve all the fussy model naming and exploration. We tend to make moves like this in model space, so it seems like it’d be nice to have tools to support it. Something that’d munge all the data, give us a menu of predictors and outcomes with convenient discovery of types (categorical, ordinal, vector/scalar, etc.). None of us like to have to have a series of models like logistic.stan, hierarchical-logistic.stan, hierarchical-logistic-correlated-prior.stan, ad infinitum.

I’m not sure about the granularity of model in Andrew’s picture of this. If we have constant parameters for priors in our model, then there’s a neighborhood around the parameters of those models that brings us into something much more point-set topological than graphical. Then we can start looking at things like sensitivity of inference w.r.t. changes in these constant parameters.

]]>