Here is the draft version on the dining room table.

https://github.com/bioinfonm/pystan_musings/blob/master/beginners_excercise/network_of_models.jpg

Yup really infinite and actually a continuum but finite approaches are likely all that’s need ;-)

]]>Andrew’s thinking of a network in the graphical sense, where models are nodes and edges connect “adjacent” models. As an example the other day, Andrew mentioned that if you’re doing a compartment ODE with 3 compartments, you have natural 2 compartment and 4 compartment models that are adjacent. If you have ten predictors and you’re using 5 of them, there are 5 you can drop and 5 you can add at that point, to give you ten adjacent models. Oh, and you can have interactions. So there’s really infinitely many possible sets of predictors you can produce and that’s just considering polynomials.

This was the first thing Matt Hoffman and I got set to work on when we started working with Andrew in 2010/11. We pretty quickly decided it would be impractical to explore automatically given the difficulty of model comparison and the combinatorial explosion of potential models. I don’t see how it could help in a practical setting.

What Andrew dreams of is that some kind of IDE would solve all the fussy model naming and exploration. We tend to make moves like this in model space, so it seems like it’d be nice to have tools to support it. Something that’d munge all the data, give us a menu of predictors and outcomes with convenient discovery of types (categorical, ordinal, vector/scalar, etc.). None of us like to have to have a series of models like logistic.stan, hierarchical-logistic.stan, hierarchical-logistic-correlated-prior.stan, ad infinitum.

I’m not sure about the granularity of model in Andrew’s picture of this. If we have constant parameters for priors in our model, then there’s a neighborhood around the parameters of those models that brings us into something much more point-set topological than graphical. Then we can start looking at things like sensitivity of inference w.r.t. changes in these constant parameters.

]]>