Terence Teo writes:

I was wondering if multilevel models can be used as an alternative to 2SLS or IV models to deal with (i) endogeneity and (ii) selection problems.

More concretely, I am trying to assess the impact of investment treaties on foreign investment. Aside from the fact that foreign investment is correlated over time, it may be the case that countries that already receive sufficient amounts of foreign investment need not sign treaties, and countries that sign treaties are those that need foreign investment in the first place. Countries thus “select” into treatment; treaty signing is non-random. As such, I argue that to properly estimate the impact of treaties on investment, we must model the determinants of treaty signing.

I [Teo] am currently modeling this as two separate models: (1) regress predictors on likelihood of treaty signing, (2) regress treaty (with interactions, etc) on investment (I’ve thought of using propensity score matching for this part of the model). However, this doesn’t really get at the problem.

Here is the (non-nested) multilevel model I have in mind:

investment_{i,t} = investment_{i,t-1} + treaty_{j,i,t-1} + X_{i,t-1} + error_{i,t-1}

treaty_{j,t} = political institutions_{j, t-1} + X_{j,t-1} + error_{j,t-1}

Can I do this? Would a Bayesian framework along the lines of chapter 16 in your book work?

My reply: first off, Jennifer’s the causal expert, not me, so my response will be kinda vague. Second, don’t you need some sort of instrument for the treaty signing? Or is that what you’re trying to get around?

Beyond this, if you are working in a more traditional instrumental variables framework, I suspect that multilevel models will be useful for all the usual reasons.

Finally, if you’re stuck, I recommend trying to set up a plausible theoretical model, including selection bias based on a latent variable that is not measured in reality but you could define in theory, then simulating everything from this model including fake data, then fitting your model (but not using the latent data which are assumed unobserved), and see how your method works.

I hear you describing a causal loop: investment treaties can (might) cause foreign investment, and foreign investment might cause (fewer) investment treaties. That’s the domain of system dynamics: feedback control theory (servomechanisms) applied to a broad range of systems, not just electromechanical ones. John Sterman’s Business Dynamics is one good, not-so-short text in the field.

Classical control theory methods might apply, except that most non-engineered feedback systems have strongly nonlinear components. For estimating models, though, you might look at MCSim. It does exactly that sort of modeling, and it can do the hierarchical MCMC modeling. The MCSim Web site has a short reference card that shows SD practitioners how to use MCSim. Feel free to ignore the use of J; you can do the same post-processing in R, if you want.

MCSim comes with some compartmental PBPK models, which are really the same as system dynamics (SD) model but applied to physiological systems.

It sounds very similar to a Heckman Selection model, with an equation modeling the selection into treaties, and an equation modeling the effect of treaties (or the outcome equation).

This is an imperfect substitute for 2sls because unbiased estimation requires that one of two conditions be met:

1. Perfect prediction in the selection equation (i.e. we have to be able to model with perfect accuracy selection into the treaties)

2. The variance in selection that is left unexplained must be exogenous to outcomes, which strikes me as relatively unlikely

Heckman models have been replaced by IV in the economics literature for basically this reason.

I may be missing something in the initial post, but it’s not clear to me how a multilevel model would get around this fundamental inferential problem. If it were me, I would go the matching route, in addition to other options you might try.