Chris Weiss writes with a question about propensity score matching with multilevel data:
Part of our proposed analysis looks at effect of characteristics of urban n’hoods on bmi [body mass index]. Huge issue with the data, of course, is selection. So in thinking through possible fixes and responses, psm [propensity score matching] has come up – but does psm work with multilevel data like this? I’m having trouble thinking through the algorithm. I can see, technically, estimating a multilevel regression and generating a score from that – but have never seen it done. Alternatively, I could see doing the psm matching within units – so that you’d only make matches within tract/zip/etc. What’s your thought about how to execute this? I just looked at you all’s book and it seems like there’s not a good solution – but thought I’d ask.
My quick answer: Let’s assume you’re going to use matching (whether propensity score or otherwise). Then, yes, you can match on groups as well as individual characteristics. One approach is to strictly match within group (tract, or whatever) and then match on individual-level characteristics within groups. Another way is to match on group-level and individual-level characteristics and not worry about the groups themselves (except inasmuch as they differ on measured characteristics). Yet another way (the “partial pooling” way) is to fit a multilevel model and match on propensity scores; to the extent there is unexplained group-level variance, such a method will favor matching units in similar groups.
Aren't we getting ahead of ourselves? Without knowing the specifics of the research, I've often wondered how appropriate it is to apply matching in contextual studies. To steal from Rubin, the first question is: what's the treatment? The second question is: what's the assignment mechanism? It's not at all clear that modeling context as a treatment lends itself to propensity score matching, or that it lets you get around the "reflection" problems described in Manski (1993). Are you modeling whether people choose to be born in a neighborhood? Move in? More information would be helpful.
Let's assume we all understand the issues with matching (I don't, but let's pretend) and propensity scores (nor here, I, but again, not on-topic). What differentiates a multilevel model from a stock glm? Not all that much in this context, it seems to me. Perhaps I'm missing something. What?
Andrew, I asked this of isabel lugo over at god plays dice, but she didn't feel like answering, my stupid question, so i thought maybe you could be of some assistance?:
so I was thinking about the stock market today, and what the expected return for the market is. As you know, thanks to William sharpe, all financial theory is based on the expected return of equities…. except no one can agree on what to "expect". Most of the time its the arithmetic mean of past returns, but sometimes people use the geometric mean, still others the arithmetic mean plus dividend yield. No one seems to use the harmonic mean, that i've encountered.
So my question is more general. What can the difference between the arithmetic mean and the geometric mean (and harmonic) tell us about the dispersion of the data we are looking at?
also, the harmonic mean seems to relate to the exponential function,as does the normal distribution, which brings me back to my question about what difference between the means tells us about dispersion.
Matt: You are raising important issues; I was just answering Chris's technical question, which is something that will remain after these conceptual points have been resolved.
Wcw: It's a little thing called partial pooling; see my book with Jennifer for more.
S.K.: The geometric mean is just the arithmetic mean on the log scale. That's all I can say; I know nothing about finance.
I'm very sorry to be a pest…. but…
I'm having difficulty understanding what the harmonic mean is. (not its application). In finance the geometric mean is commonly used, and is called the compound annual growth rate and is intuitive. But, the harmonic mean? the arithmetic mean of the reciprocals of the set? What is this in common language? The rate at which the growth rate and the original quantity both increase at?
Also is there no meaning to the degree of difference between these means? surely they must tell us something?
Chris Weiss et al. might wanna have a look at Markus Gangl (2006)'s article in ASR. He is estimating the propensity score using a multilevel model. I think he is doing a nice job in discussing his approach. The online-supplement could be of interest as well.
I just need statistics advise. Specifically on structural equation modelling. I have a sample size of 260 and about 20 variables. My individual basic 2 variable regressions and ANOVAs are all significant and in the correct direction. For some reason though I added the data from a pilot sample that had a lot stronger relationships for all basic analyses. Once this data was added, the SEM model became weaker. My knowledge of SEM is limited and I can not understand why the SEM model can become weaker when each individual variablecorrellates more strongly with the others in the model. I think it may be because one of the latent variable is negatively related to all of the other latent variables…