In 2019, Andrew blogged about collinearity in Bayesian models. In the comments, he pointed to an example from Bayesian Data Analysis, 2nd edition (BDA2). I think it is a useful example to keep in mind when extrapolating from sample to population. Since folks (like me) may only have BDA3 on their shelf, I thought I’d talk thru it.
Pretend it is 1980 and we are at the US Census Bureau. We just revamped the occupational coding system, and it’s so much better ! We want 1980-style codes on all our old data that only had 1970-style codes. Let’s trade in our peasant blouses for some shoulder pads.
Say we have double-coded training data (n = 10,000) with:
- O_1980 = occupation coded in the 1980 coding system
- O_1970 = occupation coded in the 1970 coding system
- E = education, either high or low
- I = income, either high or low
We want to impute O_1980 for the single-coded full dataset (N = 1,000,000) with only O_1970, E, and I.
Consider everyone with the a specific occupation according to the 1970 codes, e.g. Accountants. Say there are 200 accountants in the double-coded training data and they have either high income and high education or low income and low education. They have either OCCUP1 or OCCUP2 according to the 1980 codes.
From BDA2 Table 9.1:
Say we use standard regression software to fit p(O_1980 | O_1970 = Accountants, E, I). It will flag the predictors E and I as perfectly collinear, because in the double-coded training sample, education and income are perfectly correlated.
Suppose you drop education and use only income. The single-coded data actually has some low education and high income folks. The model only uses income, so 90% of them get OCCUP1. But suppose I drop income and use only education. My model only uses education, so only 10% of them get OCCUP1. Who is correct ?
As the authors say:
the truth is that we have essentially no evidence on the split for these units… the occupational split for the ‘E=low, I=high’ units should vary between, say, 90/10 and 10/90. … If some variable should or could be in the model on substantive grounds, then it should be included even if it is not ‘statistically significant’ and even if there is no information in the data to estimate it using traditional methods.
So the low-income, high-education (and vice-versa) accountants should get coded as 50/50?
Great question, Proxomitron !
The authors of BDA2 say:
I think they’re making an implicit assumption that income and education point in the same direction in terms of their effects on probability of OCCUP1. Otherwise, it is theoretically possible for the split in the E=low, I=high to be anything from 100/0 to 0/100, since we’ve not seen any of these folks and who knows what they’re like.
You are correct that the middle of these ranges is 50/50, but the authors are wanting to consider the full uncertainty range in downstream analyses, e.g. with multiple imputation. In a later chapter of BDA they write:
“We want 1980-style codes on all our old data that only had 1970-style codes. Let’s trade in our peasant blouses for some shoulder pads.”
Shoulder pads: perfect linearity concerning the shoulders but confounding concerning the silhouette.
Thank you for the shoulder pads & statistics humor, AAAnonymous ! I am the audience for it.
I know almost nothing about statistics so my attempt at a joke might be poorly executed because I kind of don’t know what most terms exactly mean. I think I used the “linearity” and “confounding” words appropriately enough, at least in the more general interpretation of these words. I also know little about shoulder pads, but I looked it up and saw the word “silhouette” written somewhere and then tried to use the title from your post as a format to rearrange a bit to fit with the shoulder pad stuff.
I was born in 1977 and consider the 80’s to be my childhood time period, and I do remember the clothing style, mostly because I connect it to music videos. I like music a lot, and I think this might also be why I have such a strong connection to the 80’s. I bought a second hand radio-cassette player when I was about 6 or 7 years old and music has been a big part of my life from that moment on. I guess that’s why your shoulder pads and 80’s sentence stood out for me.
One of the songs I remember from that time is Janet Jackson’s “What have you done for me lately”. I’m listening and watching the video as I write this, and of course shoulder pads are present in the video! Another song by Janet Jackon from that time is “Miss you much” which has an impressive chair dance part at the end in some video versions which might be cool to check out.
So pleased to bounce from Census to 1980s music videos ! Thanks again, AAAnonymous.