Skip to content

Survey weights and poststratification

A question came in which relates to an important general problem in survey weighting. Connie Wang from Novartis pharmaceutical in Whippany, New Jersey, writes,

I read your article “Struggles with survey weighting and regression modeling” and I have a question on sampling weights.

My question is on a stratified sample design to study revenues of different companies. We originally stratified the population (about 1000 companies) into 11 strata based on 9 attributes (large/small Traffic Volume, large/small Size of the Company, with/without Host/Remote Facilities, etc.) which could impact revenues, and created a sampling plan. In this sampling plan, samples were selected from within each stratum in PPS method (probabilities proportionate to size), and we computed the sampling weights (inverse of probability of selection) for all samples in all strata. In this sampling plan, sampling weights for different samples in the same stratum may not be the same since samples were drawn from within each stratum not in SRS (simple random sample) but in PPS/census.

After all samples were drawn, we want to estimate the revenue of large/medium/small size of companies (the above sampling plan was created for other purpose) respectively. We poststratified all samples based on size of the company only. Obviously, this re-classification was totally different from the original stratifying based on 9 attributes. Let’s assume that there are x samples falling into large company group under this re-reclassification. Obviously, these x samples could be from any original stratum or several original strata since this poststratification was regardless of the original stratifying. e.g., some samples in original stratum h could fall into large company group under this re-classification and other samples in original stratum h could fall into medium company group under this re-classification. Now my question is: are the original stratified sampling weights valid for the large company group from poststratifying! ? e.g., can we multiply the revenue of each sample by its original stratified sampling weight for all x samples in the large company group poststratified and add up all products to get the total revenue of the large company group (subtotal)?

I guess the sample weights are not valid for the post-stratified large company group, noting that the check rule on sample weights: add up all sample weights in a stratum (poststratified) and see if the sum is equal to the stratum size (subpopulation). So, we cannot multiply the revenue of each sampling unit by its original sampling weight to get the total revenue.

My thinking is that the original sampling weights need to be adjusted (rescaled) to get new sampling weights for the post-stratified new strata, and then the revenue of each sampling unit can be multiplied by its new sampling weight to get the total revenue. Another reason to adjust for sampling weights is non-responses.

My reply:

1. You use PPS sampling. Usually PPS is used in cluster sampling, where the first stage is PPS and the second stage samples a fixed number of units per cluster, so that the unit-level sampling is equal probability of selection. (See, e.g., these references for more on these issues.) So if you’re just doing PPS, I don’t know that you’ll need unit-level weights (or, if you do, they shouldn’t vary much if the survey was designed well.)

2. The full solution to your problem is to poststratify on the 2-way table of the 11 design strata crossed with the 3 size categories–that’s 33 post-strata in all. If you have enough data, you can simply do full poststratification–that is, get a separate estimate for each of the 33 cells, and then add up the rows of the table to get estimates for each of the 3 size categories.

3. Another option, which might work better if the sample size is smaller, is to rake: poststratify on your 11 strata, then on your 3 size categories, and iterate this a couple times so that your weighted sample matches the population in both these dimensions.

You certainly shouldn’t need to take a new sample – unless the original sample has too small a sample size in one or more of your post-strata of interest. If you need to take a new sample, you might want to consider using a survey sample calculator to help determine the minimum sample size.

P.S. Connie sent me a new version of her question so I altered her wording above as requested.

One Comment

  1. George Joseph says:

    I have a sampling weight related problem and I would like some response/ suggestions on that.The basic structure of the household survey is that we have several clusters and within clusters there are households.

    I am working on a household dataset in which a particular section is about migratnts. The
    I have a dataset of individuals who have migrated to different destination countries( which includes their age, education, gender, marital status,destination countries , amount of remittances sent home etc). But this information is given by households at home when they are asked about migrats from the household . But when we look at the census data in the destination countries, we find significantly more migrants than what we find from the household data. One reason for this is that migrants, especially the educated ones have taken their entire famiies with them and hence have no households at home to report their information. This underreporting of migrants from home causes significnat problems in analysis .

    Therefore, only a fraction of the migrants who appear in the national census of the destination countries are captured in the household survey at home

    I have weights for the houehold data and quite a bit of demographic and eduction information from the census data

    now, my problem is the following. Can I use the census information in the destination countries to reweight the household data in the home country.

    Is this possible in the first place? This is like using external information to reweight the houshold weights, I think. Could you suggest me some references.

    Any help, thoughts, suggestion and recommendations are deeply appreciated.