So far we’ve focused on a population mean E[Y] as our quantity of interest. We’ve talked about estimating this with weights or MRP.

• Inverse-response-Probability Weights (IPW): 1/E[R | X]
  Or normalized as importance sampling weights W = E[R]/E[R|X] = p(X)/p(X|R=1).
  E[Y] = E[YW | R = 1] is how we identify the quantity of interest.
• (Multilevel) Regression and Poststratification ((M)RP):
  E[Y] = E[E[Y | X, R = 1]] is how we identify the quantity of interest.

These are equivalent if we use fully saturated models to estimate E[R|X] and E[Y|X,R=1], see Lumley 2010 Section 9.2.2 and Hernán and Robins (2020) Technical Point 2.3.

But now what if we care about a subgroup like voters E[Y | V=1] ? Suppose the election hasn’t happened yet, so we don’t have V (whether you are a voter) in the population. And V in the survey suffers from “vote overreport bias” as Ghitza and Gelman 2013 and Ghitza and Gelman 2020 note. The New York Times estimates E[V | X] to use as part of the weights:

Other polls, including The Times/Siena Poll, use what’s known as a probabilistic model. Rather than eliminating low-likelihood voters from the sample entirely, pollsters combine the available data to estimate how likely each respondent is to vote, and their responses are weighted accordingly.

Since p(X) is known, we can use Bayes to get p(X | V = 1), as done on p.157 of Gelman 2007. And on p.764 of Ghitza and Gelman 2013 they mention estimating the cell sizes N_j or cell probabilities p(X=j | V =1) needed for MRP:

We might be averaging over the voting-age population, or the voting-eligible population, or the population of voters…When the N_j’s are known…poststratification is easy. When the N_j ’s are merely estimated, we continue to apply (1), this time plugging in estimates of the N_j ’s obtained from some preliminary analysis.

So the above methods now look like:

• Weights: 
  Now importance sampling weights are W = E[V|X]/E[V] / E[R|X]/E[R] = p(X|V=1)/p(X|R=1).
• Multilevel Regression and Poststratification (MRP):
  E[Y | V=1] = E[E[Y | X, R = 1, V = 1] | V = 1] now requires p(X | V=1) for the outer expectation.
  We have at least 3 ways to use something like the NYT likely voter model E[V | X]:
  1. use Bayes to estimate cell-level probabilities p(X=j | V=1) as in Gelman 2007 and Ghitza and Gelman 2013
  2. make turnout weights T = E[V|X]/E[V], the numerator of the importance weights above. Then score our MR model on everyone but weighted:
     E[E[Y | X, R = 1, V = 1] * T]
  3. Talking to Andrew at Playroom today he suggested that turnout weights can also be added to the “X” in the MRP.

This taxonomy sweeps a lot under the rug ! It was made to show how turnout weights relate to inverse-response-probability weights. If likely voters are also more likely to respond to surveys, nonresponse bias can be seen as extra turnout weighting.