Xinru Wang, Lauren Kennedy, and Qixuan Chen write:
The two-phase sampling design is a cost-effective sampling strategy that has been widely used in public health research. The conventional approach in this design is to create subsample specific weights that adjust for probability of selection and response in the second phase. However, these weights can be highly variable which in turn results in unstable weighted analyses. Alternatively, we can use the rich data collected in the first phase of the study to improve the survey inference of the second phase sample. In this paper, we use a Bayesian tree-based multiple imputation (MI) approach for estimating population means using a two-phase survey design. We demonstrate how to incorporate complex survey design features, such as strata, clusters, and weights, into the imputation procedure. We use a simulation study to evaluate the performance of the tree-based MI approach in comparison to the alternative weighted analyses using the subsample weights. We find the tree-based MI method outperforms weighting methods with smaller bias, reduced root mean squared error, and narrower 95% confidence intervals that have closer to the nominal level coverage rate. We illustrate the application of the proposed method by estimating the prevalence of diabetes among the United States non-institutionalized adult population using the fasting blood glucose data collected only on a subsample of participants in the 2017-2018 National Health and Nutrition Examination Survey.
Yes, weights can be variable! Poststratification is better, but we don’t always have the relevant information. Imputation is a way to bridge the gap. Imputations themselves are model-dependent and need to be checked. Still, the alternatives of ignoring design calculations or relying on weights have such problems of their own, that I think that modeling is the way to go. Further challenges will arise such as imputing cluster membership in the population.
This is a good idea they have. But the comparison they’re making is not quite fair when they refer to “weighting methods.” Typically, weighting methods for two-phase samples include a calibration adjustment to calibrate the second-phase sample to known information about the phase-one sample. But the paper from Xinru et al. doesn’t even mention this kind of calibration adjustment, which is standard practice. Instead, when they talk about “weighting methods”, they only consider sampling weight and nonresponse adjustments. So of course their imputation approach is going to do much better than just using nonresponse-adjusted sampling weights. But is it notably better than what practitioners are actually doing in practice, which is calibrating the second phase to first phase information? That’s an important thing to talk about in the paper.
For reference, this R package vignette describes the two-phase calibration adjustments and how to do them in R:
https://cran.r-project.org/web/packages/svrep/vignettes/two-phase-sampling.html#calibration-estimators