Yesterday Nate Cohn wrote about The Big Changes Coming to the Times/Siena Poll, with
more details in their poll of Maine.
Say we want to estimate average Platner support in Maine’s likely electorate, E(Y). But we only have survey respondents, R = 1.
The NYT uses survey weights to weight respondents, E(YW | R = 1). In contrast, some pollsters use MRP, fitting a Multilevel Regression model for Platner support, then applying it to the population, E(E_model(Y | X, R = 1)).
Nate discusses 2 Big Changes to how they construct the weights W.
(The polar bear has not yet hiked in ME, but he is training for it. This above is in TN.)
Big Change 1: Support score
A few weeks ago we saw the NYT started weighting on “synthetic 2024 vote”, which is recalled 2024 vote that is validated with the voter file and imputed if needed.
Now they’re also weighting on support score = E(2024 vote | other X variables). Nate explains the motivation:
While a poll can’t weight on dozens of variables, the support score lets us pile a lot of information into a single measure.
This reminded me of the causal inference context, where D’Amour and Franks (2021) “see especially strong performance for propensity weights computed with respect to the prognostic score”, where the prognostic score is E(Y | X, control). In our survey context, this would be a model for Platner support Y. Instead, the NYT use 2024 vote, perhaps for applicability across multiple outcomes Y ?
Big Change 2: Energy balancing
Beyond adding new weighting variables, they’re also changing how they calculate the weights. Nate notes the challenge of weighting on many variables and interactions with typical sample sizes. So they are turning to the R package WeightIt, which implements the energy balancing method from Huling & Mak (2024):
This article introduces a new weighting method, called energy balancing, which instead aims to balance weighted covariate distributions. By directly targeting distributional imbalance, the proposed weighting strategy can be flexibly utilized in a wide variety of causal analyses without the need for careful model or moment specification.
The energy balancing weights do not use outcome Y, but the paper notes that estimates can be improved with a model for Y.
How do energy balancing weights handle the challenge of jointly weighting on many variables with typical sample sizes “without the need for model specification” ?
I just they’d say “adjust for,” rather than “weight on.” Adjustment is the general term; weighting is just one particular method, and it’s a method that often has problems (as discussed here).
Thanks, Andrew ! Yes, Gelman 2007 (the Struggles paper) came to mind a lot here. They specifically talk about weighting though, not adjustment more generally.
Andrew –
I think I understand why you generally prefer “adjust for” rather than “control for” but I don’t understand why you prefer “adjust for” rather than “weight on” here? If you can answer without getting too technical? Why is it adjusting and not weighting?
Joshua:
“Weighting” is a specific form of adjustment. “Adjustment” is the more general term. Weighting and MRP are two different adjustment methods. If they’re using weighting, that’s their choice; I just prefer the more general framing that is less tied to the particular method being used.
Thanks Joshua and Andrew. They are using weighting, which is why I use the more specific word rather than general adjustment.
To your question, energy balancing is based on the energy distance of Gabor Szekely which is a nonparametric measure of the distance between two multivariate distributions (and can be seen as an extension of the Cramer von Mises distance beyond univariate distributions). The basic idea is weights are defined as the optimizer of the energy distance between a weighted (empirical) distribution and a sample from a target distribution subject to constraints that the weights live on a probability simplex. By minimizing this distance, they aim to rebalance all aspects of the covariate distribution to the target distribution. Although we didn’t prove this in our paper in JCI, the energy balancing objective is convex and you can show that even though it’s set up purely as an optimization problem, you can show the energy balancing weights (in the context of ATE estimation) are an implicit nonparametric estimate of the inverse propensity score weights. But since they optimize for balance rather than estimation of the propensity score, they tend to behave more nicely empirically than flexible estimates of the propensity score.
Thank you so much, Jared !
How does energy balancing handle positivity/overlap violations ? i.e. what the MRP (Multilevel Regression and Poststratification) literature would call empty cells (unsampled regions of the joint covariate space).
causal case from Huling & Mak (2024): control F_{n,0} has mass where treatment F_{n,1} has none (or vice versa).
survey case: region of joint covariate space has target population folks but no sampled folks.
It has no explicit mechanism to deal with positivity violations, however, the “improved” version that adds an extra term to make F_{n,0,w} close to F_{n,1,w} will tend to behave empirically better in these instances, leaning a bit towards the average treated effect on the overlap population. We have another work in a different context https://doi.org/10.1093/biomet/asad038 (with a slightly different distance) that shows that if you let this last term completely dominate, you will estimate the ATO. So the improved version gives you a sort of smooth transition towards that, which is a reasonably safe option in some circumstances if you do have severe overlap issues
Thanks again, Jared ! Does this help with the survey case, where we have to reweight a sample to the target population (e.g. voters in Maine) ?
No, there the structure is simpler and the third term doesn’t have an analog. You *could* formulate the balancing problem differently like in the work of former student Martha Barnard here: https://arxiv.org/abs/2510.22072 in Section 7.2 with extension to an energy-balancing type approach in Supp Section 3.3, but that wouldn’t be what happens out of the box and would need to be done with care and some specific intent about what you want to estimate
Thanks again, Jared !
Starting to take a look at Barnard et al. (2025). I feel silly but I can’t find a definition of ATO ? Is this acronym defined in the paper ? Is “O” for overlap ?
In Section 2.1 Notation, assumptions, and covariate balance: is the population the target population ?
Shira, as always, I appreciate reading these survey statistics posts that help explicitly frame ideas with notation like E(Y |X). I wrote a little bit more about the energy balancing weights here, collecting some resources about the method and discussing how it’s practically different from other weighting methods being used in surveys.
https://www.practicalsignificance.com/posts/energy-balancing-weights-for-surveys/
This is so great, Ben ! Thank you for sharing. Silly question, what do you mean by: