“How Can America Be So Miserable When It’s So Rich?”

The above is the title of a recent a newspaper op-ed (pdf here) by columnist David French. He’s asking a good question! My quick take is that there are four things going on:

1. People want improvement. If you’re at X, then it’s great to go to 1.2X, it’s kinda scary to stay at X, and it’s distressing to go to 0.9X. It does seem that people react to the change in their economic situation rather than than the absolute level.

2. People want more. Housing is more expensive than it used to be, but part of that is that people want to live in bigger apartments and bigger houses. Cars are much more reliable than ever, but nowadays lots of people want two or three cars in their household.

3. Entitlement. Some people are struggling and it makes sense for them to be distressed with the economy. On the other hand, lots of people are sitting there with expensive houses all paid for and some money in the bank, but they’re not saying how amazingly wonderful things are, because they feel that they deserve every bit of it. This averages out to unhappy in the population.

4. Lack of security. The concern that, things might be ok now, but you might have to be scrambling in the future.

This is not to deny that lots of people are struggling economically. I’m just talking here about average poll findings. The question is not why some or even many people are economically miserable, so much as why the average response to such questions is lower than in earlier decades when people had a lot less. The point is that people are thinking about their future, they’re not comparing to how things were in past decades.

French gives some examples of how the competitive economy has distorted the upper middle class. He even talks about youth sports travel teams. It’s hard for me to believe that the high cost of Disney tickets and travel team fees are driving the affordability crisis–neither of these is anything like the struggle to pay the rent and take care of kids or elderly relatives while commuting to some faraway job with rigid working conditions. French also talks about the frustration of flying, but most people get on planes only rarely.

One other thing. French writes:

In this story, maybe the problem isn’t oligarchy. Elon Musk’s billions don’t tangibly change my life.

But . . . Elon Musk’s billions do tangibly change your life! Remember, they went in and screwed up the government. Planes are crashing, people are getting shot on the street, your data are at risk, they’re reducing childhood vaccinations and making it more difficult to develop future adult vaccines . . . so, yeah, you can blame those billions. You can also blame the zillions of dollars spent by lobbyists to legalize gambling on people’s phones.

I agree that “oligarchy” is not the only problem–and, arguably, oligarchy does lots of good things too—but it’s naive to think that it’s not changing your life. Even if you are not personally a gambling addict, you might have a loved one who is. Even if the kids in your family are currently vaccinated, you or they could get killed by the next pandemic. Etc.

Who were “The Makers of Public Policy” in 1965?

I was in the Playroom the other day and noticed this book on the shelf, The Makers of Public Policy, by R. J. Monsen and M. W. Cannon, copyright 1965. Here’s the table of contents:

It’s interesting to see here what is and isn’t included.

1. The section on Formal Power Groups contains one chapter on business, one on agriculture, and two on labor. Organized labor is so much less now that I don’t think it would merit even one chapter.

2. There’s a chapter on “negroes.” I guess makes sense from the standpoint on 1965 that there are no chapters on other social outgroups such as Latinos, women, gays, involuntary celibates, immigrants, disabled people, etc.

3. There’s no chapter at all on organized religion, which is a notable missing power group now but even more so for the 1960s.

4. There’s a chapter on “intellectuals” but nothing on political ideology.

5. There is no chapter regarding the organized Left, but I guess it’s covered in the penumbra of “labor” and “intellectuals.”

6. There’s no chapter regarding the organized Right, and that’s more of a loss. The organized Right has been very influential in American politics and policy ever since the late 1970s, but it was a big deal even as of 1965. This book does have one section on “the white reaction” in the “Negroes” chapter, but that’s hardly enough to cover it.

7. There’s no chapter on the ultra-rich, which in 1965 would pretty much fall in the “business” category but now surely deserves its own chapter.

8. There’s no chapter on science and technology. The book is called The Makers of Public Policy, and my point here is that scientists and technologists make public policy even without (or in addition to) traditional lobbying, on issues ranging from vaccines and DNA testing to social media to climate change.

So I think it’s instructive to look at the topics covered in this sixty-year-old book, partly to see what they missed even then and partly to reflect on how much things have changed.

Also, this won’t be news to many of you but it’s still interesting to see:

And here’s the first page of the index:

Lots has changed in what’s considered important.

Also relevant to the question of how things have changed: What rule did Louis Menand use to select what went into his book on U.S. “cold war” culture of 1945-1965?

“Sane-washing”

After quoting Donald Trump at a political rally with Robert Kennedy Jr.,

That is why today I am repeating my pledge to establish a panel of top experts, working with Bobby, to investigate what is causing the decades-long increase in chronic health problems and childhood diseases, including auto-immune disorders, autism, obesity, infertility, and more.

Palko writes:

Coming at this from a conspiracy theorist/pseudoscience crank worldview, the one word that should set off immediate alarm bells is “autism.” . . . If you had to list the remaining terms in order of tinfoil hat appeal, they would be infertility, autoimmune disorders, and at a distant fourth, obesity. A reporter covering the speech might also feel compelled to add that the causes which RFK Junior has proposed in the past include not just vaccines but also chemtrails, Wi-Fi, and 5G.

Sounds about right: the candidate who endorses the ludicrous Pizzagate conspiracy theory, tried to overturn an election, and is allied with child-murder-denialist Alex Jones is teaming up with a public figure whose big political issue is opposing vaccines.

Palko continues:

Here’s how the NYT covered that quote in the third paragraph from the end of the article on Kennedy endorsing Trump:

Earlier in the day in Phoenix, at his speech announcing the suspension of his campaign, Mr. Kennedy said Mr. Trump had offered him a role in a second Trump administration, dealing with health care and food and drug policy. In Glendale, Mr. Trump said that, if elected to a second term, a panel of experts “working with Bobby” would investigate obesity rates and other chronic health issues in the United States.

I agree with Palko that this is absolutely ridiculous in the context of the actual quote from Trump. The Times is “sane-washing” Trump and Kennedy, working really hard to take dangerous and extreme positions and treat them as reasonable and normal.

I guess this doesn’t matter to political junkies for whom Kennedy Jr. is associated with anti-vaxx conspiracy theories. To many normies, though, I’m guessing that he’s perceived more as a generic Kennedy heir—maybe they don’t know that he makes Dr. Oz and Andrew Huberman look like purveyors of solid medical advice.

This seems like
another example of how junk science and conspiracy theories have moved from the periphery of our culture to the mainstream, and I agree with Palko that it was bad journalism for the Times to report on that speech and sane-wash it.

Newspapers make mistakes all the time; the reasons for posting this one are: (1) this was the first time I’d heard of “sane-washing,” and (2) it’s scary to see pseudoscience escape the narrow world of Lancet/PNAS/Ted/NPR/Gladwell/Freakonomics and potentially become government policy.

I do think it’s possible that the mainstreaming of pseudoscience into elite discourse through the Lancet/PNAS/Ted/NPR/Gladwell/Freakonomics nexus has made it easier for pseudoscience like election denial and vaccine denial to get elite traction (even if it’s different subset of elites).

On the other hand, Merchants of Doubt, etc: Cigarette companies and fossil fuel companies have been working for many decades to blur the lines between science and propaganda, so maybe this would all be happening even without all the junk science that entered the mainstream in the replication-crisis era. I don’t know.

“In that era, undergrad males at Madison and elsewhere, had to take ROTC classes, and he kept intentionally failing them because he was very leftwing politically.”

Paul Alper writes:

Regarding your blog of today, believe it or not, I knew Marshall Brickman pretty well before he became famous. He was an undergrad at the University of Wisconsin in Madison when I was a grad student there. We belonged to the same very left-wing eating co-op, “The Green Lantern.” In that era, undergrad males at Madison and elsewhere, had to take ROTC classes, and he kept intentionally failing them because he was very leftwing politically.

Inasmuch as I am a helpful sort, I offered to nominate him for “Military Ball King,” but he declined. Marshall was a really gifted musician as well as a writer. In addition, he looked strikingly like the actor Richard Carlson who played Herb Philbrick in the 1950s television series I Led 3 Lives.

Why quantitative understanding of effect sizes matters, even if all you care about is the presence of the effect

In reaction to my article with Andy King proposing post-publication review, Dan “Fast and Frugal” Goldstein writes:

Your process limits information search, computation, and time so it seems fast and frugal to me. Happy you still associate me with that term. It was something I coined as a grad student.

In your proposal, only hit papers get audited. It reminds me a bit of Mel Brooks’ The Producers in which the protagonists use the logic “who would audit a flop?” and stay under the radar by intentionally producing a bad show. Fraudsters have likely attempted to make their work seem worthy of publication while ensuring it doesn’t attract too much attention. Just like in The Producers, though, this sometimes backfires.

I don’t know about that! My impression with fraudsters is that they think that fraud is normal science, perhaps out of some mixture of bad education in research methods, a view that “everybody does it,” and a general lack of understanding of how non-cheaters (like you and me!) think. Think of people like Wansink who gave general advice to to the world on how to p-hack, or Gino and Ariely, who published papers on dishonesty, or Mary Rosh, who surely believes that whatever shady statistical manipulations she does are nothing compared to the dastardly deeds done by the Democrats.

I’m sure there’s tons of below-the-radar cheating and bad science that we don’t hear about, but a fair number of prominent science fraudsters seem to enjoy the limelight. One reason for this seemingly self-sabotaging behavior, I think, is that cheating enabled these people to attain great professional success for years. They had no reason to think the juice would stop flowing.

To return to my proposal with Andy King: I think it’s ok that only the hit papers get audited. Bad papers that get no intention aren’t doing much damage, right?

Goldstein adds:

By the way, I was just having a conversation about your sensing that something was amiss with the LaCour study. For years now I have used this quote of yours in a talk I give about putting numbers into perspective. I argue that it’s really important that people learn how to put numbers into perspective because if they don’t, they won’t notice that something is unusual and worthy of a deeper audit. You somehow sensed something was up with the Lacour result. You didn’t think it was fraud yet but you knew it was strange because you know how to put such differences into perspective:

A difference of 0.8 on a five-point scale . . . wow! You rarely see this sort of thing. Just do the math. On a 1-5 scale, the maximum theoretically possible change would be 4. But, considering that lots of people are already at “4” or “5” on the scale, it’s hard to imagine an average change of more than 2. And that would be massive. So we’re talking about a causal effect that’s a full 40% of what is pretty much the maximum change imaginable. Wow, indeed. And, judging by the small standard errors (again, see the graphs above), these effects are real, not obtained by capitalizing on chance or the statistical significance filter or anything like that.

My colleagues and I recently wrote a paper on this general topic of average effect sizes. It’s our contention that people generally are way too optimistic about possible effect sizes, in large part because they don’t think about variation. If you ask someone to hypothesize an effect size, you’ll typically get a guess of the largest effect that might occur.

But what if you don’t really care about effect size–you just want to know about the effect?

For example, maybe you don’t believe that women during certain times of the month are three times more likely to wear red or pink shirts, but you are interested in some sort of evolutionary psychology theory of sexual display. In that case, why should the effect size matter? Why care that a study reported an estimate that was ridiculously implausible?

I have two to this questions, and thus two reasons why effect size is important even for problems where you don’t directly care about effect sizes:

1. Effect sizes vary. An treatment that has an effect (that is, a true effect, not just an estimated effect) of 0.1 for one group of people in one setting could have an effect of -0.2 in some other scenario. A treatment effect in an experiment is the sum of all sorts of things, positive and negative, and there’s no logical reason to think the sign of the effect will be preserved. Effect size matters. The issue is not just that a smaller and more realistic effect size is less important; it’s also that smaller effects can be more easily produced by other factors, and this reduces the generality of any claims, even if the experiment at hand was done well.

2. Experiments produce standard errors as well as estimates. If the standard error from a study is large compared to any realistic effect size, then the study contains very little information. Effect size is important in understanding the informativeness of an experiment, and to do this right you need to have some sense of what the true effect size could be. You can’t just use a point estimate from the study itself, as this estimate will inherently be too noisy to use to judge the information in the study. As I wrote in this note for the Annals of Surgery, Post-hoc power using observed estimate of effect size is too noisy to be useful.

Survey Statistics: equivalent models, equivalent weights (locally)

Last month we saw that the Times/Siena Poll is now using energy balancing weights (Huling & Mak, 2024). In a toy example, we saw under which outcome models these weighting methods might do well. I was inspired by Little 2004, who saw under which outcome model the inverse-probability-weighted estimator (a.k.a. Horvitz-Thompson) does well. This explains the HT estimator’s poor performance in Basu’s (1971) elephants example (which I’ve used in my post Basu’s Bears).

From Little 2004:

Andrew’s 2007 “Struggles” paper and the 2026 MrPlew paper go the other way: start with an outcome model and back out the weights. These folks all worked pretty hard. I wondered if I could just use Thomas Lumley’s survey package to get the weights. See my posts “struggles with equivalent weights” and my continued struggles. But in the simulation I used a linear outcome model. It would have been more interesting with a logistic outcome model !

The linear case was handled by Andrew’s 2007 paper. The 2026 MrPlew paper extends this by noticing that the linear equivalent weights are derivatives, and using this to define locally equivalent weights, green highlighting by me:

Then the 2026 MrPlew folks use these equivalent weights to compare the target population to the weighted sample, as an MrP diagnostic. (Unlike the Huling & Mak paper on energy balancing weights, the MrPlew folks look at one covariate function at a time, not the entire covariate distribution at once.) One of their examples is from Lax & Phillips 2009 Gay Rights in the States, which I blogged about for pride. In this example, the implied covariate (im)balance for MrP looks not great ! Here’s their Figure 5b:

As Andrew blogged: “It makes sense that implied covariate balance can sometimes be worse for MRP than for raking. MRP is a smoothed version of raking, and unsmoothed raking can overfit.” Yes, but here it’s worse than the raw uncorrected balance ! The MrPlew authors caution here that the locally linear approximation may be extrapolating poorly, and that we shouldn’t over-rely on this model check. They also say that “logistic regression generally balances the variance-weighted covariates, but not the covariates themselves”. I am glad they included this illustrative example and I want to think more about it.

Thoughts ?

People sometimes talk about “the Jewish vote,” but what’s relevant is not really the Jewish vote or Jewish public opinion; it’s really about campaign contributions and the news media. Also similar with Mormons.

rel1.png

rel2.png

At the end of the second world war, Jews were a bit over 3% of the U.S. population, voted at a high rate, and were concentrated in the swing state of New York. Jews had two big issues–Israel and political liberalism, and the Jewish vote was a thing. Not the biggest thing in politics, but a powerful voting bloc in a swing state, that’s something.

Nowadays, Jews are about 2% of the population, and New York is no longer a swing state. When it comes to national politics, the Jewish vote doesn’t really matter. As I wrote nearly twenty years ago:

The underlying question, though, is why should we care about a voting bloc that represents only 2% of the population (and even if Jews turn out at a 50% higher rate than others, that would still be only 3% of the voters), most of whom are in non-battleground states such as New York, California, and New Jersey? Even in Florida, Jews are less than 4% of the population. I think a lot of this has to be about campaign contributions and news media influence. But, if so, the relevant questions have to do with intensity of opinions among elite Jews rather than aggregates.

The reason this comes up is that sometimes Jewish-related issues come up in politics and people will point to some poll or another of Jewish opinion. But Jewish public opinion doesn’t matter. When it comes to Jews in politics, what matters are the opinions and actions of campaign contributors and media executives.

Here’s the thing. Twenty years ago, it’s my impression that rich Jewish political donors and elite Jews in the news media were mostly politically aligned with average American Jews. Not exactly–I’m guessing the rich donors were not as far to the left on economic issues as random Jews in the country–but pretty much politically liberal and pro-Israel. Since then, things have changed: American Jews are split on Israel and remain strongly Democratic, but now there are many prominent Jewish donors and media executives on the right, both with regard to Israeli and American politics.

So, when a Jewish lobbying group taking some conservative position, and people point out that this is not the majority opinion of American Jews, my take on it is that, from a political perspective, the majority opinion of American Jews doesn’t really matter; to the extent that national politicians would think about Jewish issues, it’s the big donors and media executives that are the most relevant.

Similar issues arise with any ethnic group. The average voter will disagree on key issues with donors and influencers. It’s just particularly clear with Jewish Americans because the direct relevance of the voters has declined so much over the years, but people are still often in the habit of framing ethnic politics in terms of voters.

Mormons as the conservative counterpart to Jews

Indeed, you could say something similar about Mormons, who, like Jews, are a minority religion in the United States, with a strong ethnic identity, adherents concentrated in a few states, many rich people, strong political views, and lots of political involvement.

A few years ago we looked at political attitudes among rich and poor among different groups:

Jews mostly were voting for Democrats, Mormons for Republicans. No surprise.

More interesting was the difference between rich and poor. Rich Jews and poor Jews had similar voting patters–ok, actually, we’re comparing the upper third of income to the lower third here, so we’re not really talking about rich people here, we’re just using data from the higher- or lower-income people in national surveys. Rich Mormons and poor Mormons voted much more differently, with rich members of that religion being much more likely to vote for Republicans.

Since 2004, things have changed, and now there might be a divergence between rich Jews, who seem to be very pro-Israel and moving toward the Republicans (at least from what we hear about big campaign contributors), and the general mass of Jews in the country who are more divided on Israel and mostly remain supportive of the Democrats.

Similarly with Mormons: I guess the rich Mormons remain staunch Republicans, but lower-income Mormons may well have moved along with other lower-income white people toward the Republicans too. So the rich-poor voting gap among Mormons might not be so much larger than among Jews anymore.

How generic language shapes the development of social thought

Recently in the sister blog:

Generic language, that is, language that refers to a category as an abstract whole (e.g., ‘Girls like pink’) rather than specific individuals (e.g., ‘This girl likes pink’), is a common means by which children learn about social kinds. Here, we propose that children interpret generics as signaling that their referenced categories are natural, objective, and have distinctive features, and, thus, in the social domain, that such language affects children’s beliefs about the social world in ways that extend far beyond the content they explicitly communicate. On this account, even generics expressing uncontentious content (e.g., ‘Girls are great at math’) can lead children to think of categories as defining fundamentally distinct kinds of people and contribute to the development of stereotypes and other problematic social phenomena.

Here’s the full article.

Pretty maps of the NY mayoral election vote. (The meta-point here is that people have a (false) intuition that any complicated piece of information can be conveyed in a single plot.)

In reaction to my recent post, If Cuomo had been able to run against Mamdani head-to-head, would he have won?, sociologist Kieran Healy posted a pair of maps showing precinct-level results from the recent New York mayoral election. One of them is above, and below is a detail from the other.

The top map uses a bidirectional color scheme of the sort that I like, except that usually I’d see the darker colors at the extremes, fading to white in the middle, so it was surprising for me to see this go the other way.

In his post, Healy shares a lot of detail, not just on his choices for what the maps should look like, but also on the data processing and all the steps along the way.

Following up, Healy writes:

The overall effect of a dot-density plot is sensitive to the choice of colors, particularly when there is more than one kind of dot being displayed. This in turn is heavily related to relative brightness, which in practice itself depends not only on the values encoded on the plot but on the sort of monitor or screen it’s being displayed or projected on, how much ambient light there is, etc, etc.

Again–also as noted in the post–while dot-density plots do better than choropleths in overcoming the “Land Doesn’t Vote” problem (in this case, “Precincts aren’t real”), at the end of the day any spatial representation of something like individual voting data is going to be caught out by this issue one way or another. So overall you just have to show multiple representations of the data, many or most of which will be better off not being maps at all. (Cartograms, whether based on grids or some sort of sphere-packing methods, are another solution, and of course create their own problems.) There’s no one beats-all-comers method, and I don’t present the dot-density map as one. I use stuff like this in my own classes precisely because you end up with a lot of choices to make on how to view the data, and it’s good to encourage students to work through the choices and their consequences.

It’s also good for getting across to students how those choices, and the tradeoffs associated with them, can’t really be effectively communicated in the graph or map itself, because they’ve already been made. And hopefully the students end up recognizing (as with any sort of method or tool) the importance of some working community of researchers providing the context in which these things get made, interpreted, and trusted. Images, graphs, and maps are a pointed case of the general issue, just because it’s so easy for them to escape that context when they circulate. (I have a recent general-audience talk about this.)

I agree with Healy’s points. Phil and I once wrote a paper, All Maps of Parameter Estimates are Misleading.

The meta-point here is that people have a (false) intuition that any complicated piece of information can be conveyed in a single plot. One reason I hate the famous Napoleon-in-Russia graph is that it has encouraged this sort of thinking. When trying to make or read a graph, it can be helpful to start by stepping back and acknowledging hat, in general, one single plot (or even two plots) won’t do it all.

He “washed his hands in a can of tetraethyl lead at a press conference, claiming he was ‘not taking any chance whatever’. He knew this to be a lie, having already succumbed to a bout of lead poisoning.”

OK, this is absolutely horrifying:

The ill effects of ingested lead and other heavy metals had been known since the 1920s, when employees at TEL [tetraethyl lead]-refining plants began hallucinating butterflies and going into convulsions of violent insanity (at least ten died). ‘Smelter nose’, a finger-sized hole in the septum, was an occupational hazard at plants. Horses near the Bunker Hill stack dropped dead; children were hospitalised with kidney damage, forced to undergo excruciating chelation therapy. By the 1970s scientists were beginning to link lead emissions with surging delinquency and crime rates.

The industry’s response was to deny everything or, at best, occasionally raise the height of its smokestacks. Company quacks put out statements asserting that high levels of lead in human bodies were not only harmless but ‘natural’. Thomas Midgley Jr, a General Motors engineer with the diabolic distinction of having invented both leaded gasoline and chlorofluorocarbons, washed his hands in a can of TEL at a press conference, claiming he was ‘not taking any chance whatever’. He knew this to be a lie, having already succumbed to a bout of lead poisoning. (Years later, paralysed with what was said to be polio, he strangled himself in the ropes of a contraption designed to hoist him out of bed.)

In the 1970s, my dad worked for the EPA in their mobile source enforcement division: their job was to stop people from illegally selling leaded gasoline and to adjudicate petitions from mom-and-pop refineries that, for various reasons, wanted exemptions from the new rules on unleaded gasoline.

But that story about Thomas Midgley, Jr.: Wow. What an evil guy. The linked article (a review by James Lasdun of a book by Caroline Fraser) is just full of horrible stories.

I guess that the world is full of evil people and always will be. The challenge is to avoid putting them in positions where they can do a lot of harm.

Survey Statistics: poststratification without population level information

Poststratification uses population data on X to estimate E(Y) via E(E(Y | X, R = 1)), where R = 1 are survey respondents who provide Y and X. When the inner expectation “E” is estimated via Multilevel Regression, this is called MRP. The outer “E” needs p(X), population data on X.

Sometimes we have to estimate the population distributions. We’ve seen a few examples:

  1. “2 flavors of calibration”: Say we have p(X), but we also need p(Z | X), the population distribution of another variable Z. We can estimate p(Z | X, R = 1) using survey data, but nonresponse could make this unreliable. Say we have population data on aggregates p(Z) (e.g. from census tables), then we can logit-shift to anchor to the population aggregate. See Kuriwaki et al. 2024.
  2. “MRPW”: Say we have p(X), but we also need p(W | X), the population distribution of the survey weights W. We can estimate p(W | X, R = 1) using survey data. Then because we assume survey weights are proportional to inverse probability of response 1/p(R = 1 | W, X), we can get what we need by Bayes Rule.
  3. “weights and MRP for voters”: Say we have p(X), but here we need p(X | V = 1), the distribution of X among the population of voters. By Bayes Rule we can get this via p(X) and p(V = 1 | X). The latter can be estimated from population turnout history and vote intent among survey takers.

In all cases, we have some anchor to the population, e.g. via aggregate totals, survey weights, or turnout history. This brings me to Andrew’s post asking 2016 pollsters to poststratify on party ID. We don’t have population aggregates to logit-shift to. But Andrew comments:

party ID is changing much more slowly than the distribution of vote preference, which itself is changing much more slowly than differential nonresponse.

So in our sample party ID (Z) is changing over time quickly, but mostly due to differential nonresponse:

p(Z | t, R = 1) = p(R = 1| Z, t)/p(R =1 | t) * p(Z | t) = differential nonresponse at t * party ID at t

Andrew cites his coauthored paper Reilly et al. 2001, which fits a model to smooth the poststratifying variable Z over time. This won’t help with the component of differential nonresponse that is constant or slowly changing over time, but it prevents the polls from jumping around with every swing in differential nonresponse.

Andrew also cites his coauthored paper The Mythical Swing Voter, which uses 2008 exit poll data on Z to adjust polls from 2012. This also relies on the assumption that Z changes slowly over time, our anchor in the absence of population data.

We’ve been talking about adjusting for party ID. Another variable we might want to adjust for is interest in politics. In “adjusting for interest in politics” we cite Andrew and Gustavo‘s Challenges in Adjusting a Survey That Overrepresents People Interested in Politics. They see an increase over time in interest in politics, which they say could be nonresponse bias (people more interested in politics taking surveys) and/or a change in the population due to increased political polarization.

Was this USDA survey really “redundant, costly, politicized, and extraneous”?

Joshua Brooks writes:

I know you’ve posted on the topic more generally but don’t recall if you’ve discussed this in particular. Given the timing in relation to cuts in food assistance, It seems a particularly egregious example of the politicization of data.

The news article, published in late 2025 by an organization called Food Tank (“The Think Tank for Food”) is titled, USDA Ends Key Food Security Report, Leaving Advocates in the Dark, and it begins:

The U.S. Department of Agriculture (USDA) recently announced it will terminate its long-running Household Food Security annual report. The resource is one of the country’s most comprehensive tools for measuring hunger and food insecurity.

The USDA justified the decision as a cost-saving measure, claiming in a statement that the survey is “redundant, costly, politicized, and extraneous.” . . .

Produced for the past three decades by the USDA’s Economic Research Service (ERS), the report offers insights used by researchers, policymakers, and advocates working to reduce food insecurity in the U.S. Anti-hunger advocates argue the move will make it far more difficult to track the impacts of policy changes, including recent cuts to the Supplemental Nutrition Assistance Program (SNAP). . . .

Although advocates are looking for options to fill the research gap, Karen Perry Stillerman, Deputy Director of the Union of Concerned Scientists argues that there are no options that match the scope. “How are the data redundant?” she asks. “The USDA survey serves as the official data source of national food insecurity statistics.”

I followed the link [here’s a version from the Internet Archive] and here’s the USDA’s official statement:

Ummmm, what is this? The Ministry of Propaganda?? Imagine what’s it’s like if you’re a normal person working for the USDA, you just want to do your job, but this is the kind of crap you have to deal with.

But I have a serious question here. The USDA claims that the survey is “redundant, costly, politicized, and extraneous.” Just to go through these:
– I guess the study could be “costly”; to assess whether it’s too costly to be worth it, I guess you’d have to talk about its benefits.
– The study could be “politicized,” but nowadays just about everything is politicized, so that seems kind of irrelevant.
– I can’t see why they are saying the study is “extraneous”; it seems very relevant to USDA-related issues.

But the thing I wanted to focus on here is the claim of redundancy. The USDA says the study is redundant, while the advocates say, no, the data aren’t redundant.

One way for the USDA to address this is to give users help on how to get the equivalent of these data from other sources. I did a quick google and found this page on the USDA’s website:

This seems more serious. I assume the people who are in charge of this webpage have no connection to whoever is the hack who wrote that earlier press release.

So here’s my question. Do the data described on that linked page provide the equivalent information to the now-canceled Household Food Security annual report? Actually, both the Food Tank news article and the USDA press release leave me confused, as they refer both to a “Food Insecurity Survey” and to “Household Food Security Reports.” Does that mean they’re canceling a survey and also canceling a report?

If it’s the survey that’s expensive and redundant, then they could still do the report, no?

The obvious conclusion to be drawn from the ridiculous press release is that the government is canceling the survey and the reports for purely political reasons, some combination of wanting to avoid bad news coming out and ideological opposition to aid to the poor. But it would be good to know if they’re correct in saying this survey is irrelevant.

Maybe I’ll contact Karen Perry Stillerman of the Union of Concerned Scientists and ask what is the basis of her claim that the data are not redundant. Also I can contact the USDA economists on this page. The USDA economists have direct emails; Stillerman doesn’t, but there’s an email given for her media contact, who I guess can connect me to whoever is data-knowledgeable at that organization.

I’ll report back to you!

It’s all about the Super Pacs: How the New York Times completely misreported campaign contributions in the Maine Senate race

Tom Ferguson came across this news article, Who Really Has the 2026 Midterms Cash Edge?, and was disappointed to see this completely wrong graph:

The problem here is not the inclusion of no-longer-candidate Platner, as that’s noted in a footnote. Rather, as Ferguson says,

The Times shows Platner outraising Collins; whereas she is millions and millions of dollars ahead, as our charts show.

Here’s the chart that Ferguson shared with us the other day:

If Collins raised $39 million, why did the Times say that Collins only raised $8 million? (They also understated Platner’s total, but by a lot less, counting $13 million instead of $16 million.)

I asked Ferguson how this happened–how did the Times screw up so badly? He replied:

Probably they just used the campaign fund of the candidates. The Super Pacs report elsewhere. You have to look them up. This is normal; we’re clear about that when we did Platner. Republican candidates like Collins are really operating with a pack of funds. That’s the famous coordination discussion, BTW. Now rendered even legally moot by the Supreme Court decision.

Collins has many different vehicles supporting her. Easy to find and not new.
The Times reporters are just lazy; they know about the Super Pacs, but can’t bring themselves to do work. That’s the kind interpretation.

Dayum.

Ultimately, the problem here is no so much with the New York Times–large as they are, they’re just one news organization, and they’re trying to do their best–but with the hollowing-out of the news media more generally.

To put it another way, the problem is not New York Times is not the problem. The problem is that the New York Times is one of the few large independent news organizations out there. If there were lots of other orgs reporting these things, we wouldn’t have to rely on the Times not screwing up.

Ferguson continues:

Contrast the endless articles about Democrats talking in Maine deliberations. The billionaire-tasked Times guy should do some work on Collins.

Cf. our discussion of sources in the first post:

We have used data from the Federal Election Commission to construct similar figures for the much-discussed Maine Senate race. Incumbent Senator Susan Collins is running on the Republican ticket, while Graham Platner is her Democratic challenger. Our totals reckon in contributions from Super Pacs and other outside organizations spending on behalf of either candidates or against one (which we count as spending for the candidate’s opponent).

The note’s a killer, so I’ll copy it here:

Federal Election Commission bulk data downloads are not updated at lightning speed. There is a time lag before individual electronic filings are incorporated into those files. In this case, the bulk data downloads are missing the 12-day Pre Primary Report (12P) (filed before the June 9 primary) and contain contributions to the principal campaign committee up to and including May 20, 2026. The bulk downloads are also missing the independent expenditures spent through election day. We obtained the electronic filings of the candidates’ principal campaign committee and the independent expenditures to fill the gap in the bulk data downloads. We downloaded these electronic filings June 12-14. Collins uses multiple committees to raise and spend money, and these committees have different filing deadlines. The Pine Tree Results PAC filed a 12P and reports contributions up to and including May 20. The Lead Maine Committee has contributions until April 28. The Stronger Maine Super PAC has contributions until March 31, The Collins Victory Committee is March 31, and the Susan Collins for Maine JFC is March 31. Collins also raises money for her principal campaign and leadership committees via joint fundraising committees (JFCs). These are shared accounts that allow several candidates or party committees to raise money together. A single donor writes a “parent” check to the JFC, which then is divided among the participating committees. When Collins is the clear beneficiary of such arrangements, such as with the Collins Victory Committee, we count the full parent check as part of her donor distribution. When Collins is merely one of several candidates involved in the JFC, such as with One Team Senate Majority, we count only the subdivided portion given directly to Collins as part of her donor distribution and not the full parent check. Counting the parent check for committees she controls but only the subdivided check for committees she merely joins lets us credit each donor’s true contribution to Collins exactly once without double-counting the same dollars or absorbing money raised on behalf of other candidates.

So, yeah, they had to do some work.

The above-linked New York Times article sucks for two reasons:

1. It got things way wrong, completely missing the story of the Republican candidate’s massive fundraising edge.

2. It was written in an overconfident style with no indication to the reader that the news story was actually missing more than half of the campaign cash out there.

I’ll forward this to my colleagues at the Times. Maybe they’ll run a correction?

A ranked-choice election in Maine: Using voting data to understand preferences

Evan Rosenman writes:

The implosion of Graham Platner’s Senate campaign in Maine has upended a marquee Senate race, leaving the state Democratic party just a few weeks to choose a substitute nominee. A planned nominating convention on July 25th has drawn considerable candidate interest. But the mathematical properties of ranked choice voting add a strange wrinkle to these deliberations.

The Maine Democratic Gubernatorial Primary

Three of the top contenders to replace Platner are former gubernatorial candidates: Nirav Shah, former director of the Maine Center for Disease Control and Prevention; Troy Jackson, former Maine State Senate president; and Shenna Bellows, Maine’s secretary of state. All three ran for the Democratic nomination for Governor, losing the primary to Hannah Pingree, former speaker of the Maine State House.

June’s primary results are given below. (Data from Wikipedia.) Maine uses ranked choice voting (RCV) in primaries and federal elections, so voters could rank up to six choices for Governor. Using the instant runoff algorithm, candidates were sequentially dropped based on who had the fewest first-choice votes, and ballots were reallocated to each voter’s next-ranked choice. Jackson, Shah, Bellows, and Pingree were highly competitive, each receiving between 20% and 27% of first-choice votes. Of the four, Bellows was eliminated first, then Jackson. Shah fell to Pingree in the final tabulation round.

Candidate Round 1 Round 2 Round 3 Round 4
Pingree 50,552 (23%) 55,360 (26%) 75,671 (36%) 111,750 (56%)
Shah 58,606 (27%) 62,860 (30%) 72,681 (35%) 86,950 (44%)
Jackson 45,959 (21%) 47,597 (22%) 60,010 (29%) Eliminated
Bellows 44,770 (21%) 47,049 (22%) Eliminated
King III 17,860 (8%) Eliminated
Exhausted ballots 4,881 (2%) 9,385 (4%) 19,047 (9%)
Continuing ballots 217,747 212,866 208,362 198,700

These results have taken on extra significance as the state party seeks democratic buy-in for the selection of a substitute Senate nominee. Media outlets, for example, have routinely referred to Shah as the “runner-up” in the Governor primary. But analyses of the individual ballots cast in the primary reveal a surprising mathematical fact: though she was eliminated before them, Bellows would have defeated either Shah  or Jackson in one-on-one elections.

Mathematical Details

This unintuitive fact is a generalization of a well-known feature of ranked choice voting elections: it does not satisfy the Condorcet winner criterion.

First, some definitions. Suppose we have an election with a set of candidates C:

  • A “Condorcet winner” is a candidate in C who would defeat all the other candidates in a head-to-head election. A Condorcet winner need not exist for any given C; think of rock-paper-scissors, where each option wins against one alternative and loses against the other. But Condorcet winners exist in many standard election settings.
  • The Condorcet criterion is a feature of electoral methods: a method satisfies the criterion if it always selects a Condorcet winner when one exists.

Standard plurality elections – in which voters make one selection, and whomever gets the most votes wins – do not obey the Condorcet criterion. This is well-understood due to the “spoiler effect.” For example, a Libertarian candidate may attract voters who would otherwise prefer a Republican to a Democrat, siphoning enough voters such that a Democrat obtains the most votes.

Because voters express richer preferences in RCV elections, the method is considered better at identifying Condorcet winners. But it can easily be shown that RCV also does not satisfy the Condorcet criterion. This is not purely hypothetical. In a 2022 U.S. House special election in Alaska, Democrat Mary Peltola was elected against two Republican opponents: Sarah Palin and Nick Begich III. An analysis of the underlying ballot data revealed that Begich was a Condorcet winner. But he was eliminated in the first round because he received slightly fewer first-choice votes than Palin, allowing Palin to advance and lose to Peltola.

As RCV does not obey the Condorcet criterion, it stands to reason that the order of elimination need not correspond to who would win head-to-head elections. This is indeed true. A candidate eliminated in an earlier round may well have defeated one eliminated in a later round in a head-to-head election.

Results in Maine

We can understand the electorate’s preferences in Maine because the state releases its cast vote record: the anonymized set of rankings for every ballot cast. These data are available online and have also been analyzed extensively by the election advocacy group FairVote.

To assess how two candidates A and B would fare in a head-to-head election, we look at the set of ballots that rank at least one of them. Any ballot in which A appears before B, or A is ranked and B is not, represents a voter who prefers A to B; any ballot in which B appears before A, or B is ranked and A is not, represents a voter who prefers B to A.

In the table below, we summarize all the head-to-head matchups among the top four candidates. Note that if the final column is positive, then A defeats B; if it is negative, B defeats A.

Candidate A Candidate B % of Ballots
Listing Neither
% Who Prefer A % Who Prefer B A vs. B Margin
Bellows Pingree 14% 41% 44% –3%
Bellows Jackson 19% 48% 33% 15%
Bellows Shah 12% 45% 43% 3%
Shah Pingree 10% 39% 50% –11%
Shah Jackson 13% 50% 37% 12%
Jackson Pingree 14% 33% 52% –19%

Pingree wins all three of her matchups, indicating she was indeed the Condorcet winner. But notably, Bellows wins every matchup except the one against Pingree. She was preferred to Jackson on 48% of ballots while he was preferred on 33%, with the remaining ballots listing neither candidate. Bellows had a narrower margin against Shah, but she was preferred on 45% of ballots to his 43%.

These results reflect the strengths and pitfalls of RCV. Because voters’ ranked choices are recorded, we can better assess the electorate’s head-to-head preferences among many candidates. But elimination orders under instant runoff needn’t reflect these preferences. In closely contested elections like the Maine Democratic gubernatorial primary, this can yield unintuitive results – with big implications for the next big question: whom to choose as a substitute Senate nominee.

Following up on Rosenman’s analysis, I have a few points to raise:

  1. Why should I care who would win in a head-to-head race? I’m not trying to ask this in an aggressive way; it’s just not clear to me why this should be the question to ask, or why we should care about a Condorcet winner. Another way to say this is that intensity of preference could matter too.
  2. A related issue is that there are lots of people who could potentially be qualified to be the senator from Maine–after all, a senator doesn’t really have to do much, their staff does all the work, right? Just ask Senator Grassley from Iowa! My point here is not to trivialize the election–people live or die based on who is elected to Congress–just that the steps of choosing a candidate involve a winnowing from many many possible choices. The Condorcet winner criterion and other similar rules apply only after drastically limiting the number of options.  From that perspective, I’d be more inclined to rate candidates based on a summing of pluses and minuses for various attributes, rather than head-to-head comparisons. I get that the general election is a head-to-head race so you need to think about such things, but from a political theory perspective, or from a which candidate-to-choose perspective, I see this Condorcet thing as a blind alley.
  3. Who you’d want to run for governor isn’t necessarily the same as who you’d want to run for senator.  I say this for two reasons.  First, they’re different jobs:  what it takes to run the executive branch of a state is different than what it takes to be a member of the national legislature.  Second, the main goal of a political party is to win the election, and it could take different things to win in the two races in Maine this year.  I don’t know how important this is, as I have no sense of politics in that state. I’m just raising the issue.

Survey Statistics: quantifying uncertainty in ranked choice voting polls

We’ve talked about uncertainty in polls (see Margin of Error, Total Margin of Error, Total Margin of Error II) and we’ve talked about ranked data (see exploded logit !). A new paper, Rosenman & Liang 2026, looks at uncertainty in ranked choice voting (RCV) polls.

Recall the multinomial logit model that Train (2009) Chapter 7 calls the exploded logit:

P[ranking Other then Left then Right] = exp(f_Other) / sum_c’ exp(f_c’)   *   exp(f_Left) / (exp(f_Left) + exp(f_Right))

Without covariates, it has only 3 parameters: f_Other, f_Left, f_Right. It makes the independence from irrelevant alternatives (IIA) assumption to go from these 3 parameters to rank probabilities.

In contrast, the multinomial model in Rosenman & Liang 2026 does not make the IIA assumption and has 14 parameters, one for each of 15 possible rankings minus one so they sum to 1:

P[ranking Other then Left then Right] = pi_{Other, Left, Right}

Rosenman & Liang 2026 note that in RCV the election outcome is not expressable as one parameter. Instead, the winner is determined by instant runoff:

  1. If a candidate wins >50% of first choice votes, they win.
  2. Otherwise, the candidate with the least first choice votes is eliminated, and each ballot counts for its top remaining choice. Return to step 1.

Say you use polling data to estimate rank probabilities pi_j for each ranking j. These estimates differ from the true probabilities due to many sources of error (see our favorite Figure 2.5 from Groves et al. shown in quantity vs quality and is a mismeasured X better than none at all ?). Rosenman & Liang 2026 focus on sampling error.

How can we propagate uncertainty about the rank probabilities pi_j to uncertainty about the RCV winner ? If you have draws from the posterior of pi_j, you can do instant runoff on each to get a winner for that draw. This gives win probabilities according to your model and data.

To see the importance of uncertainty in RCV, let’s look at their 2022 Alaska House special election example. With 3 candidates, RCV is determined by 5 margins (see their Lemma 1). Most of these margins are well-identified by the data, but 2 were quite close: Palin vs Begich first choice margin and Peltola vs Palin pairwise margin. They plot these 2 margins in the right panel of Figure 1. The true outcome is the black dot, with sampling uncertainty shown as ellipses around it. For small sample sizes (the biggest ellipse), we see that a plurality of the mass falls into green, where point estimates would declare that Begich wins. Uncertainty quantification would help put this in context, giving all candidates win probabilities around 20-40%, showing the race is difficult to call with such small data.

For details, see Rosenman & Liang 2026.

 

It’s all about the nonlinearity: An interesting statistical example of flaws in a voter impact index

The following came in the email the other day:

I’m reaching out to introduce the Voter Impact Index, a new data tool from PowerMoves that assigns every U.S. zip code a voter impact score based on the recent competitiveness of six federal and state elections tied to that location.

The Index may be useful in your teaching or research in a few concrete ways:

— Classroom discussions on political geography, voter mobilization, and the relationship between where people live and how much their votes matter
— Research applications exploring electoral competitiveness, voter sorting, and the civic behavior of movers (we estimate 15 million registered voters relocate annually)
— Student projects analyzing zip-code-level electoral data across districts

The underlying data, code, and methodology are fully open and accessible via GitHub through our website at PowerMoves.Vote — making it straightforward to build on or replicate.

PowerMoves is a nonpartisan project. The Index draws from trusted nonpartisan sources and assigns scores regardless of party affiliation.

I was curious so I looked up my own zip code, and here’s what came up:

A “medium” voter impact of 44/100. Are you kidding? Yes, you can get lower impact scores (just try typing in 02139), but something close to the midpoint on a 0-100 scale doesn’t sound right to me. We almost never have close elections. New York is not a swing state, and even our local elections are never close.

OK, the 2022 governor’s election in NY was pretty close, I’ll grant them that, and the 1994 race was even closer, as were 1982 and 1978 . . . but that’s going back pretty far, and they’re only weighting the governor races at 15% (go here and scroll to the bottom), so I was puzzled as to how voters in our district can be judged to an impact of 44 on a 0-100 scale. Even if you count the governor’s election as close (and it wasn’t that close), that would still only you to 18.

If you read through that document carefully, you can figure out what’s going on:

OK, there’s this weird bit about dividing by 2 or 3, but that’s not the key issue. The big problem, I think, is linearity. For example, in the 2024 presidential race, Kamala Harris won the two-party in New York by a 13-point margin. Not close at all! Really not close, considering that, had the state election been close, there’s no way that New York’s electoral votes would’ve been decisive. My voter impact for this election was approximately zero (see some calculations here, albeit from an earlier year). If you want to get technical about it, the probability my vote is decisive is something like 1/100 of the probability that a swing state’s voter will be decisive.

So if the “presidential election” contribution to this index is 100 for Wisconsin, Michigan, and Pennsylvania, and something like 50 in a state like North Carolina or Georgia, then it should be approximately 1 in New York. Or maybe 0.1. Or maybe 2. In any case, some tiny number. Even the governor’s race, which Hochul won by 6 percentage points . . . ok, that’s close, but, again, there are closer races for governor. I went online and looked it up, and there were a couple races decided by less than 1 percentage point of the vote. If those tossups count as a voter impact as 100, then maybe the New York race would be a 50? or maybe something less than that?

So if you add all up all these voter impact score and weight them, you might get something like a 10 for my district, if you’re being generous. Not 44.

It’s an interesting example. At first, doing this linear scaling could seem to make sense. But not if your goal is to measure voter impact.

To put it another way, their measure is underestimating the value of voting in a swing state or a swing district. The linear mapping smooths out the signal.

P.S. I replied to the above email to share my concern with the creators of this index. We had a cordial email exchange but ultimately they didn’t seem convinced by my argument and so they left the index as it is. Too bad. But, hey, they’re doing the work, it’s their call: if they want to categorize my zip code as having “median” voter impact . . . well, it’s a free country!

“Archaeology can’t give social scientists population or GDP, but here are some things we can measure that might be useful for social science.”

Apropos of our recent discussion on the estimation of historical population sizes, Sean Manning writes:

Some archaeologists have measured house sizes for Gini-coefficient-style studies aside from studying human remains to measure nutrition and rates of illness. I think that was what Michael E. Smith meant when he talked about hypothetical data: “archaeology can’t give social scientists population or GDP, but here are some things we can measure that might be useful for social science.”

I asked Manning where the quote came from, and he replied:

I think I got the idea from this response by Smith to a published paper:

This model of inequality in the Aztec Empire is not based on empirical data. While there is nothing wrong with hypothetical models per se, the paper is phrased as if it presents empirical findings. … There are simply not enough data available to do the kind of analysis presented in this paper. The tweaking of data and methods do not produce results that satisfy me as being reasonable estimates of the level of inequality in the Aztec Empire. Perhaps this is just an epistemological difference between our approaches to science and knowledge. Economists might look at this paper as a fine analysis, whereas archaeologists and historians will probably look at it as a study based on hypothetical data, and therefore divorced from the Aztec reality that we study.

Smith has a book that talks about the archaeology of inequality in Aztec Mexico: Timothy A. Kohler and Michael E. Smith, editors, Ten Thousand Years of Inequality: The Archaeology of Wealth Differences (University of Arizona Press, 2019).

Often in social science there is tension between what we can measure and what we would like to know.

Survey Statistics: toy example for energy balancing weights

Last week we talked about The Big Changes Coming to the Times/Siena Poll:

  1. New weighting variable: support score = E(2024 vote | other X variables).
  2. New weighting method: energy balancing (Huling & Mak, 2024)

Ben Schneider helpfully blogged about energy balancing as well:

Raking and similar calibration methods are based on balancing means or totals for specific variables…The energy balancing method does something different: it calibrates based on an entire multivariate distribution, as measured by an empirical cumulative distribution function (ECDF).

Jared Huling (of Huling & Mak, 2024) helpfully answered questions in the comments. I’m still puzzling over how energy balancing handles empty cells (unsampled regions of the joint covariate space). I need a toy example.

Consider 2 binary variables, so 4 population cells, with known population shares:

       k=0    k=1    total
j=0    .4     .2     .6
j=1    .2     .2     .4
total  .6     .4

Say the sample is missing folks in cell 11:

       k=0    k=1    total
j=0    .5     .3     .8
j=1    .2     0      .2
total  .7     .3

Consider 4 methods:

1. Classical Poststratification: not defined because of division by 0.

2. Raking: match only the margins. Correct when Y | X1, X2 is additive.

       k=0    k=1    total
j=0    .2     .4     .6
j=1    .4     0      .4
total  .6     .4

3. Energy balancing: minimize the Energy-Distance(F_w, F_pop) between the weighted sample distribution of X1, X2 and the population distribution. Correct when Y | X1, X2 is such that nearby cells have similar means.

Say X1 = young/old, X2 = man/woman, Y = percent Democrats, and no old women are sampled.

Raking is correct when additivity holds: old women = young women + (old men − young men)

Energy balancing is correct approximately when: old women = (old men + young women)/2 ?

library(WeightIt)

pop  <- data.frame(X1 = rep(c(0, 0, 1, 1), c(40, 20, 20, 20)),
                   X2 = rep(c(0, 1, 0, 1), c(40, 20, 20, 20)))

samp <- data.frame(X1 = rep(c(0, 0, 1), c(50, 30, 20)),
                   X2 = rep(c(0, 1, 0), c(50, 30, 20)))

dat <- rbind(cbind(pop,  A = 1),
             cbind(samp, A = 0))

W <- weightit(A ~ X1 + X2, data = dat, method = "energy",
              estimand = "ATT", focal = "1",
              dist.mat = as.matrix(dist(dat[, c("X1", "X2")])))

w <- W$weights[dat$A == 0]
tapply(w, interaction(samp$X1, samp$X2), sum) / sum(w)
       k=0    k=1    total
j=0    .381   .309   .69
j=1    .309   0      .309
total  .69    .309

4. MRP: fit a model for Y | X1, X2. The interaction term’s posterior equals its prior, propagating uncertainty around additivity.

Am I understanding this correctly ?

Guess who’s getting the big-money donations in the Maine U.S. Senate race?

Just in time for July 4th, Tom Ferguson, Paul Jorgensen, Matthias Lalisse, and Jie Chen share the above graph and write:

What can one Senate race reveal about the hidden machinery of American politics? In Maine, donor patterns expose how campaign finance can shape party competition, political narratives, and the choices voters are asked to make long before ballots are counted. . . .

Platner is strongly supported by Senator Bernie Sanders and other progressives, while many establishment Democrats dislike him. Major media keep printing articles questioning his character. By contrast, Collins’ somewhat contradictory legislative history attracts less coverage. . . .

Our tabulations of the race show that Collins is much closer to a typical Republican pattern (or, to be fair, those of the Old Guard Democratic leaders [Nancy Pelosi and Chuck Schumer, along with Paul Ryan and Mitch McConnell]) in a key respect: the size profile of her donors. . . .

The Republican Senator from Maine is hugely dependent on very large donors. By contrast, Platner strikingly resembles Sanders: he attracts essentially no big money. Recently the numbers of billionaires supporting the candidates has emerged as an issue. A very few have supported Platner with small sums. Almost a hundred (counting spouses) have made contributions of varying sizes to Collins. The overall configuration is as shown [above] and is perfectly obvious.

They also report:

If you put aside contributions that are below the $200 threshold for disclosure, the percentage of money received from Maine donors differs sharply between the candidates. Senate elections have been nationalized for a long time. Contributions from Maine itself make up approximately 20% of all money for Platner; by contrast, Collins’ rate is slightly under 3%. (Not a misprint.) Her biggest contributors include a Who’s Who of prominent financiers in private equity and hedge funds, including Steve Schwarzman of BlackRock, Ken Griffin of Citadel, along with other well known Republican donors, including Larry Ellison of Oracle.

And they give an example of how this works:

A day after a Super Pac backing her received a $2 million dollar contribution from a private equity magnate who, according to press reports, stood to gain munificently from President Trump’s One Big Beautiful Bill, [Collins] provided a crucial vote to spring the bill out of committee. Then she loudly voted against it on the floor.

Another way of looking at this is to ask, why a person living outside of Maine give $100,000+ to Susan Collins? Roughly speaking, the following conditions are needed:
1. The donor has to be rich enough to be able to spare $100,000 as loose change.
2. It has to be legally possible to give this amount of money, or the perceived consequences of violating the law have to be minimal.
3. The donor has to consider Republican Party control of the U.S. Senate has to be important enough to be worth spending $100,000 to make a small change in the probability of this happening.
4. It has to be easy to write the check; that is, the donor does not need to get the agreement of many other people to release the money.
5. Any negative political, social, and economic consequences of revealing oneself to be a strong partisan have to be mild, compared to the perceived benefits of making the donation.

And in recent years these five conditions have increasingly been present:
1. There are more and more super-rich people who can spend $100,000 without blinking an eye.
2. The Supreme Court keeps liberalizing campaign finance laws, also the government has become much more encouraging and tolerant of corruption. On the rare occasions where people are prosecuted, they get off, and even on the rare occasions are imprisoned for corruption, they get pardoned.
3. With political polarization, the two parties are further apart than ever, and party-line voting in Congress has become the norm.
4. The money is being given by individuals, or by companies controlled by single individuals. It’s not like the old days, where, if General Motors made a campaign contribution, they’d need the coordination of some board of directors.
5. This last one is the most interesting. A flip side of partisan polarization is that, if you give a lot of money to the Republicans, it will piss off a lot of Democrats, and vice versa. Political independents might not be so happy either. One way out is that it’s becoming easier and easier to skirt the regulations and campaign in secret. Beyond this, I guess these donors have decided that the Republican business sphere is large enough that they can afford to alienate Democrats and independents. And Black Rock, Citadel, and Oracle are not primarily customer-facing businesses.

Survey Statistics: Big Changes in the Times/Siena Poll

Yesterday Nate Cohn wrote about The Big Changes Coming to the Times/Siena Poll, with
more details in their poll of Maine.

Say we want to estimate average Platner support in Maine’s likely electorate, E(Y). But we only have survey respondents, R = 1.

The NYT uses survey weights to weight respondents, E(YW | R = 1). In contrast, some pollsters use MRP, fitting a Multilevel Regression model for Platner support, then applying it to the population, E(E_model(Y | X, R = 1)).

Nate discusses 2 Big Changes to how they construct the weights W.

(The polar bear has not yet hiked in ME, but he is training for it. This above is in TN.)

Big Change 1: Support score

A few weeks ago we saw the NYT started weighting on “synthetic 2024 vote”, which is recalled 2024 vote that is validated with the voter file and imputed if needed.

Now they’re also weighting on support score = E(2024 vote | other X variables). Nate explains the motivation:

While a poll can’t weight on dozens of variables, the support score lets us pile a lot of information into a single measure.

This reminded me of the causal inference context, where D’Amour and Franks (2021) “see especially strong performance for propensity weights computed with respect to the prognostic score”, where the prognostic score is E(Y | X, control). In our survey context, this would be a model for Platner support Y. Instead, the NYT use 2024 vote, perhaps for applicability across multiple outcomes Y ?

Big Change 2: Energy balancing

Beyond adding new weighting variables, they’re also changing how they calculate the weights. Nate notes the challenge of weighting on many variables and interactions with typical sample sizes. So they are turning to the R package WeightIt, which implements the energy balancing method from Huling & Mak (2024):

This article introduces a new weighting method, called energy balancing, which instead aims to balance weighted covariate distributions. By directly targeting distributional imbalance, the proposed weighting strategy can be flexibly utilized in a wide variety of causal analyses without the need for careful model or moment specification.

The energy balancing weights do not use outcome Y, but the paper notes that estimates can be improved with a model for Y.

How do energy balancing weights handle the challenge of jointly weighting on many variables with typical sample sizes “without the need for model specification” ?