Your tax dollars at work (junk social science edition)

A couple people pointed me to this article:

With this sort of work, I always wonder whether people who do this sort of thing really believe what they’re doing, or if they’re purposely complexifying things, the way that in chess you might try to make the board position more complex if you’re down a couple pieces already. This is a variant of the usual knave-or-fool question.

I’m guessing it’s a mix of the two: Rosh is coming into this with the belief that Trump is the legitimate victor (even if he didn’t receive more votes, the mail votes aren’t really legitimate, or early votes shouldn’t count, or people only voted for Biden because of the lying press, or Biden is really Harris’s puppet so the election doesn’t count, or Trump won more counties, or if the Hunter Biden scandal hadn’t been suppressed Biden would’ve lost, or etc etc), so any argument that the election is illegitimate is fair game. Once he gets to that point, he can use whatever statistical tricks he can think of to muddy the waters. If this were a serious study he would look for suspicious outcomes in other locations, not just the ones that are being pushed by the Trump campaign. The scandal is that this guy is on the government payroll. If my tax dollars are going to be spent on make-work projects, can’t they do something more interesting like hire people to dig up holes and fill them in again?

This article features the kind of bad statistical reasoning you’d expect to see in the Journal of Theoretical Biology or the Journal of Personality and Social Psychology or Psychological Science or (of course) the Proceedings of the National Academy of Sciences; it’s disappointing to see it being used as part of an active political debate.

The article was also promoted by the president. Disgraceful. We laugh about institutions such as Cornell and the University of California being complicit in junk science, but when it’s the U.S. government . . . this is Stalinesque. I guess that from their perspective, we’re in a war, and so this is all legitimate disinformation tactics. But that’s what the Stalinists said too, basically: everything is us vs. them and so any tactics are acceptable.

The sad thing is that the author of the above document used to teach economics in college, and one of his students wrote, “I have to say that he was the best professor that I ever had.” I hope that wasn’t a statistics class he was teaching!

33 thoughts on “Your tax dollars at work (junk social science edition)

  1. The President’s greatest responsibility is the security of the United States. Someone who let an election get stolen doesn’t deserve the job. If the Democrats can put one over on him, then the Russians, Chinese, Saudis, or heck even the French will play him for a fool all day long. “I got cheated” is the whining pout of a loser.

    • When you have the CIA, FBI, DoJ corrupted to the point of conspiring a soft coup and you thwart them. I would say he did pretty good. That’s why misguided crowds thought it their patriotic duty to rush the Capitol Building on Jan 6. It was not planned. It was not armed and it was not an insurrection. That is the lying, cheating, corrupt winner’s story.

      • Ron:

        I can’t exactly blame you for thinking this, since Fox News has been pushing this story 24/7 for a few months now, but, no, if you lose an election by 7 million votes and there is zero serious evidence of problems with the vote counts, then, no, it’s not “pretty good” to incite a riot and to attempt to invalidate the election. It’s dangerous and anti-democratic, an attempt to take one person’s desire and overrule a national vote.

      • Ron –

        > That’s why misguided crowds thought it their patriotic duty to rush the Capitol Building on Jan 6.

        I have to say, that is a spectacular comment.

        “Misguided” is a fantastic euphemism for…delusional? fanatical? extremist?

        “Thought it was their patriotic duty” could be used to rationalize practically anything – McVeigh bombing in OKC, lynchings, assassinations (say Lincoln), and not in the least, every single thing that any members of the CIA, FBI, and DOJ did before, during, and after the Trump administration.

        What else would have been acceptable for them to do to try to disenfranchise the majority of American voters, because they were deluded and thought the election was stolen from them?

  2. Andrew wrote:
    The scandal is that this guy is on the government payroll.

    The Wikipedia article on Rosh says that (s)he resigned the position with DoJ a few days before Biden was sworn in.

    I know an economist who worked with Rosh. He had nothing positive to say about Rosh whatsoever. The person I know can be negative speaking of others, but these comments were a new level of negativity.

    It’s fascinating to look at the success of “academics” such as Bellesiles, Rosh, and Wansink and to try to understand the steps and motivation underlying that success.

    Bob76

    • I might be mistaken, but I think Lott may be (in)famous for his extensive work on guns – ostensibly showing that gun control either doesn’t work or actually increases gun violence. And, who is “Rosh?”

    • Bob:

      Bellesiles, Rosh, and Wansink succeeded with some combination of lack of scruple (very helpful if you want to make big claims with no data) and telling people what they wanted to hear. Perhaps it’s worth noting, though, that none of them was very successful as an academic. I don’t think Lott was ever tenured anywhere, and Bellesiles and Wansink got fired.

      There’s one thing, though . . . before they were disgraced, these guys each had their niche, and we can see that with other academics who’ve succeeded with fishy research. There was the disgraced primatologist at Harvard, and he had his shtick, the sleep scientist at the University of California with his pitch, etc. Sometimes these guys can stay afloat for a long time because their work is judged by their friends (“peer review”). It was harder for Lott and Bellesiles because they worked on controversial topics. Lott could well be on Fox News until the end of time, but his work never got a pass in the way that Wansink’s did for many years.

  3. Sort of sounds the same as the anti vaccine crowd.

    Cherry pick facts here and there and muddy the waters. Ignore the holistic assesments and select articles that match their viewpoint.

    The crux of the matter is that it takes an order of magnitude lesser effort to muddy the waters than to do a thoughtful, holistic assessment.

    OTOH, cherry picked factoids are somehow far more convincing to the lay audience than a large, nuanced assesment.

  4. “They don’t have to prove the lie, they just have to perpetuate it.” Can’t remember who said this or where I heard it but it really stuck with me.

  5. As I was skimming the first couple of pages of Lott’s article I was reminded that a few months back I wondered how US voting practices compare w/ other countries. Are there databases or survey articles that this blog’s readership could recommend for this topic? Over the past 15 months I have found it fascinating to read about cross-country comparisons of C19 policies and outcomes, and I wonder if we find similar differences in voting practices.

    • I wondered how US voting practices compare w/ other countries.

      I am no expert but I think it is fair to say that no electoral system in the world compares with the US system. You might want to have a quick glance at the Australian or Canadian Federal elections websites for a taste. I suspect that state/provincial commissions may differ slightly in details but are substantially the same.

      If I have this correctly the Australian system uses the ranked choice method while Canada remains with the First-Past-the-Post method.

  6. of course cherry-picking a bad study on the vote-fraud issue does not settle the primary question, even slightly.

    serious audits of the massive 2020 election are difficult and not even attempted in most of the 3,000+ counties in US.

    one exception is Maricopa County Arizona (a key state in 2020 election).
    Arizona Senate ordered a deep independent audit. That audit is still in progress, but has alreadt found huge problems with untraceable votes, missing vote records, deliberate deletion of entire hard-drives containing primary official vote records.
    The Maricopa Board of Elections is fighting the audit tooth & nail, with repeated desperate attempts to have the audit canceled entirely.

    • Yuma4:

      I’m not sure what you mean by “cherry-picking a bad study.” This is just a paper that was sent to me in the email. Election fraud claims that were just as bad were promoted by leading politicians such as Ted Cruz. Lots of people still refuse to admit that Biden won 81 million votes. No cherry picking is needed; for a few months we were flooded with this crap. Justin Grimmer, Andrew Eggers, and Haritz Garro wrote a paper discussing some of these; see here for discussion.

        • On the off chance that some others might might miss my attempt at satire, although egg foo young and chow mein are common items in Chinese restaurants in the U.S., they are totally unknown in China.

        • Andrew: You might be right in that we have become more internationally Asian, and thus, current Chinese restaurant owners might not be exclusively from Canton. Consequently, chow mein and egg foo young may have disappeared from the menus. I do not know if the Maricopa County re-counters have that sophisticated a cuisine data base.

          Wikipedia seems to suggest that chow mein and egg foo young linger on in the U.S., albeit not necessarily in NYC.

          Chow mein (/ˈtʃaʊ ˈmeɪn/ and /ˈtʃaʊ ˈmiːn/, simplified Chinese: 炒面; traditional Chinese: 炒麵; Pinyin: chǎomiàn) are Chinese stir-fried noodles with vegetables and sometimes meat or tofu; the name is a romanization of the Taishanese chāu-mèn. The dish is popular throughout the Chinese diaspora and appears on the menus of most Chinese restaurants abroad.[1] It is particularly popular in India,[2] Nepal,[3] the UK,[4] and the US.

          Egg foo young (Chinese: 芙蓉蛋; pinyin: fúróngdàn; Jyutping: fu4 ‘jung4 daan6*2, also spelled egg fooyung, egg foo yong, egg foo yung, or egg fu yung) is an omelette dish found in Chinese Indonesian, British Chinese,[1] and Chinese American cuisine.[2][3] The name comes from the Cantonese language.

    • Yuma4 –

      I’m always curious when someone writes a comment like yours and then fails to reply to a response like Andrews. I’m wondering if maybe you could explain why you didn’t reply here?

      • Joshua:

        Also a little bit of history is helpful here. Lots of people naively think that when there’s smoke there’s fire, but it’s harder to think that if you’re familiar with historical lies such as Dreyfus, Protocols, etc. Sometimes the disinformation comes when creating the lie in the first place; other times it comes from spreading confusion. Or, to put it another way, when there’s smoke there’s fire, but sometimes the first is not coming from the purported villains of the story but rather from the people who are promoting it. As discussed somewhere else in this thread, I have no real sense of when Rosh is promoting these theories, whether he knows they’re nonsense or if he’s genuinely fooled by it, but maybe this doesn’t matter because, as a true believer, he might not really think about this in terms of any usual rules of evidence.

        • Andrew –

          > As discussed somewhere else in this thread, I have no real sense of when Rosh is promoting these theories, whether he knows they’re nonsense or if he’s genuinely fooled by it, but maybe this doesn’t matter because, as a true believer, he might not really think about this in terms of any usual rules of evidence.

          I baapwn to think as a general principle, it’s a safer bet to believe that if someone proudly promotes an idea to a widespread audience, they are mostly to believe in its veracity. It’s a pretty unusual person would would take on the reputation al risk to promote something the know to be probably false.

          I also think it’s unusual for someone like Roah to operate outside the usual rules of evidence.

          The challenge, imo, is to figure out how someone who a idea by the same rules of evidence I try to abide by, comes to a belief that seems obviously false to me – through cognitive empathy or perspective taking

  7. Thank you, Andrew, for continuing to fight the good fight. American Democracy unnervingly remains vulnerable months after Biden’s clear victory and shining bright disinfecting light on Trumpist mendacity is vitally important.

  8. Could you include a couple of constructive sentences with these sorts of posts summarizing what exactly is wrong with the analysis? I don’t need much detail but would like to learn what to look for in the actual study, not only paying attention to who is on whose payroll and what reputation they may or may not have.

      • Thanks for pointing to the article!

        I thought I’d give a quick follow up to explain what went wrong.

        In sections 3.2 and 3.3 of this paper we explain the errors (https://www.dropbox.com/s/zyd5gt177a9fdm0/pnas.pdf?dl=0 ).

        Section 3.2 shows that the analysis of counties in GA and PA was wrong in Lott’s initial paper because his specification depended on an arbitrary ordering of the data. We corrected that to remove the dependence on arbitrary ordering and show that using Lott’s data and a reasonable specification there is no evidence of a systematic Biden over performance. (FWIW John Lott agrees that he initially made a mistake, but he continues to be confused about the specifications. He thinks that dropping an intercept from his original specification is somehow different than our proposed solution, even though we proved the equivalence for him https://twitter.com/justingrimmer/status/1349146054839599104?s=20 ).

        The second analysis in Lott’s paper argues there was oddly high turnout in “suspicious” counties. We show that Lott’s result depends on including an arbitrary subset of states in the analysis and *not* including state-level fixed effects. Once we use state fixed effects there is no turnout difference between suspicious and non-suspicious counties.

        • Justin:

          I suspect that Rosh’s confusion of which you speak is not so much a confusion about the specifics of the analysis but rather a larger confusion about the purpose of empirical investigation. To us, there are empirical facts to uncover; we might have our prior beliefs but we see there being some underlying quantitative truth which may or may not be what we want to hear. But I suspect that an advocate such as Rosh starts with the conclusion—in this case, that Biden did not deserve to win the election—and then just keeps working with the data until he gets what he wants. So if his particular analysis is problematic, no problem, he can keep noodling around until he reaches the desired endpoint. His statistical analysis workflow is a kind of random walk where the desired conclusion is an absorbing state. So really it doesn’t matter what you say; he can always get to where he wants to be. Also, I suspect that, given this is his approach, that he doesn’t fully grasp that other people don’t do statistics that way. If this guess of mine is correct, he might well just be assuming you’re a pure advocate too, which would make it that much harder for him to understand your analysis.

  9. “If my tax dollars are going to be spent on make-work projects, can’t they do something more interesting like hire people to dig up holes and fill them in again?”

    Nice to see a call to increase funding for archaeology on the blog! Hear hear!

Leave a Reply to Joani78 Cancel reply

Your email address will not be published. Required fields are marked *