Crap papers with crude political agendas published in scientific journals: A push-pull problem

Posted on by

From time to time we get pointed to low-quality research papers pushing political agendas that get published in legitimate journals and sometimes get publicity in the news media. A couple examples are a paper from 2016 purportedly estimating the effects of gun control published in the Lancet—this one was so bad that, when it hit the press, leading gun control advocates dismissed it—and a terrible paper from 2022 attempting to demonstrate widespread election fraud, published in Public Choice—this one was rebutted by serious scholars even before it appeared in the journal.

Here I’m distinguishing crap papers with crude political agendas (gun control is bad, Trump is a victim) from crap papers pushing dubious theories (Out of Africa, ESP, etc etc), crap medical claims (Surgisphere), crap causal inference (all those regression discontinuity examples), interesting but imperfect papers whose results are overstated or misinterpreted (as here), or papers that might have seemed reasonable on first glance but disintegrated under scrutiny (the notorious Iraq survey, the gremlins study on the effects of climate change, Nudge, etc).

There is some overlap here: the Iraq paper aligned with a left-wing, anti-U.S. foreign policy agenda, the Tol paper aligned with a right-wing, climate-change denialist agenda, and the nudge papers aligned with a technocratic, listen-to-us-we-know-what’s-good-for-you-agenda—but fundamentally I’d say that these were applied science research projects that just happened to be of low quality, unlike, say, the gun control and election fraud papers which seem more to be to be political exercises that took the form of research papers.

How do these papers get published? I’ve thought of this is a combination of two factors: (1) politicized journal editors, and (2) general acceptance of low-quality research.

But now it occurs to me that there’s a third factor going on.

I thought of this after Kevin Lewis made me aware of an article in the journal Demography making poorly substantiated claims with a heavy political bias.

Are the editors of Demography left-wing political activists? I expect so, at least whoever was handling that particular article; I say this because I think an editor without a strong bias would’ve been likely to notice the problems with the article. Similarly, I can’t say anything about the editorial board of Public Choice, but I expect the particular editors who handled that election-denial paper are right-wing political activists, because otherwise it would’ve taken them so little effort to see how bad that paper was.

But, but . . . I don’t think the editors of those journals are out there soliciting political hackwork! They have political views, sure, but I think their goal as editors is to publish the best papers in their field.

To say it another way: I think the editors’ political biases come into play when they’re evaluating the articles that are sent to them. But I think the driving factor is what papers get sent to them. Which in turn is influenced by what got published and publicized in the past.

Here’s how I see it:

Step 1: Journals establish a track record of publish papers of low quality that make dramatic claims. Examples include the aforementioned Out of Africa and ESP papers, the ovulation-and-voting study, the governors-live-longer study, the collected works of Brian Wansink and Richard Tol, a zillion things published in Lancet, and various other examples we’ve discussed in this space over the years. As discussed many times in this space, I have no reason to believe the authors of these papers are insincere in their claims; they just happen to be doing bad work, making claims that go far beyond what their data can tell us.

Step 2: There are some researchers out there with strong political agendas and who are either willing to do bad work in support of those agendas, or are so committed to these agendas that they don’t realize what they’re doing.

Step 3: Having seen the publication and publicity attached to studies of the root-around-in-the-data-until-you-find-something-that-allows-you-to-tell-a-good-story-and-then-play-the-statistical-significance-and-robustness-check-game, you do some of your own. This is science, right? And if anyone doubts you, they’re just haters who can’t handle the truth, right?

Step 4: Lots and lots of papers from Step 3 get submitted to legitimate journals. Most of them get rejected—most journal submissions of all sorts get rejected—but some of them get just the right editors and reviewers to make it through the review process (PNAS, anyone?), and the papers that do get rejected get resubmitted to other journals and, with enough persistence on the authors’ part, eventually get published. Some of the crap papers with crude political agendas get published because the journal editors have a political affinity; other times the papers just slip through without anyone really noticing what’s going on (I think that’s what happened in this notorious case; the journal editors were just asleep on the watch); other times the editors may have a suspicion that the paper in question is no good, but they don’t want to being in the position of censoring anything.

Non-censorship is an excellent principle. Just don’t forget that if your policy is to publish some subset of the articles that are submitted to your journal, then it’s hard to avoid being influenced by the mix of the submissions. To some extent that’s desirable—an academic journal should reflect what’s currently being done in its subfield—but it also means that if a flood of poop is being sent your way, some of it will evade your filtering system.

I’m offering no solutions here. The point of this post is just to suggest that one reason for the publication of political hackwork in seemingly legitimate journals is that lots and lots of political hackwork is being produced and submitted. It’s push as well as pull.

24 thoughts on “Crap papers with crude political agendas published in scientific journals: A push-pull problem”

  4. Could you expand on “the Tol paper aligned with a right-wing, climate-change denialist agenda”? I mean, yes, the paper is crap, but I don’t think Tol ever denied climate change in any form, and even the crap paper justifies a carbon price. What do you mean by that comment or “align”?

    Reply ↓

    • Martin:

      I’d summarize Tol’s basic claim as that he’s saying global warming is not such a big deal. That’s a position that one can take; the problem is that he supports his position with bad statistics (notably, that “gremlins” paper that we discussed a few times on this blog). I agree that Tol does not deny climate change. It seems to me that his work aligns with a climate-change denialist agenda in that it leads to the same policy recommendation, which is to do very little or nothing.

      That said, for the above post, all that matters is that Tol seems to be pushing some sort of strong political agenda with that paper, and with his work in general.

      Reply ↓

      • Thanks for the clarification. I agree that Tol downplays the effects of climate change. But I disagree with the climate denier alignment: There is a difference between people who downplay impacts and risks and those who deny the existence of impacts and risks. For better or worse, the former can be counted on to support climate policies (Tol, for example, has consistently argued for a carbon price, albeit a low one, at least initially), while the latter are a lost cause. Or, more coarsely: The latter will go full MAGA, the former will not (yes, I checked for Tol).

        I agree with Tol on one point, namely the following: If we want to take climate policy measures, we need broad social agreement on their scope, because they have to be permanent/long-lasting and survive political changes. If we can’t even see that Tol is on team climate action, I’m afraid we won’t get very far with that.

        None of this is meant to defend this paper or his output and behavior more broadly..

        Reply ↓

        • Martin –

          You say:

          I agree with Tol on one point, namely the following: If we want to take climate policy measures, we need broad social agreement on their scope, because they have to be permanent/long-lasting and survive political changes.

          A problem there is that Tol is an obnoxious and toxic participant in the climate wars, and attacks climate scientists as being the cause for a lack of progress on mitigation policies.

          He’s entitled to his perspective, but I’d say his actions aren’t consistent with a goal of creating broad social agreement.

        • Thanks for the link, never seen it. Not your job, but I am really having difficulty with the context of this estimate: Could you point me to where it comes from and what exactly was calculated under which assumptions? A negative SCC is rather obviously at variance with all results he published coming out from either FUND or his literature surveys (crappy as they might be). It also flies in the face of the other DeSmog page linked by Andrew below that is basically and endless repetition of “it’s a small problem, just put a tax on it”, deformed by obnoxious behaviour.

      • Martin, John:

        Regarding Tol’s positions on climate change and its effects, I found this page which seems to be from a few years ago. It seems that Tol has taken a range of positions on the topic, with the most consistent aspect being a contrarian stance.

        I’m reminded a bit of the edgelord attitude taken by the Freakonomics team, who can’t seem to bring themselves to take a hardline climate change denialist stance but who still seem to feel culturally compatible in some way with the deniers, leading them to oscillate between positions.

        Reply ↓

        • I’m not sure I completely agree with that: I agree that he has a tendency to be very edgy and obnoxious and even contrarian when it comes to specific regulatory policies. But on the underlying issue, the whole page confirms my overall impression that he supports a (small) price on carbon. Maybe I have missed a vibes shift of what climate change denialism means, but IMO that’s not it. (Though note John’s link above that he calculated a negative cost in some context, which I wasn’t aware of, but which I also don’t understand).

        • Anon –

          . Maybe I have missed a vibes shift of what climate change denialism means,…

          I’d say it’s more that the term “climate change denialism” doesn’t really mean anything. Or maybe more that it means many things and so has no clear or consistent meaning.

          I’d say probably Tol fits into the category of “lukewarmer.” Although actually that term can also mean many different things to different people, of course. Like “denier” or “alarmist,” “lukewarmer” functions more as an empty label than anything else.

          The labels then function as pejoratives, which is their real purpose. For the most part, imo, “lukewarmer” mostly functions as a way for some folks to be able to hate on those they call “alarmists,” while distancing themselves from the pejorative label of “denier.”

          What then is interesting (to me) is how many “lukewarmers” like Tol hate on “alarmists” even as they cozy up to “deniers” – basically under the logic of “the enemy of my enemy is my friend.”

  5. There is a culture of mendacity in sociology. To get a left-socialist perspective on the importance of facts, truth, academic and research integrity (and the complexities of moral judgment and inherent prejudices and biases), there is no better resource than Norm Finkelstein’s “I’ll Burn That Bridge When I Get To It: Heretical Thoughts on Identify Politics, Cancel Culture, and Academic Freedom”. (2023).

    The dishonesty is prevalent at the highest levels of the discipline. For example, the ASA President Joya Misra wrote articles on retail store credit cards that are just one falsehood after another. I have been a part-time retail store worker since 2018 and have deep knowledge and experience of this topic, gained from experience at my store and several other credible sources. I have a strong interest in this, mostly from a moral and business ethics perspective.

    Even the discipline titans are untrustworthy. When I read Matthew Desmond’s books, I had an unsettling feeling that they are not reliable. Fortunately, over time bigger brains than me began to elucidate. I refer to the brilliant financial statistics guy Aaron Brown’s eviscerating debunking of Desmond in Reason Magazine. Next, an exceedingly distinguished judge in Milwaukee and lifetime resident pointed out that Desmond didn’t really understand the law nor the legal system, although he seemed to have considered himself to be an expert. Finally, the distinguished sociologist and ethnicity expert, Arthur Sakamoto, concluded that Desmond is ‘scientifically dishonest’ and provided rich evidence. But as long as ‘progressive’ activists, who comprise 8 percent of the population, continue to dominate the social sciences and humanities (as well as many/most institutions), Desmond will continue to be feted and honored – through his speaking agency of course

    Reply ↓

    • Joey:

      Can you supply links to the articles on retail store credit cards and explain where they are false? I’m not saying I agree or disagree with you or Misra on this, it’s just something I’d never heard about.

      Reply ↓

      • Here are a couple.

        https://theconversation.com/store-credit-cards-generate-corporate-profits-and-disgruntled-workers-179818
        https://marxistsociology.org/2022/04/walking-mannequins/

        And these are from a scholar who is concerned about politicizing education”

        https://www.tampabay.com/opinion/2024/02/01/removing-sociology-florida-core-course-demonstrates-danger-politicizing-education/
        https://theconversation.com/whats-sociology-a-sociologist-explains-why-floridas-college-students-should-get-the-chance-to-learn-how-social-forces-affect-everyones-lives-222365

        I explain below where they are false. I re-read her article from the Conversation, and found 3 additional dubious/incorrect/outrageous claims in the first 16 lines of the article.

        Also, my experience and knowledge of credit cards and credit card solicitation also comes from: (1) experience with data science vendors, including IBM and SAS Institutes Financial Services Units/Teams; (2) dinner table conversations with my father who is a retired (and widely respected) bankruptcy judge.

        Ironically, convincing evidence against the value of sociology is found in articles written by the ASA President Joya Misra. Her article, “Store Credit Cards Generate Corporate Profits and Disgruntled Workers” is a Marxist screed about exploitative companies forcing cashiers to aggressively and deceptively push their store credit cards on clueless vulnerable customers.

        Misra’s article “Walking Mannequins: How Race and Gender Inequalities Shape Retail Clothing Work”, published in “Marxist Sociology”, shares “sociological findings” about harmful and discriminatory ‘racialized beauty hierarchies that are built into the 21st Century services workplace’.

        These two articles contain one false assertion after another. I can attest to this because I am a store worker who has offered the card to thousands of customers.

        Some of her ‘findings’ based on sociology’s ‘objective scientific pursuit of knowledge’ include:

        Sociological Finding: Store workers aggressively pitch credit cards to young people who don’t realize that cards hurt your credit score. “They are 18 years old and a credit card sounds awesome”. Reality: If it happened it is very rare. Young customers are wary of cards. Few apply.

        Sociological Finding: Customers are unaware of the high interest on the store credit cards. Reality: Most know and pay their balances in full each month thereby avoiding the high interest and fees. The interest rate is in the pamphlet, but if a customer doesn’t know much about cards, they may not look into it.

        Sociological Finding: Stores discriminate against Black and Hispanic card applicants. Reality: Cashiers know this is illegal and against policy. Also there is no reason to do this. Cashiers are incented on the total number of card applications opened.

        Sociological Finding: Workers cannot refuse to tell customers about the card. Reality: This is the policy at some stores but enforcement is at the discretion of management who understand that some cashiers are not comfortable asking for cards.

        Sociological Finding: Management uses ‘electronic surveillance’ to see how many cards each worker and store opens. Reality: Most businesses and NPOs use enterprise software to track performance. They are not watching all the time.

        Sociological Finding: At one retailer, a supervisor was required to sell 2.5 cards for every 10 transactions. Reality: Completely made-up. No one achieves that.

        Sociological Finding: Cashiers who perform above expectations “may get a gift card, a bonus of $1-$5 or a pack of gum”. Reality: Every store is different. Cashiers can receive $20 gift cards, $10 for every card opened, or gifts. High performing cashiers are fast-tracked for promotion. Their success also opens up job opportunities such as banking or sales.

        Sociological Finding: Cashiers who don’t sell enough credit cards “may find themselves off the work schedule and without a job”. Reality: They may be assigned to non-cashier roles. It is unlikely they will be terminated or their hours significantly reduced.

        Sociological Finding: Store managers rarely admit to workers that credit cards are profitable. Instead they claim they are ‘for brand awareness’. Reality: Wrong. This may be one person’s experience. Managers themselves don’t know the retailer’s business model around cards.

        Sociological Finding: Store workers believe that credit cards are “the worst things ever”, “morally not what’s best for customers” and “leads them to financial ruin”. Reality: Wild exaggeration. It is partly true to a limited extent because there is pressure put on cashiers, and in turn on customers.

        Reply ↓

        • “Sociological Finding: Customers are unaware of the high interest on the store credit cards. Reality: Most know and pay their balances in full each month thereby avoiding the high interest and fees. The interest rate is in the pamphlet, but if a customer doesn’t know much about cards, they may not look into it.”

          That one caught my eye. Most data seems to be aggregate amounts, but from time to time there are reports concerning the fraction of credit card holders who carry a balance (subject to interest), for example:

          https://www.bankrate.com/credit-cards/news/credit-card-debt-survey/#number-with-credit-card-debt-at-4-year-high

          You say “most” so I guess you are correct, since the Gen Z credit card holders proportion is “only” 42%. It is around 50% for all age groups, so “most” is not really correct in total. In any case, your statement here seems overly conclusive, as is your statement about “Sociological Finding.” I’d like a more nuanced and accurate portrayal of both what the sociological research says and what the actual data tells us. I’ve been astounded for many years at how many people carry balances on their credit cards, despite the high interest rates. And my perception is that many people do not understand the way the interest on those balances works, except that they keep finding that line item on their monthly statement.

      • At work now. Just spoke with my Operations Manager who has extensivr store leadership experience across retailers and geographies. She said that Misra’s contention that stores require supervisors to open a card for one out of four transactions is pure BS. She said that the very top cashiers in stores attain 3 percent, i.e. one card opened for every 100 transactions.

        Reply ↓

        • The part about “sell 2.5 credit cards for every 10 transactions at the cash register” in that article may actually mean “for every 100 transactions.”

          That would bring the number in line with the percentage you mention and comparable to the sentence following that one: “[managers elsewhere] also expected cashiers […] to sell two cards per shift.”

        • Here is the response from the author

          Dear Joe,

          Thanks for your email. I’ve checked the transcripts for this worker, and she did say 2.5 for every 10 transactions; she also said 10 for 10 on getting them to sign up for the store rewards program. Tara may have misstated, but other workers also noted very high expectations for credit card sales.

          Thanks, Joya

          Joya Misra (pronouns: she/her/hers)
          https://joyamisra.youcanbook.me/
          Distinguished Professor
          Department of Sociology & School of Public Policy
          President, American Sociological Association

  6. I just want to note that the “Out of Africa” hypothesis of biologists is *not* the hypothesis that there is an optimal level of genetic diversity for economic growth, which was the claim of the two economists discussed in Andrew’s 2013 post. The biological “Out of Africa” claim is that anatomically modern Homo sapiens arose in Africa and then began to spread to the rest of the world around 10^5 years ago. The original idea was that earlier Homo were replaced, but it is now know that substantial interbreeding occurred between anatomically modern humans and Eurasian Neanderthals and Denisovans. There has been debate about the exact timing of migrations out of Africa, whether there were distinct phases, and the nature of back-migrations to Africa. “Out of Africa” is not a “dubious theory” pushed by a “crap paper”, but a reconstruction of part of the history of our species. (An estimate rather than an hypothesis, to be precise, but that’s a different question.) It is an area of active research and discussion based on generic and fossil data.

    Andrew and, I’m sure, most readers know this, but I mention it to insure that more casual readers do not encounter the biological “Out of Africa” idea, and mistake it for the dubious economic idea that Andrew has discussed.

    Reply ↓

    • Thanks Gregory — not being familiar with the paper I was taken aback by the mention of “Out of Africa” here. The original post that Andrew links has enough in it to suggest that it is a distinctly different meaning, though the context of migration and population diversity somewhat muddied the waters.

      Reply ↓

Leave a Reply

Your email address will not be published. Required fields are marked *