Chetan Chawla points us to this disturbing news article from Harriet Alexander:
In 2015, molecular oncologist Jennifer Byrne was surprised to discover during a scan of the academic literature that five papers had been written about a gene she had originally identified, but did not find particularly interesting.
“Looking at these papers, I thought they were really similar, they had some mistakes in them and they had some stuff that didn’t make sense at all,” she said. As she dug deeper, it dawned on her that the papers might have been produced by a third-party working for profit. . . .”
The more she investigated, the more clear it became that a cottage industry in academic fraud was infecting the literature. In 2017, she uncovered 48 similarly suspicious papers and brought them to the attention of the journals, resulting in several retractions, but the response from the publishing industry was varied, she said.
“A lot of journals don’t really want to know,” she said. . . .
More recently, she and a French collaborator developed a software tool that identified 712 papers from a total of more than 11,700 which contain wrongly identified sequences that suggest they were produced in a paper mill. . . .
Even if the research was published in low-impact journals, it still had the potential to derail legitimate cancer research, and anybody who tried to build on it would be wasting time and grant money . . . Publishers and researchers have reported an extraordinary proliferation in junk science over the last decade, which has infiltrated even the most esteemed journals. Many bear the hallmarks of having been produced in a paper mill: submitted by authors at Chinese hospitals with similar templates or structures. Paper mills operate several models, including selling data (which may be fake), supplying entire manuscripts or selling authorship slots on manuscripts that have been accepted for publication.
The Sydney Morning Herald has learned of suicides among graduate students in China when they heard that their research might be questioned by authorities. Many universities have made publication a condition of students earning their masters or doctorates, and it is an open secret that the students fudge the data. . . .
In 2017, responding to a fake peer review scandal that resulted in the retraction of 107 papers from a Springer Nature journal, the Chinese government cracked down and created penalties for research fraud. Universities stopped making research output a condition of graduation or the number of articles a condition of promotion. . . . But those familiar with the industry say the publication culture has prevailed because universities still compete for research funding and rankings. . . . The Chinese government’s investigation of the 107 papers found only 11 per cent were produced by paper mills, with the remainder produced in universities. . . .
As Chawla writes, what’s scary is the idea that this Greshaming isn’t just happening in the Freakonomics/Gladwell/NPR/Ted/Psychological Science axis of bogus social science storytelling; it’s also occurring in fields such as cancer research, which we tend to think of as being more serious. OK, not always, but usually, right??
I continue to think that the way forward is to put everything on preprint servers and turn journals into recommender systems. The system would still have to deal with paper mills, but perhaps the problem would he easier to handle through post-publication review.
Work all published in reputable journals with ethics approval but veracity concerns https://journals.lww.com/pain/Abstract/9900/Investigating_the_veracity_of_a_sample_of.70.aspx
Permeating recent publications https://www.bmj.com/content/376/bmj-2021-067718/rr-0
Impacting 100s of guidelines and policy recommendations. We need the open science and evidence synthesis and forensic investigations to detect and correct this until a new way of disseminating science emerges
Yikes.
But the problem stems from the publishing requirement. If you have a publishing requirement for graduate students and junior faculty and you accept students incapable of doing the work, this is the result you have to expect. The market will meet the demand.
Selling slots on publishable papers seems like a much less socially harmful practice than flooding the zone with crap, though.
The recommender system idea seems to me flawed. With the explosion of crap science that would make it to a preprint server, the burden of post publication review will be too much to handle.
Wouldn’t it also inhibit out of the box thinking that goes against the consensus?
Salomon:
I guess the preprint servers would need to be expanded for the recommender system to work. But it could be done gradually:
Suppose you start with a journal that gets 1000 submissions a year and publishes 200. Now the journal changes to be a recommender system. If they get 1000 submissions to recommend—this time the submissions are not manuscripts, they’re links to manuscripts on preprint servers—then this would take up 1000 slots on preprint servers. I don’t think that would overwhelm the system. If enough journals switch to becoming recommender systems, then I guess some improvements might be needed to the system of preprint servers. If it were the journals being overwhelmed, with too many papers submitted, then they could charge a fee for reviewing, as many journals currently do already.
I don’t see why changing journals to recommender systems would inhibit out-of-the-box thinking. It’s the same accept/reject decision, right?
I guessI didn’t understand the recommender system idea. I was imagining some sort of popularity contest in which qualified scientists have votes. My imagination just went to a completely different place.
What you describe is not much different than the current situation no? Wouldn’t university still require papers (read recommendations) from high ranking journals? How would the recommender mechanism change things? Could it be that multiple journals can recommend the same manuscripts?
Such a system, called peer community in (https://peercommunityin.org/), currently exists in the biological sciences. It is fairly new and still developing. It will only work if scientists submit good research to it, which can somehow go against the current incentives based on impact factors… Time will tell.
The rot is deeper. The way we do Science funding is the root cause.
The managers at the granting agencies don’t have skin in the game. Most articles are barely read be anyone other than the authors and people closely associated.
Publication was designed to be a one-to-many medium. What’s the point in publishing an article that you need to pay to publish even if to defray costs. Even a comic strip can sustain itself from reader subscriptions. Most journals cannot survive without institutional subscriptions.
Publication has become a mere ritual of passage producing a product that’s not really self sustainable based on demand.
I can not speak to science funding, but in the social sciences the research funding agencies are very narrow in what type of research gets grants and they are getting increasingly social justice oriented as well. The best way to get grants is doing abstracted empiricist research in demography, public health, inequality, education etc. on minorities or junk like CRT.
“Publication has become a mere ritual of passage producing a product that’s not really self sustainable based on demand.”
This has always been the case.
I almost took you seriously until you started whining about social justice and sure enough the usual dog whistles started blowing.
Anonymous
I can’t figure out what you are saying. Are you complaining about Mark’s citing social justice? It looks to me that Mark is saying that “politically correct” social justice work is becoming necessary to get social science grants or publications and that Mark is calling that “junk.” You appear to be complaining that Mark mentions social justice. So, I must be misreading either you or Mark (or both). Sometimes your (assuming you are the same Anonymous as in other threads) contributions appear to be no more than tirades – often misplaced.
Publishing by preprints followed by post-publication review wouldn’t hurt, but wouldn’t help. A few quick points:
(1) Lots of papers, many of them excellent, many of them awful, are posted on the bioRxiv preprint server. One can comment on these. The *vast* majority have zero comments. I’ve taken the trouble to comment a few times and have never gotten a response.
(2) There are simply far too many papers being published, which contributes to #1. This reminds me of a title that made me laugh when reading it a few days ago: “Exposure to the gut microbiota from cigarette smoke-exposed mice exacerbates cigarette smoke extract-induced inflammation in zebrafish larvae”. I haven’t bothered to read it, despite it being in an area I work in, and I can’t imagine why this study was worth the effort to perform.
(3) Our microbial ecology journal club is, more often than not, disappointed by the articles we read. The paper from a few weeks ago would make an excellent topic for this blog: forking paths, overblown conclusions, inadequate controls/comparisons, writeups in popular media, etc., but also interesting data of the sort that hadn’t been collected before. I thought I should write to the authors with questions, or write a blog post about all this, but (i) this takes time, and I’m already neglecting writing a quiz for my class, working on a grant proposal, etc., and (ii) I’ll undoubtedly make enemies of the authors, for no benefit to myself. How, given this, does one expect relying on post-publication comments to work?
《 “Exposure to the gut microbiota from cigarette smoke-exposed mice exacerbates cigarette smoke extract-induced inflammation in zebrafish larvae” […] I can’t imagine why this study was worth the effort to perform.》
None of you gives a damn about lab animals, huh? Are they simply unfeeling objects to you? Do you kick pregnant dogs, like Descartes? Oh no, are you getting mad because I’m speaking up for my animal friends, but science says they’re just automatons?
Fair criticism.
But if we want to investigate causal effects on biological structures and processes, would you rather do it in humans?
Want to figure out what the toxicity and ld50 etc of new drugs are? Good luck using humans and arguing your point.
What would be your suggestion to address the question of junk science/fake science infiltrating journals? Or the problem of too many papers being published? If not publishing by preprints followed by post-publication review, then what else?