Hi all, here to talk about one of my favorite scientific topic: integrity and correction of science.
Here comes some good news for most of us and of humanity. More than 10k scientific papers have been retracted this year. Aside from the researchers who have received these notices of retractions (some of them for multiple papers), and the publishers, this is quite good news I would argue. This comes after a big year on this topic and the topic of finding fraudulent practices (see, for instance, how Guillaume Cabanac easily found papers generated by ChatGPT) and very problematic journals with, for instance, Hindawi journals probably being more problematic than others. Many retractions and reports have focused on duplicated images or use of tortured phrases. New fraudulent practices have also emerged and been found (see for instance our findings about “sneaked references” that some editors/journals have manipulated the metadata of accepted papers to increase citations of specific scholars and journals).
Of course, some like me may always see the glass half empty and I would still argue that probably many more papers should have been retracted and that, as I have lamented many times, the process of correcting the scientific literature is too slow, too opaque, and too bureaucratic while at the same time not protecting, funding, or rewarding the hard-working sleuth behind the work. Most of the sleuthing work takes place in spite of, rather than thanks to, the present publication and editorial system. Often the data or metadata to facilitate investigations is not published or available (e.g., lack of metadata about ethics or lack of metadata about reviewing practices).
Still, I guess it is kind of victory that sleuthing work is taken seriously these days I suppose, and I would like to take the opportunity of this milestone of 10k retracted paper to invite some of you to also participated in Pubpeer discussions. I am sure your input would be quite helpful there.
Happy to read thoughts and comments on the milestone and its importance. I will continue to write (a bit more regularly I hope) here on this topic.
Lonni Besançon
There are two things going on here. One is the slow (and steady??) drip of retractions from what might be considered pukka institutions (research Universities in the US; Canada; Aus/NJ; Western Europe) that are highly represented on Pubpeer.
The second, rather recent, horror show is the expansion of rubbish journals and the very large numbers of AI-generated, or citation-ring, or false peer reviewed or other sham “papers” that is now massively outnumbering the more “traditional” reasons for retractions.
So 8,000 of those 10,000 retractions were from Hindawi journals, and Saudi Arabia, Pakistan, Russia and China seem to be the dominant source of much of this stuff (from the article you referred to: “Among large research-producing nations, Saudi Arabia, Pakistan, Russia and China have the highest retraction rates over the past two decades, a Nature analysis has found.”
Whether this is a serious problem rather than just a headache remains to be seen – most of the garbage is easily identifiable and is simply ignored (apart from by the individuals that benefit). I expect very little of it appears on Pubpeer which rightly focuses on issues with what are expected to be decent papers (but might turn out not to be).
While I would tend to generally agree with your comment, there are some things to correct I believe, in particular in your last statement.
> Whether this is a serious problem rather than just a headache remains to be seen – most of the garbage is easily identifiable and is simply ignored (apart from by the individuals that benefit). I expect very little of it appears on Pubpeer which rightly focuses on issues with what are expected to be decent papers (but might turn out not to be).
This is not quite so accurate actually. On multiple parts that I’ll try to explain below.
First “the garbage is easily identifiable”: the badly done one is. The fake clinical trials with fake data for instance are a lot harder to find and require a lot of effort. Detecting clever use of ChatGPT is also a lot harder and almost impossible, and this is taking quite a lot of time from a lot of individuals.
“I expect very little of it appears on Pubpeer” –> An overwhelming part of it appears on Pubpeer, especially when it comes to tortured phrases. A quick search for “tortured phrases” yields 5112 results on Pubpeer actually.
“is simply ignored” –> Again this is probably not so much the case. This is a pervasive problem and it impacts funding decisions on a probably-not-so-minor scale given how widespread it is and how idiotic research assessment is.
Ok that’s interesting – I haven’t perused Pubpeer in detail recently and had sort of assumed that junk papers weren’t much represented there, though to be honest, I’m not sure why I would have assumed that other than my focus on what are the much more interesting problematic papers discussed there.
And yes, you’re right that in some countries decisions around appointments and funding are likely to be adversely impacted by sham publications. It has to be said that the onus should also be on funding and faculty/postdoc recruiters to recognise sham papers on grant applications and CV’s since otherwise they are degrading their own academic systems.
I do agree that the most interesting papers are definitely not the obvious ones about fraudulent practices, although some obvious cases can also be interesting while obvious (see another post of mine about ethical issues in 400+ papers from the same team statmodeling.stat.columbia.edu/2023/08/23/editorial-processes-and-ethics-approval-case-study-of-248-studies-with-the-same-ethics-approval-number/)
“sham papers on grant applications and CV’s since otherwise they are degrading their own academic systems.”
I completely agree, but this is something very hard to scale up when you get hundreds of applications right?
Lonni, I guess you’re responding to this news article in Nature, right? https://www.nature.com/articles/d41586-023-03974-8
Just useful for those of us looking for the source of the 10,000 retractions statement.
You are totally correct, I somehow used the blog post as a reddit post and forgot to link. Thanks for the pointers :)
It is now corrected :)