Dorothy Bishop has the story about “a chemistry lab in CNRS-Université Sorbonne Paris Nord”:
More than 20 scientific articles from the lab of one principal investigator have been shown to contain recycled and doctored graphs and electron microscopy images. That is, results from different experiments that should have distinctive results are illustrated by identical figures, with changes made to the axis legends by copying and pasting numbers on top of previous numbers. . . . the problematic data are well-documented in a number of PubPeer comments on the articles (see links in Appendix 1 of this document).
The response by CNRS [Centre National de la Recherche Scientifique] to this case . . . was to request correction rather than retraction of what were described as “shortcomings and errors”, to accept the scientist’s account that there was no intentionality, despite clear evidence of a remarkable amount of manipulation and reuse of figures; a disciplinary sanction of exclusion from duties was imposed for just one month.
I’m not surprised. The sorts of people who will cheat on their research are likely to be the same sorts of people who will instigate lawsuits, start media campaigns, and attack in other ways. These are researchers who’ve already shown a lack of scruple and a willingness to risk their careers; in short, they’re loose cannons, scary people, so it can seem like the safest strategy to not try to upset them too much, not trap them into a corner where they’ll fight like trapped rats. I’m not speaking specifically of this CNRS researcher—I know nothing of the facts of this case beyond what’s reported in Bishop’s post—I’m just speaking to the mindset of the academic administrators who would just like the problem to go away so they can get on with their regular jobs.
But Bishop and her colleagues were annoyed. If even blatant examples of scientific misconduct cannot be handled straightforwardly, what does this say about the academic and scientific process more generally? Is science just a form of social media, where people can make any sort of claim and evidence doesn’t matter?
They write:
So what should happen when fraud is suspected? We propose that there should be a prompt investigation, with all results transparently reported. Where there are serious errors in the scientific record, then the research articles should immediately be retracted, any research funding used for fraudulent research should be returned to the funder, and the person responsible for the fraud should not be allowed to run a research lab or supervise students. The whistleblower should be protected from repercussions.
In practice, this seldom happens. Instead, we typically see, as in this case, prolonged and secret investigations by institutions, journals and/or funders. There is a strong bias to minimize the severity of malpractice, and to recommend that published work be “corrected” rather than retracted.
Bishop and her colleagues continue:
One can see why this happens. First, all of those concerned are reluctant to believe that researchers are dishonest, and are more willing to assume that the concerns have been exaggerated. It is easy to dismiss whistleblowers as deluded, overzealous or jealous of another’s success. Second, there are concerns about reputational risk to an institution if accounts of fraudulent research are publicised. And third, there is a genuine risk of litigation from those who are accused of data manipulation. So in practice, research misconduct tends to be played down.
But:
This failure to act effectively has serious consequences:
1. It gives credibility to fictitious results, slowing down the progress of science by encouraging others to pursue false leads. . . . [and] erroneous data pollutes the databases on which we depend.
2. Where the research has potential for clinical or commercial application, there can be direct damage to patients or businesses.
3. It allows those who are prepared to cheat to compete with other scientists to gain positions of influence, and so perpetuate further misconduct, while damaging the prospects of honest scientists who obtain less striking results.
4. It is particularly destructive when data manipulation involves the Principal Investigator of a lab. . . . CNRS has a mission to support research training: it is hard to see how this can be achieved if trainees are placed in a lab where misconduct occurs.
5. It wastes public money from research grants.
6. It damages public trust in science and trust between scientists.
7. It damages the reputation of the institutions, funders, journals and publishers associated with the fraudulent work.
8. Whistleblowers, who should be praised by their institution for doing the right thing, are often made to feel that they are somehow letting the side down by drawing attention to something unpleasant. . . .
What happened next?
It’s the usual bad stuff. They receive a series of stuffy bureaucratic responses, none of which address any of items 1 through 8 above, let alone the problem of the data which apparently have obviously been faked. Just disgusting.
But I’m not surprised. We’ve seen it many times before:
– The University of California’s unresponsive response when informed of research misconduct by their star sleep expert.
– The American Political Science Association refusing to retract an award given to an author for a book with plagiarized material, or even to retroactively have the award shared with the people whose material was copied without acknowledgment.
– The London Times never acknowledging the blatant and repeated plagiarism by its celebrity chess columnist.
– The American Statistical Association refusing to retract an award given to a professor who plagiarized multiple times, including from wikipedia (in an amusing case where he created negative value by introducing an error into the material he’d copied, so damn lazy that he couldn’t even be bothered to proofread his pasted material).
– Cornell University . . . ok they finally canned the pizzagate dude, but only after emitting some platitudes. Kind of amazing that they actually moved on that one.
– The Association for Psychological Science: this one’s personal for me, as they ran an article that flat-out lied about me and then refused to correct it just because, hey, they didn’t want to.
– Lots and lots of examples of people finding errors or fraud in published papers and journals refusing to run retractions or corrections or even to publish letters pointing out what went wrong.
Anyway, this is one more story.
What gets my goat
What really annoys me in these situations is how the institutions show loyalty to the people who did research misconduct. When researcher X works at or publishes with institution Y, and it turns out that X did something wrong, why does Y so often try to bury the problem and attack the messenger? Y should be mad at X; after all, it’s X who has leveraged the reputation of Y for his personal gain. I’d think that the leaders of Y would be really angry at X, even angrier than people from the outside. But it doesn’t happen that way. The immediate victims of the con would rather act as if the con never happened. Instead, they’re mad at the outsiders who showed them that they were being fooled. I’m sure that Dan Davies would have something to say about all this.
Don’t waste any more resources attempting to distinguish between fraud and incompetence.
Someone else needs to repeat the experiments (or whatever) to see if they get the same results. This is very simple and was already figured out hundreds of years ago. In fact, most people assume that is what scientists are actually doing.
I was not a lab scientist, but my wife was, and I gather from her that repeating laboratory experiments in biology is sometimes not that simple. Consider that people do sabbaticals at other labs to learn how to work with an experimental system developed there. There can be a lot more to the experiments than what gets described in detail in the methods section of a paper.
I was a lab scientist, and the methods sections of the papers are largely worthless.
They are that way because no one is repeating each others work (it is actively discouraged due to “lacking novelty”), so we don’t know the important factors that need to be communicated in the method sections.
The “its so complicated” excuse is just an excuse for doing a bad job. Of course the individual scientist will find it nearly impossible to do a good job because it takes so much longer than doing a crappy job. So they simply can’t compete (Gresham’s law).
It won’t stop until people stop paying researchers for doing a bad job.
Unfortunately the incentives for scientists are generally to arrive at novel findings—replication studies don’t get as much attention, at least in my experience.
It can also be difficult to reproduce some experiments, where complex procedures aren’t completely described (e.g. assumptions about the procedures aren’t articulated) or in observational studies data may not be readily available (e.g. studies of rare diseases).
There’s a spectrum of reproducibility (highest standard, same data + same open-source code provide same results) to replicability (same procedures yield similar results), and different fields accommodate different levels of capability to recreate a previous study’s results.
Part of the problem is also there are few (almost zero) senior scholars with the combination of prestige and personal temperament to rail against such things. Within our discipline, look at Don Green after the LaCour scandal: rather than people rightly rail against the fact he at best committed research fraud by putting his name on an article he didn’t contribute to, he was fêted with a workshop at Stanford on research ethics and transparency (invite-only and non-public, naturally).
And even those who are inclined, like Andrew, often shade into vague references or euphemisms rather than directly calling out the miscreants: I’m fairly keyed into disciplinary stuff, but I’m not aware of what award that went to a plagiarized book he is referring to, or who the plagiarizer is (and it’s not like this is the sort of thing that one can discover via some Googling).
Random:
I’m not shading into vague references or euphemisms! I just find the examples more interesting than the people. Here’s the background to different awards that went to two different plagiarizing social science faculty, one at Rutgers and one at Columbia.
Andrew, for the record, I did not think you weren’t doing it on purpose. It was just offered an example of something I generally observe in academia, which exacerbates the issues your post highlights.
It was the opposite of a criticism: if more senior scholars rejected the wagon-circling of elite academic circles like you, we’d be in a better place.
Sturgeon’s law in combination with Clarke’s law and the Piranha Principle tell me that fraud isn’t actually a big problem for science at all. At least, not every crappy research paper should have a big negative impact, there are way to many crappy research papers!
I’m not convinced by many of the 8 points they make. Does fraud slow down science? Is there any proof for that claim? Actually, after reading many papers from the 40ies and 50ies, I’m convinced that the standards have risen by quite a lot. Nowadays, it’s a widespread practice to cite one’s sources and just making up numbers is generally frowned upon.
On another note: according to some paywalled NYT article that is cited by wikipedia, some study “discovered [that piranhas] are timid fish that schooled for protection from their own predators”. So maybe “Piranha Principle”, although a very intuitive term, is still a misnomer.
Anon:
1. Excellent to see my various laws and principles used in combination!
2. Yes, I’ve heard that piranhas can coexist peacefully. Indeed, in our most recent trip to the zoo, we saw a fish tank full of piranhas swimming around. We took a picture but it somehow got lost on the phone, which was too bad because I was going to use that photo as the basis for a blog post on just this topic.
That reminds me: where have all the cat photos gone?
Howard:
If people send ’em in, I’ll post ’em.
The linked July 2023 review of the Davies book raises an interesting point about trying to get a rough handle on the prevalence of fraud by thinking about one’s personal associates. (I can’t recall reading the July post; maybe I should comment back there, but I prefer to stay in the present timeline.) I’m not an academic but I attended “fancy schools.” I have (at least) one classmate out of ~1600 from undergrad in the 1980a who went on to be a notorious financial fraudster and another from my professional school class of ~170 in the 1990s whose business fraud became known more recently. I know them both personally (meaning they would remember me). And these are just people that got caught! That gives me the impression of a fairly high rate.
Kyle:
Nobody I knew from college became a financial fraudster, but I do have a cousin who was known as the King of Spam and was sent to prison, and I have a friend—ok, I haven’t seen him for a few years now, I guess that would make him more of an acquaintance, who, ok, I don’t know if he was an actual financial fraudster but he did have to pay a hundred-million-dollar fine to the government. I think he’s still doing fine in business, so I guess he had a few million left over after paying the fine, or maybe he got a favorable payment plan. And then I’ve had various academic friends and colleagues who’ve engaged in scholarly and professional misconduct, but I guess that’s so common we don’t even think about it.
Curious, what prompted you to write this piece? Only asking because I had assumed it was the linked Open letter to CNRS, but it seems that was published on March 2023. I’m wondering if there have been more recent instances of research fraud that got you thinking about it again in January.
Alan:
We have a backlog of posts; see discussions of this point here and here. And sometimes posts get bumped multiple times and end up appearing long after I wrote them.
Referring to the last paragraph, I think that in a lot of cases Y is essentially in bed with X, because Y depends on X for part of their prestige, and, moreover, as a local “star”, X is probably also on various boards and committees that mean that they interact with members of Y on a regular basis. So, I imagine that the Y/X relationship is fraught with social awkwardness in addition to everything else.
I had already been working several years full time in Mexico studying the effects of prenatal and postnatal lead exposure on pregnant women and their children. Our group had published our first article in a Mexican journal in Spanish describing the results of the project pilot study. A year later, one day in the late 1980’s the project officer of one our funding sources called and started the conversation by exclaiming that he didn’t know that there were other investigators studying the same thing in Mexico. He immediately faxed me a copy of an English language paper that looked just like what we were doing, but with different authors from a different Mexican institution.
The suspect journal article was identical in text, figures and tables to our original Spanish paper but with an original introductory and an original summary last paragraph and all in English. The translation was excellent! After one of our research assistants familiar with the offending academic institution came back from a couple of days of on site investigation, we called the first author of the plagiary, the associate director of a prestigious Institute within the National Autonomous University of Mexico, to tell him what we discovered. We assured him we would not involve the Mexican judiciary for the plagiary if he would only come over to our laboratory and sign confessions of his plagiary. He arrived the next day, read the confession we wrote and signed the confession and copies. He asked again if we would prosecute him (possible under the then newly revised Mexican copyright law), we assured him no and he left.
The next day we hand delivered the confession and copies of the original and plagiarized paper to the rector of the University, the director of the Institute, the director of the government institution that is the major funder of research in Mexico as well as directing a scholarship program for researchers in Mexico originally intended to retain investigators in Mexico during periods of hyperinflation in the country. We also sent to the editor in chief of the Springer Verlag journal that had published the plagiary by certified mail a signed copy of the confession and the two versions of the paper and requested him to immediately withdraw the plagiarized paper with a note of the plagiary. Within a month we received letters from the University Rector and the government sponsored scholarship fund that the plagiarist and his wife (coauthor of the plagiary) had been fired from their tenured positions at the University with no possibility of return, losing their pensions and that the scholarships both authors were receiving had been revoked for life.
The Director of the Institute never responded. Two months went by and no word from the editor of the journal. We called. He had received the confession, the original and the plagiarized translation but still hadn’t been able to find a translator to render our original paper and the confession into English and so could take no action. We advised him that he should look at the numbers in the tables and the figures of the two papers, none of which needed to be translated, to see that the data and analysis presented were identical, so no translations were necessary to prove the plagiary. Similar exchanges followed over the next months with no resolution. After we sent the confession and the two papers to the managing editor of this series of journals at Springer Verlag we finally received a phone call from their offices in New York City. He said he would direct the editor of the specific journal to publish a retraction once we returned a signed statement relieving Springer Verlag of all legal responsibility for the plagiary. We told him we would sign no such statement and if we did not see a retraction printed in the journal within three months, the next time he would hear from us would be from our lawyers. We backed that up with a certified letter. Three months later the journal printed a retraction, but not before the plagiary had garnered three citations in the literature. We followed the plagiary citations for several more years, noting it was receiving 1 or more cites per year. The plagiary still came up in electronic searches with no notice attached that it had been withdrawn. The only notice of the plagiary was the withdrawal statement published more than a year after the original plagiary was published.
After the withdrawal notice was published, the editor of the journal offered me a position on the editorial board of a new journal he was trying to convince Springer Verlag to publish. I never answered his letter.
In this case the local officials, fluent in both English and Spanish, acted promptly and correctly. The on site investigation by our research assistant revealed that the Associate Director of the Institute and his wife were serial offenders. They had been preying on their students for years, copying their student research and publishing it under their own names, giving no credit to the student authors. Because of the inequality between the highly regarded plagiarist and lowly status of the student victims, coupled with the strong threats the plagiarists made to the few students who protested the intellectual theft, they had never been accused to higher authority. They had built their professional reputations almost entirely by plagiary and bullying. The best we could guess from talking to students at the time was that one of the plagiarists’ current students had finally gotten even by presenting to the professors a manuscript lifted entirely from our original published paper, turning the tables on the plagiarists. The professors got caught by plagiarizing a plagiarist.
Aside from the plagiarists themselves, the bad guys were the ones with commercial interests, the editors and publisher of the copied paper. The academic institutions acted with dispatch.
I conjecture (I have one personal example,so just anecdote) that FFP (fabrication, falsification, plagiarism) complaints to universiteis may get more traction if they are against grant-funded papers and one copies the complaints to the funding agencies or entities like the Office of Research Integrity, whose case summaries are:
https://ori.hhs.gov/content/case_summary
Here’s NSF summary of recent cases:
https://oig.nsf.gov/investigations/research-misconduct/by-the-numbers
Universities get big % of grants and are supposed to monitor the research, but they certainly don’t make it easy, and often evade ever telling complainant the outcome of an investigation.
https://www.researchgate.net/publication/332362012_Shooting_the_messenger
“[…] Study 4 shows that when bad news is unexpected, messenger dislike is pronounced. Finally, consistent with the notion that people fulfill the desire to sense-make by attributing agency to entities adjacent to chance events, messenger dislike is correlated with the erroneous belief that the messenger had malevolent motives (Studies 5A, 5B, and 5C).“
Freakonomics Radio recently had an episode on “Why Is There So Much Fraud in Academia?”
In the transcript, Dubner does cite the book Freakonomics on cheating, but doesn’t discuss there ever getting duped himself (although I recall that a while back they did acknowledge their chapter on a guy infiltrating the KKK was inaccurate since the guy in question falsified the record to protect his sources). He does, however, have Max Bazerman discussing how he co-authored a paper with Francesca Gino whose data he no longer trusts. Bazerman still credits himself with having been suspicious of a contribution by Dan Ariely, but not Gino at the time because he knew and had come to trust her.
Wonks:
Yeah, a couple people emailed me about that one; I’ll post something on it. The trust angle is interesting, because one of the problems that Kaiser Fung and I noticed about the Freakonomics franchise was that it was built upon a chain of trust that often failed.