The paper “The association between early career informal mentorship in academic collaborations and junior author performance” that I (Jessica) previously blogged about has been retracted from Nature Communications.
Here’s the authors’ statement:
The Authors are retracting this Article in response to criticisms about the assumptions underpinning the Article in terms of the identification of mentorship relationships and the measure of mentorship quality, challenging the interpretation of the conclusions. These criticisms were raised by readers and confirmed by three experts post-publication as part of a journal-led investigation.
In this Article, we analysed publication records to identify pairs of junior and senior researchers working in the same discipline, at the same institution, who are co-authors on papers with no more than 20 authors. We use co-authorship, as defined above, as a proxy of mentorship by senior researchers, with the support of a survey that was targeted at a random sample of a recent cohort of researchers. We measure the quality of mentorship using the number of citations and the connectedness of the senior investigators.
The three independent experts commented on the validity of the approaches and the soundness of the interpretation in the Article. They supported previous criticisms in relation to the use of co-authorship as a measure of mentorship. Thus, any conclusions that might be drawn on the basis of co-authorship may not be automatically extended to mentorship. The experts also noted that the operationalisation of mentorship quality was not validated in the paper.
Although we believe that all the key findings of the paper with regards to co-authorship between junior and senior researchers are still valid, given the issues identified by reviewers about the validation of key measures, we have concluded that the most appropriate course of action is to retract the Article.
We are an interdisciplinary team of scientists with an unwavering commitment to gender equity, and a dedication to scientific integrity. Our work was designed to understand factors that influence the scientific impact of those who advance in research careers. We feel deep regret that the publication of our research has both caused pain on an individual level and triggered such a profound response among many in the scientific community. Many women have personally been extremely influential in our own careers, and we express our steadfast solidarity with and support of the countless women who have been a driving force in scientific advancement. We hope the academic debate continues on how to achieve true equity in science–a debate that thrives on robust and vivid scientific exchange.
All Authors agree with the retraction.
When I read the original study, I balked at how most of the implications suggested by the authors assumed a causal link between female mentors and poor career outcomes, at how the authors failed to mention the existing evidence that female authors are associated with lower citations, and at the label “mentorship” being applied so loosely to coauthorship.
If I’d reviewed the paper, I would have wanted, at the very least, for the authors to soften the claims about mentorship to better acknowledge the limitations of the measurements. But, given that it was published, as I said in comments on the previous post, I didn’t think the paper needed to be retracted, since reading it, it wasn’t hard to tell how they were operationalizing “mentoring” and even the egregious interpretations about women being worse mentors at the end were hedged enough to be identifiable as speculation. I tend to think retraction should be reserved for cases where the discerning reader cannot tell they are being duped, because of fraud, plagiarism, or errors in analyses that they couldn’t be expected to find.
I guess I’m comfortable with the idea of retraction having high precision but low recall. Otherwise there would be much more to retract than would seem possible. This would probably also open the door for all sorts of heuristic decisions based on political views, etc. in the less clear cut cases, which, at the extreme, could end up disincentivizing work on any controversial topics. I’m not sure how much retraction adds over issuing a correction attached to the original article in many cases, but if we’re going to do it, it makes sense to me to try to keep it relatively rare and our definition as simple as we can.
Perhaps what makes me most uncomfortable with retractions for reasons like bad speculation is that retracting some, but not all, science with misleading labels or conclusions seems to play into a myth that the scientific record is a reflection of truth and we have to keep it pure. In cases like this one where a paper makes some seemingly bogus claims but the analysis itself seems to hold, if that paper made it through peer review I’d rather trust the reader figure out which claims or conclusions are exaggerated or not rather than trying to protect them from bad speculation. Realistically, we should expect to see sloppy unsubstantiated claims in much of the science we encounter. Corrections should happen a lot more than they do, and we should not treat them like a huge earthshaking deal when authors, or journals, make them.
So initially, I was glad to see that the official retraction notice is from the authors themselves, not the journal. I assume that they will still publish their analysis else without those problematic claims. If they chose to hold themselves to a higher standard, good for them.
But here’s where the impetus behind this retraction gets slightly confusing. While the retraction notice on the website is written by the authors as though they made the decision, it seems Nature Communications did appoint a three person committee to do an additional review after posting a notice on the paper saying they were examining it more closely in November. They describe this process here:
We followed our established editorial processes, which involved recruiting three additional independent experts to evaluate the validity of the approaches and the soundness of the interpretation. They supported previous criticisms and identified further shortcomings in relation to the use of co-authorship as a measure of informal mentorship. They also noted that the operationalisation of mentorship quality, based on the number of citations and network centrality of mentors, was not validated.
According to these criticisms, any conclusions that might be drawn on biases in citations in the context of co-authorship cannot be extended to informal mentorship. As such, the paper’s conclusions in their current form do not stand, and the authors have retracted the paper.
During the investigation, we also received further communications from readers highlighting issues with the paper and are grateful to all the researchers who have contacted us and who have invested their time in reviewing the work.
Simply being uncomfortable with the conclusions of a published paper, would and should not lead to retraction on this basis alone. If the research question is important, and the conclusions sound and valid, however controversial, there can be merit in sharing them with the research community so that a debate can ensue and a range of possible solutions be proposed. In this case, the conclusions turned out not to be supported, and we apologise to the research community for any unintended harm derived from the publication of this paper.
So maybe Nature Communications would have retracted the paper even if the authors didn’t. Or maybe they brought to the authors’ attention that they were considering retracting, and the authors then felt pressed to. I’m not sure.
At any rate, the backlash against the Nature Communications paper made clear to me how ambiguous language can be in cases like this, and also how different people’s views can be about the responsibility that readers should have. I found the liberal use of the terms mentor and mentorship quality throughout the paper very annoying but not fatal, since the details of what was measured were there for the reader to see, and I read enough social science to be accustomed to authors at times adopting shorthand for what they measure, for better or worse. But if you read many of the sentences in the paper without the mental substitution, they are pretty problematic statements, so there is some space for judgment. Perhaps because of this ambiguity in language, the idea of protecting the reader from misleading claims by retracting everything that someone later points out makes misleading claims seems fraught with challenges to me.
I’ve also seen some people suggesting it’s unfair for papers like the Nature Comm one to be subject to so much public criticism based on their topics striking a political chord. It seems naive to expect that papers that make strong claims counter to things that many people firmly believe wouldn’t get some extra scrutiny. People simply cared about the topic. I don’t think the paper warranted retraction in this case over something like a statement warning that the measures of mentorship quality weren’t validated. But suggesting that it shouldn’t have gotten so much scrutiny in the first place contributes to a belief that the published record should be venerated. I can sympathize with authors who feel singled out based in part on the topic of the work; I had one of my first published papers ever critiqued by a well-known practitioner in my field. It had won an award, and it made a controversial argument (that sometimes making graphs harder to read is better because it stimulates active cognitive processing on the part of the viewer). Getting critiqued for a paper that I suspected was singled out partly based on the topic wasn’t fun at all. As Andrew recently blogged about, there often seem to be worries in such cases about how public criticism will affect the academic career, and so if the process feels somewhat random it can be unsettling. But there were some valid points made in the critique, as there are in most critiques, and so I learned something, and I assume others did too. The idea that it should have been withheld out of respect or fairness didn’t make sense to me then or now. We shouldn’t be trying to reserve public scrutiny for only the most horrible papers.
