Your (Canadian) tax dollars at work

Retraction Watch links to this amazing (in a bad way) article by “The International Consortium of Investigators for Fairness in Trial Data Sharing” who propose that “study investigators be allowed exclusive use of the data for a minimum of 2 years after publication of the primary trial results and an additional 6 months for every year it took to complete the trial, with a maximum of 5 years before trial data are made available to those who were not involved in the trial.”

It’s not just that. They also want us to pay:

Persons who were not involved in an investigator-initiated trial but want access to the data should financially compensate the original investigators for their efforts and investments in the trial and the costs of making the data available.

Ummm, we already did pay for this—it’s called “taxes”!

To the credit of the New England Journal of Medicine, they allowed open comments, which were universally, 100%, negative.

From one comment:

“A key motivation for investigators to conduct RCTs is the ability to publish not only the primary trial report, but also major secondary articles based on the trial data.” This is used as a justification for a lengthy delay in data release. THE primary motivation MUST BE to improve patient health. As long as medical investigators continue to prioritize publication rights over what is best for the patients, progress will be slowed.

Yup.

From another comment:

Researchers should get credit whenever secondary analyses are performed. Having more sets of eyes on the data could mean more discoveries are made. The incentive created by getting more articles published from their data and having enabled more discoveries should be plenty of motivation for researchers under the current publish-or-perish system.

Yup again.

From another comment:

“We believe 6 months is insufficient for performing the extensive analyses needed to adequately comprehend the data and publish even a few articles.”

Are the authors proposing that dying patients should wait a few years such that medical researchers can climb the career ladder?

Yup yup yup.

Take-home message

What’s scary is not that some researchers have these views but that they were considered so normal that New England Journal of Medicine published them without editorial comment.

And with that we close out our blogging for 2017. Let’s hope for more data sharing in the forthcoming year!

19 thoughts on “Your (Canadian) tax dollars at work

  1. Note that the 16 comments on this article in the NEJM were uniformly negative. Note also that the article was published in August, 2016 – prior to the SPRINT Challenge which the NEJM sponsored in order to address these very questions. It would be interesting to see if the NEJM has changed their stance on data sharing from clinical trials. If you read their views from the SPRINT Challenge and the associated conference (https://challenge.nejm.org/pages/free-web-event), they seem to be trying to walk a fine line between the competing interests of clinical trialists, patients, researchers, and funders. It would be nice to see a clear statement distancing themselves from their earlier views about restricting access to data. But I am not so sure (and could provide a good deal more evidence about my suspicions). But it is nice to the near-universal views of commentators on this subject.

    • This coming after the even more dubious commentary they published on “Research Parasites” (aka Data Scientists). I agree that an explicit statement from NEJM would be reassuring.

      • “This coming after the even more dubious commentary they published on “Research Parasites” (aka Data Scientists).”

        I got so tired of let’s blame everything on incentives and/or not even think about how and why these incentives possibly came into existence, that i recently wrote a little psychological science focused piece called:

        “Making the most of tenure in two acts: An additional way to help change incentives in Psychological Science?”

        I found it very fun to write, if only for the opportunity to use some of my favorite Meehl quotes. I doubt it will be useful however.

        Regardless, i thought of it as a result of the above comment as it also finds a possibly more scientifically accurate description of “research parasites”.

        https://psyarxiv.com/5vsqw/

        • Anon:

          Your article is excellent. It reminds me that when I was a graduate student I saw older faculty members who were eminent in their fields but had not done any interesting work in decades. As a student, I realized I would be eminent by the age of 30 (statistics is a small field, I was well situated, and it was clear this would happen), but I did not want to do nothing for the 40 years following. So I made it clear to myself back then that eminence would not be a good goal in itself. It seems that not everybody feels this way. For many faculty it seems that eminence is the goal, and once they achieve this, it’s all about the game of ensuring that their students climb the latter, etc.

        • I’m not sure but I think you have to be a Catholic cardinal or Orthodox archbishop to get be addressed as Your Eminence. Not even tenure is enough.

        • Sorry jrk, but your comment just doesn’t make the grade (assuming it was an attempt at humor) — conflating a noun (lower case) with a title (upper case) would need a carefully crafted context to work.

  2. My humble contribution to the argument is that we should push (hard) for a norm whereby the creation and dissemination of a data set is an entry on a CV. Like with published articles, we could use statistics on the data set to measure overall scientific impact. Small and restricted data sets would not be worth much, just like a low impact pay journal article isn’t. But a trial could be a major CV point and a sign of productivity.

    That would let people who are good at collecting high quality data focus on that, and not worry that they might sink their career. Data scientists could focus on analysis and everyone wins.

    I think that the moral argument (patients first) is correct, but if you misalign the incentives so that moral conduct costs people their jobs (which low productivity does) then the real issue is that the system is broken. Crediting the data set creator for use as a sign of impact would incentivize early to access data with very good customer support, as everyone wants their data set to be used for high impact work under this rubric.

    How does this not work better?

  3. While data sharing is important, there are a lot of other things amiss with the way clinical trial results are currently (not) shared, and a lot of those other flaws are far easier to fix.

    It’s 2018 and many trials are still not getting registered in the first place, their summary results are often not posted on registries, and many journal articles reporting trial results resemble marketing literature more than scholarship.

    Transparency International and TranspariMED recently released a study documenting these problems and outlining possible solutions:
    https://www.transparimed.org/single-post/2017/12/14/New-study-documents-the-harm-caused-by-evidence-distortion-in-medical-research

    I’m all for rapid IPD sharing but it seems a bit pie-in-the-sky as long as we can’t even get the most simple basics right.

  4. A stray idea.

    Perhaps it would help to align incentives if follow-on papers that used a dataset included the creators of the dataset as coauthors. Then, the creators wouldn’t have to do the follow-on papers themselves, others could do the follow-up papers, but the creators would still get credit.

    Possible problem: too many creators would have to be included as coauthors exacerbating the problem of too many coauthors. Plus, if later researchers extend the dataset, they would have to be included too. Could get out of control.

    I can see why dataset creators are so protective of their data. I have seen a number of academic careers based on access to a unique dataset.

  5. The 2+.5*year seems a long time but not all research institutes are set-up like North American institutes. There are lots of smart people doing research on a shoe string and have a small team to do a lot of the non-clinical jobs e.g. data cleaning, analysis, writing.

    It seems to advantage big and well-funded institutes.

    I think such a strict rule will invite people to game the system e.g. purposefully recruit patients more slowly so the trial last longer but gives more time to clean the data and pre-prepare analysis programs and fill-in-the blanks articles etc. Which would seem to defeat the argued purpose.

    It does actually take a lot of time to prepare data for people who have had no contact with it’s context.

    • Not sure what you think will advantage big and well-funded institutes. The advantage to everyone of sharing is that the data will be out there. Presumably the bigger institutes will be producing more data. This lets little institutions or sole practitioners into the game. The big institutes have advantages of both budget (they can hire people) and prestige (people want to work there), which compounds the winner-take-all effect in academia (and elsewhere). Smaller institutes have an advantage in terms of less bureaucracy. Lack of funding gives you a lot of freedom to pursue what you want if you can get it done by yourself and the odd student/volunteer (I had very little funding as an academic the first time around and Stan was launched on a couple postdoc salaries).

      • Bob:

        I think it is a bit different in clinical research in Canada (at least Ontario). Now, I was interviewed for a position to provide statistical mentor-ship for one of the author’s clinical trail groups but withdrew.

        It takes a lot of resources to do clinical trials – taking blood samples, monitoring patients, dispensing drugs, obtaining approval to conduct trials involving drugs, reporting requirements as the trial is on going, etc., etc. often requiringing professionals that do not like working on short term contracts.

        In fact, in 1980s and 1990s when I worked in clinical research in Toronto we decided not to try to do many clinical trials as the cost of living was to high in Toronto to keep staff. That is likely an important reason why Hamilton became the clinical trials center (along with London and Kingston).

        So the resource needs are well justified but obtaining and maintaining them through data “pimping” is totally unacceptable.

Leave a Reply to Keith O'Rourke Cancel reply

Your email address will not be published. Required fields are marked *