Paxil: What went wrong?

Dale Lehman points us to this news article by Paul Basken on a study by Joanna Le Noury, John Nardo, David Healy, Jon Jureidin, Melissa Raven, Catalin Tufanaru, and Elia Abi-Jaoude that investigated what went wrong in the notorious study by Martin Keller et al. of the GlaxoSmithKline drug Paxil.

Lots of ethical issues here, but what’s interesting to me here is something about the data analysis in the original study. Here’s Basken:

[The biggest problem was] routine professional disagreements over how exactly to classify patient behaviors.

Patients who showed some form of suicidal behavior were not included in Dr. Keller’s final count, the analysis concluded, because of failures to transcribe all adverse events from one database to another and the use of “an idiosyncratic coding system.”

Such breakdowns are widely seen in clinical trials. The effect, “wittingly or unwittingly,” is to hide the adverse effects of medications being tested, said an author of the analysis, Jon N. Jureidini, a professor of psychiatry and pediatrics at the University of Adelaide, in Australia.

It’s called the garden of forking paths. If you get to choose your data-exclusion rule, you get to win the “p less than .05 game,” you get to publish your articles in top journals, and if you’re really lucky you get $$$.

Also this, which will resonate with regular readers of our blog:

Another editorial, by Peter Doshi, an associate editor of the journal, repeated emphatic criticisms of Glaxo, Dr. Keller and his co-authors (and their universities for failing to publicly rebuke them), and the journal that published their study back in 2001, the Journal of the American Academy of Child and Adolescent Psychiatry. Mr. Doshi also described turmoil within the academy, which recently elected one of Dr. Keller’s co-authors, Karen D. Wagner, a professor of psychiatry and behavioral sciences at the University of Texas Medical Branch at Galveston, to serve as its president, beginning in 2017.

Remember, Ed Wegman received the Founders Award from the American Statistical Association.

And this:

“It is often said that science self-corrects,” Mr. Doshi wrote. “But for those who have been calling for a retraction of the Keller paper for many years, the system has failed.”

Eternal vigilance is the price of liberty.

Full disclosure: I do regular consulting for Novartis.

So horrible it’s funny; so funny it’s horrible

Basken got this amazing, amazing quote:

Dr. Keller contacted The Chronicle on Wednesday to insist that the 2001 results faithfully represented the best effort of the authors at the time, and that any misrepresentation of his article to help sell Paxil was the responsibility of Glaxo.

“Nothing was ever pinned on any of us,” despite various trials and investigations, he said. “And when I say that, I’m not telling you we’re like the great escape artists, that we’re Houdinis and we did something wrong and we got away with the crime of the century. Don’t you think if there was really something wrong, some university or agency or something would have pinned something on us?”

Wow. Call me gobsmacked. Does anyone really talk like that? He sounds like the bad guy in a Columbo episode, somewhere after he stops pretending that he doesn’t know anything about the crime, and just about the time he turns to the detective and says how, even if he had done it, there’s no possible proof.

P.S. Here’s Doshi’s editorial. Worth reading. As Doshi writes, “It’s often argued that fairness in journalism requires getting ‘both sides’ of the story, but in the story of Study 329, the “other side” does not seem interested in talking.” Reminds me of Weggy. Much worse, of course, but the same principle of stonewalling.

17 thoughts on “Paxil: What went wrong?

  1. Paxil (paroxetine) was marketed in the UK under the name Seroxat. The BBC’s program Panorama did a four-part series on this whole scandal as it was developing (a scandal that was much larger than study 329). You can find the 4 broadcasts on You Tube, and their transcripts on the sites cited below. It’s fascinating stuff, and it was by no means unique:

    1. Secrets of Seroxat (aired in October 2002)
    http://news.bbc.co.uk/2/hi/programmes/panorama/2310197.stm
    2. Seroxat: Emails from the Edge (May 2003)
    http://news.bbc.co.uk/2/hi/programmes/panorama/2982797.stm
    3. Taken on Trust (October 2004)
    http://news.bbc.co.uk/2/hi/programmes/panorama/3677792.stm
    4. Secrets of the drug trials (January 2007)
    http://news.bbc.co.uk/2/hi/programmes/panorama/6291773.stm

    • You may also find this interesting, from the German weekly news magazine Der Spiegel:

      Whistleblower on Medical Research Fraud: ‘Positive Results Are Better for Your Career’. September 10, 2015
      http://www.spiegel.de/international/zeitgeist/spiegel-interview-with-whistleblower-doctor-peter-wilmshurst-a-1052159.html
      Semi-retired British cardiologist Peter Wilmshurst — described in 2012 by the British Medical Journal as a “successful and cheerful whistleblower” — began his crusade against dishonesty in medical research in 1986. In the course of the 66-year-old’s career, he conducted studies for pharmaceutical and medical devices companies, and unlike many of his colleagues, never hesitated to publish negative results. He’s been the subject of multiple cases of legal action and risked bankruptcy and his reputation to expose misconduct in the pharmaceutical industry. Today he advises and supports other whistleblowers with the organization “Patients First.”
      He sat down with SPIEGEL to discuss mistruths and fraud in medical research and why he decided to challenge the pharmaceutical industry.

      • Peter: Interesting especially this quote –
        Wilmshurst: I think there should be routine controls in labs and during clinical studies. Research should be independently checked.

        • Keith, the statements I found most interesting in this interview are these:

          Question: Why don’t doctor’s speak up [about research fraud]?:
          Answer: “it’s a bit like speaking up in Nazi Germany, isn’t it? It’s so much in the system, you don’t know who is involved and whom to turn to. Although obviously the consequences of speaking up in medical science are not so extreme — one can end up unemployed [and/or bankrupt] rather than dead.”

          “Colleagues and superiors don’t trust whistleblowers, because they think it could be their turn to be reported next. Also, the reputation of the medical institution they work for is at risk if a whistleblower reports fraud, so loyalty to your employer is valued higher than the truth.”

          “Since it’s the money of the pharmaceutical companies that controls the researchers, it would be important to put a barrier between the company and the scientific investigators. The company could pay the money that is needed to do a study to an independent body, e.g. the Department of Health. The Department of Health then suggests which researcher could do the study, and when the company agrees, pays out the money. So there is no direct contact between the company and the researche.”

          “money is very, very important. Many doctors judge their value to society by how much they earn. Like bankers. If they haven’t got enough to drive a Porsche, then they’re not a very good doctor.”

        • I wish this were specific to doctors. Unfortunately what he says applies in large parts to University academics as well.

          Perhaps its not Porsches and money that drives academics but the fame, power, awards, funding and other trappings of success can be very strong motivators to keep quiet and not rock the boat.

        • I do think independent random audits will help address those – everyone will be at some risk, everyone will know that risk and whistle blower concerns often could be staged a simply one of those “random” audits (I know that has been done to investigate incompetence in a research group).

          Also, though pharma has a lot of money, I found this sort of environment existed in academic research institutes simply driven by funding and career advancement opportunities.

          The people who did do something about, in my knowledge, had secure positions or could act through others who had secure positions. Unfortunately too few of those and many would not have known to go to them.

        • +1, although I must add that there are quite a few people who are ignorant of things that are research misconduct — “that’s the way we’ve always done it” and “that’s the way I was taught” sway them to believe that “standard” practices are OK when they are not.

  2. “Nothing was ever pinned on any of us,”

    The next step is for them to start writing papers and editorials on how those who brought up issues in their work were incompetent and or motivated by exterior motives (given there’s no possible proof).

    Often they well wait until until the detailed study records can be destroyed.

  3. I found funny the criticism by Doshi about “their universities for failing to publicly rebuke them”.

    How often have Universities rebuked article authors for a bad study? Are there any counterexamples?

    Did Columbia criticize Donald Green for his mind boggling naivete about the LaCour study?

  4. When it comes to understanding and reporting medical matters, Susan Perry of minnpost.com is the go-to person:

    https://www.minnpost.com/second-opinion/2016/01/how-outcome-switching-can-fool-us-thinking-certain-drugs-are-effective

    Her article dealing with the infamous “Study 329” refers the reader to Julia Belluz’s piece in Vox:

    “As Belluz explains, outcome switching has long been used to dupe both doctors and the public about the effectiveness of various prescription drugs. One of the most infamous cases involved a clinical trial known as Study 329. That study, funded by the drug company GlaxoSmithKline (GSK) and published in 2001, claimed to show that the antidepressant paroxetine, or Paxil, was “well tolerated and effective” for kids.”

    “But years later — after doctors had written 2 million Paxil prescriptions for children and adolescents — “it became clear that the study’s original conclusions were wildly wrong,” writes Belluz. “Not only is Paxil ineffective, working no better than placebo, but it can actually have serious side effects, including self-injury and suicide.”

    “So how did the researchers behind the trial manage to dupe doctors and the public for so long? In part, the study was a notorious example of what’s called “outcome switching” in medical research.”

    Belluz: “In Study 329,” explains Ben Goldacre, a crusading British physician and author, “none of the pre-specified analyses yielded a positive result for GSK’s drug, but a few of the additional outcomes that were measured did, and those were reported in the academic paper on the trial, while the pre-specified outcomes were dropped.”

    In order to counter the tendency of outcome switching, Goldacre has a team devoted to ” systematically checking every trial published in the top five medical journals, to see if they have misreported their findings.”

    http://compare-trials.org/

    Thus far,

    “66 TRIALS CHECKED TO DATE
    9 TRIALS WERE PERFECT
    355 OUTCOMES NOT REPORTED
    336 NEW OUTCOMES SILENTLY ADDED

    On average, each trial reported just 58.2% of its specified outcomes. And on average, each trial silently added 5.1 new outcomes.

    54 LETTERS SENT
    5 LETTERS PUBLISHED
    25 LETTERS UNPUBLISHED AFTER 4 WEEKS
    9 LETTERS REJECTED BY EDITOR

    15 letters are still awaiting a result, having been sent to journals less than 4 weeks ago.”

    • Paul:

      I’d like for researchers in such studies to analyze all possible outcomes using a multilevel model. From a statistical perspective, the problems are (a) not sharing the raw data, (b) only presenting a selected subset of comparisons, and (c) summarizing via no-pooling estimates.

  5. Finished reading Peter Doshi 3-page article: No correction, no retraction, no apology, no comment. Paroxetine trial reanalysis raises questions about institutional responsibility. BMJ, Sept.16 2015; 351 doi: http://dx.doi.org/10.1136/bmj.h4629

    PDF is available through Google.

    I’m stunned: Journal of the American Academy of Child and Adolescent Psychiatry, in which the fraudulent analysis of study 329 appeared, refuses to retract the article; the American Academy of Child and Adolescent Psychiatry refuses to discipline the members involved; Brown University, the home of the lead investigator, refuses to formally investigate him.

    Andrés Martin, Yale University professor and current editor in chief of the journal in which this piece of research fraud appeared had this to say about the article: the “article is not perfect” (quoted on p.2 of Doshi’s article)

    This is consistent with Peter Gøtzsche (Danish physician, professor of Clinical Research Design and Analysis at the University of Copenhagen, co-founder of the Nordic Cochrane Center) arguing that psychiatry is the most corrupted part of medicine.
    I recommend again his book:Deadly medicines and organized crime: How Big Pharma Has Corrupted Healthcare. 2013
    It won the British Medical Association Medical Book Award in the category Basics of Medicine (2014).
    He also has a more recent book out: Deadly Psychiatry and Organised Denial. 2015 (so far only available in electronic format)

    Andrew,
    you allude in your post to the Garden of Forking Paths. I don’t think this allusion is appropriate here. Study 329 was definitely p-hacked and, most likely, the adverse event data were partially falsified (or, put differently, coded in a totally misleading way: suicidal thoughts and behavior was classified as “agitation”, I believe; they may also have excluded some data, don’t remember the details – and have not yet read the re-analysis published in the BMJ).

    Links to some documents about study 329 (including internal company documents showing that the pharma company knew that acccording to its own data Paxil was ineffective for adolescent depression):
    http://1boringoldman.com/index.php/2012/07/05/learnt-from-the-mistakes-that-were-made/
    http://www.healthyskepticism.org/global/news/int/hsin2010-01

    • Peter:

      As I wrote here, I see a continuity, a sliding scale, from out-and-out fraud of the Diederik Stapel variety, to deliberate miscoding à la Hauser, to deliberate p-hacking, to a a possibly innocent wandering through the garden of forking paths (Bem?). I think the garden of forking paths is relevant even for deliberate p-hacking in that the mathematical principle is the same: the nominal p-value is not the actual tail-area probability under the null hypothesis. By referring to the garden of forking paths, I am not precluding the possibility of p-hacking. For that matter, Daryl Bem might have known what he was doing too.

Leave a Reply to Rahul Cancel reply

Your email address will not be published. Required fields are marked *