Skip to content
 

Ioannidis: “Evidence-Based Medicine Has Been Hijacked”

The celebrated medical-research reformer has a new paper (sent to me by Keith O’Rourke; official published version here), where he writes:

As EBM [evidence-based medicine] became more influential, it was also hijacked to serve agendas different from what it originally aimed for. Influential randomized trials are largely done by and for the benefit of the industry. Meta-analyses and guidelines have become a factory, mostly also serving vested interests. National and federal research funds are funneled almost exclusively to research with little relevance to health outcomes. We have supported the growth of principal investigators who excel primarily as managers absorbing more money.

He continues:

Diagnosis and prognosis research and efforts to individualize treatment have fueled recurrent spurious promises. Risk factor epidemiology has excelled in salami-sliced data-dredged papers with gift authorship and has become adept to dictating policy from spurious evidence. Under market pressure, clinical medicine has been transformed to finance-based medicine. In many places, medicine and health care are wasting societal resources and becoming a threat to human well-being. Science denialism and quacks are also flourishing and leading more people astray in their life choices, including health.

And concludes:

EBM still remains an unmet goal, worthy to be attained.

Read the whole damn thing.

87 Comments

  1. Ana says:

    Ahem, collective choice as standard of evidence. This vacuum breeds what he says. How is this any mystery…
    Still wishing for a counterpart of “The wage bargain” for health insurance.

  2. mc says:

    excellent example in the last couple of days from the urologists who have protested their livelihoods and those of their lab/equipment/maintenance orgs being threatened by the AMA’s call for less PSA testing after extensive evidence-based analysis. study appeared from urology researchers looking at an increase of aggressive prostate cancers over a recent period and using that as justification to call basically for rejection of the restrictive recommendations. however, they did admit that the increase started before the AMA recommendation and, further, did not separate the populations since then into those who did or did not receive the PSA testing after the recommendations, which has continued as a high-grossing profit sector for the medical/urology community and the general practitioners who are parts of medical groups with the urologists. as someone who has been aggravated with both his last two GPs who ran PSAs as part of the blood workups since the AMA recommendation, I have been especially concerned that the AMA recommendation apparently went virtually nowhere. if that’s true, the research findings actually would indicate that something major might be happening that has nothing to do with the PSA screenings being decreased and might indicate that the still-occurring PSA screenings aren’t even catching what they claim to catch. which is what the inventor of the PSA test has said all along as he has railed at the urology community for turning it into a cash cow that actually harms far more patients than it helps. it goes beyond disgusting just for the prostate cancer problem and resulting policies, showing how/why any efforts at reducing medical costs and insuring appropriate care to the broadest number of people possible have to take far more into account than simply EBM and its abusers.

    • Carlos Ungil says:

      Recommendations for fewer mammograms get a similar response. Everyone’s incentives are aligned to get more diagnoses and more treatments. Doctors, diagnostics providers, pharma companies, patients… nobody wants less of it. What kind of monster wouldn’t want to detect and cure cancer? (Suggested readings: “Less Medicine, More Health” and “Overdiagnosed: Making People Sick in the Pursuit of Health”, by Gilbert Welch).

      Regarding EBM, this review of Goldacre’s “Bad Pharma” by David Healy discusses its limitations: http://davidhealy.org/not-so-bad-pharma/ “Introducing trials into the regulatory apparatus has created a mess. What’s been captured is the production of evidence and the more books like Bad Pharma fetish clinical trials the more captured medicine becomes.” Check his book “Pharmageddon” or Peter Gotzsche’s “Deadly Medicines and Organised Crime: How Big Pharma Has Corrupted Healthcare” if you want to lose any remaining faith in the system.

  3. Anon says:

    Depressing

    Scientific research needs to be strategyproofed somehow

  4. Jonathan (another one) says:

    Just the sort of combination of idiosyncrasy and sharp observation that we have come to expect from Ioannides. I don’t work in the medical field, but if is at all correct, the prospect is depressing. What’s worse, there is every reason to think that the situation is actually *better* in medicine than outside it. What’s even worse than that is that I think he has at least partly misdiagnosed the problem — sure, entrenched corporate interests want to skew the process to make money, but I think the problem is worse than that. I see corporations trying to fool *themselves* with these sorts of analyses against their own direct interests! High ranking corporate exec orders up an internal (or consulted from experts) study to prove X; neither Not-X nor the data are too noisy to tell are acceptable answers. That when X is implemented the company loses money can always be blamed on a million things other than the study.

    • Rahul says:

      Somehow I worry less about “entrenched corporate interests” than what shenanigans researchers will indulge in to further their *own* career interests.

      “Market pressure” worries me less than tenure pressure or publish-or-perish pressure.

      It is easy to point at external factors than to acknowledge internal problems. We much apportion a huge chunk of blame to academics themselves and to the modern system of academic incentives than to keep blaming it all on evil corporates.

      • Jonathan (another one) says:

        Absolutely right. Too many people think funding transparency addresses the issue. It doesn’t even come close. IMO, all it does it give people a means to avoid confronting whatever evidence the study actually gives.

      • Anon says:

        Yes, a shameful pursuit of self interest is deserving of much blame.

        Except of course if we’re talking corporates. That’s what they do. It is to be tolerated, even embraced. The market at work.

        • Aaron says:

          I am a periodic reader of Andrew’s blog, and what I have read here in this column and the subsequent comments is both depressing and alarming.

          Given the current state of scientific research, where the suggestion is that both corporate interests and the publish-or-perish mentality and tenure pressure is skewing towards poor research outcomes and such activities as “p-hacking”, here is the question: what can actually be done about all of this?

          Can anyone here (whether Andrew or the other commenters) provide actual suggestions towards solutions that can make a difference in practice?

        • Rahul says:

          I am not at all absolving corporates of guilt. Just saying that it misses the point to focus only on corporate greed (like Ioannidis does) while ignoring perverse researcher incentives.

  5. Anonymous says:

    If anyone else had written this, it wouldn’t have been accepted for publication. This reads like a personal diary entry packed with unargued assertions, and non-representative anecdotes. I actually agree with much, if not most, of his complaints, based largely on my own similar experiences. Notably absent from his list of complaints, however, is the fact that well-established researchers, such as himself, routinely enjoy the opportunity to rant without evidence or argument in respectable journals about whatever is on their mind.

  6. Stephen says:

    God Ioannidis is really full of himself – or maybe this bit on page 6 is satire?

    “Even the syndicalist who had once tried to annihilate me re-approached me: “John, we all know that you are the best scientist in the country. Why don’t we work together? You know how successful I am.” He presented a long list of his power attributes and connections. The catalogue was stunningly impressive. Then he added: “the only thing that I lack is major publications in top impact journals. So, here is what we will do: I will give you power and you will put my name in major evidence-based publications. I hate having power, so obviously I declined. I have always preferred to work with the young and the powerless. “

    • I took it to be simply evidence of how the people JI is castigating will cheerfully perform a volte face and try to flatter you with this type of stuff to enhance their own careers. I kind-of understand the final couple of sentences too, but only kind-of.

      • Stephen says:

        I think you’re being too charitable – with false immodesty he just comes across as an insufferable blowhard.

        • No his point is exactly what Matthews says, people come and broker power shamelessly. He isn’t in any way saying that he really IS “the best scientist in the country” or etc. He’s saying that people say these things to curry favor. I’ve seen it go just exactly like that. A urological surgeon came to my wife’s postdoc mentor and brokered a deal for one of his students to work in her lab using EXACTLY that kind of shameless power-grabbing.

      • Martha (Smith) says:

        I take this quote as evidence that Power Pose can be harmful to human health.

    • Shravan says:

      It’s just hard for academics to resist whining about how they have been wronged.

      • Stephen says:

        I’m sorry it does just come across as a half hour stuck with an opinionated bore…Wheres the evidence? The arguments? Its a screed.

        In fact it reminded me so much of a snippet in another recent story in a much better Nature article on peer review – incidentally from a Cochrane conference.

        “A distinguished editor in the audience took another view, excoriating presentation after presentation. Finally, Iain Chalmers (who later co-founded the Cochrane Collaboration) stood and addressed him: “We have listened to your incessant criticisms of everyone who has gone to the trouble of obtaining data. What we have not heard from you is one single piece of evidence for your opinions.”

    • jrkrideau says:

      I think you fail to appreciate the sarcasm. As far as I can tell the entire paper is written sarcastically as well as bitterly.

  7. Shravan says:

    Pretty scary if true. I wonder if there is any way to estimate what proportion of work in medicine is kosher. Random sampling, followed by expert assessment, or just consulting an expert like Ioannidis, could give reasonable ballpark numbers. In other words, how bad is the situation, and are there at least some people doing serious work (there must be)?

    • Rahul says:

      I’m a cynic, but doesn’t Ionnidis have an incentive to overstate how bad the situation is, given his key thesis and history of publishing on this topic?

      I’m not saying the situation isn’t bad, just that at this point, I somehow cannot see Ionnidis as the disinterested expert.

      • jmf says:

        Yes, he does have an incentive, and, yes, he does overstate the problem. Not that the situation isn’t bad. It is. But Ioannidis has now an illustrious career of sweeping claims based on shoddy analyses and weak data. Exactly the kind of thing he criticizes.

        • jrkrideau says:

          He may or may not be overstating the case but a recent retraction watch post is not encouraging

          http://retractionwatch.com/
          A journal is retracting three papers and a letter from a bone researcher who admitted to scientific misconduct, noting that all co-authors were included only for honorary reasons.

          Gift authorships anyone?

  8. Anoneuoid says:

    >”There are also so many quacks ranging from television presenters and movie stars turned into health trainers [40], and pure science denialists (e.g. climate, HIV, vaccine denialists, and religious fundamentalists) that one has to tread carefully. We should avoid a civil war on how to interpret evidence within the health sciences when so many pseudo-scientists and dogmatists are trying to exploit individuals and populations and attack science.”

    I thought Ioannidis got it until I read this part. The majority of the medical research community is already practicing a pseudoscience they learned from psychologists. The sham is that they test a default null hypothesis and then, when it is rejected, go on to accept some other (untested) research hypothesis. These people are just as bad as the quack on TV, and more dangerous because our culture gives them an air of credibility. The biggest threat to science was the professional researchers and institutions themselves. Nowadays there is little scientific left to threaten. The attacks we are seeing today are directed towards the pseudoscience establishment. These attacks are a good thing.

    >”I won’t blame anyone. These physicians have no other option. This is how the world works, they are fighting to keep their jobs. Yet, how likely is it that physicians will design studies whose results may threaten their jobs by suggesting that less procedures, testing, interventions are needed? How likely is it that, if they do design such studies, they will accept results suggesting that they should quit their jobs? How many are willing to fully resign from the field where they have built a name, as you did twice in your career, David [34,35]?”

    This is odd because the “other option” is mentioned a few sentences later. The other option is to quit the unethical job and go do something else. It will be painful, but is possible.

    • jrkrideau says:

      These physicians have no other option

      The other option is to quit the unethical job and go do something else.

      There is the other option but who can afford to take it?

      A doctor has spent, perhaps, 14-1 years learning the trade if one is a specialist, has enormous debts, and a young family.

      Leaving a reasonably well-paying and prestigious position to possibly retread completely is daunting to say the least. People do it but the cost to others (especially spouse and children) could be nasty. Making such a decision is going to be very difficult particularly if you think the system is screwed up but not all that screwed up.

      Also cognitive dissonance is very likely going to let you think even if there is a bit of problem with the system you are still making real contributions.

      • The Cassandra Syndrome at work. The history of engineering is filled with people who said “the shit will hit the fan in exactly this way…” and were ignored until it did in fact happen. Hurricane Katrina, Chernobyl, Space Shuttle Challenger, and Fukushima are prime examples. I feel like Ioannidis and a few others are of the same ilk, but the system they’re warning about is basically a social machine, not a physical one.

        • Rahul says:

          OTOH, would we ever hear about the people who said “the shit will hit the fan in exactly this way…”; and it never did?

          • Asymmetry of information raises its ugly head again. Those who hear 100 prophets of doom know nothing about which ones to pay attention to. Those who are experts with good solid reasons and evidence know that they must not be ignored. One of the biggest problems with this massive industrial farm of pseudo-scientific hogwash is that it raises the noise and costs dearly in terms of ignored real signals. The cost of the breakdown in science is not the dollars funded to Power Pose and PSA screening, it’s the lost opportunity to understand real psychological issues and find methods to customize cancer treatments making them 10x more effective.

            • Rahul says:

              My point is, for most part, even the knowledgeable among us, often find it impossible to tell who to pay heed to.

              Hindsight is everything.

              Before the 2008 recession were all econ experts forecasting it? Could we have judged who was gonna be right?

              When the Paul-Ehrlich wager was made could we have predicted which “expert” was right? Sure, a doomsday forecast comes right every once in a while.

              We are being too generous about our predictive abilities if we ignore the number of times that the predicted shit never hit the fan!

              • The Cassandra Syndrome is exactly caused by Asymmetry of Information. Cassandra is given the gift of seeing the future, and the curse that no-one will believe her. Of course it makes perfect sense for the people not to believe her, think of how many people predict the future incorrectly. Further evidence of the flaw in Frequentist logic :-)

                BTW: with things like the 2008 recession, everyone knew it was coming, it’s just no one knew when. I think it was Keynes who said “Markets can remain irrational longer than you can remain solvent”

          • There’s no shortage of examples: Thomas Malthus, Paul Ehrlich, Club of Rome, the dozens of experts included in Phil Tetlock’s famous study…Dan Gardner’s “Futurebabble” (2010) gives an entertaining tour d’apocalypse.

    • Yes, but doing this essentially leaves the field open to charlatans who will continue and even grow the level of funding and soforth.

      I have a real conflict about this because on the one hand the system for research funding is fully corrupted, large universities shameless plug terrible research as if it were their gift to humanity that someone has finally uncovered the secrets of Power Pose or the cardiovascular benefits of snorting snowcones or have purchased the largest possible farm of high performance computer nodes to study the variations in the weather hundreds of years from now in a global model that can’t even predict the weather this coming sunday or whatever…. and at the same time these universities are participating in one of the biggest land-grabs we’ve seen in a long time, in the form of essentially indentured servitude through selling non-bankruptcy-eligible student loans to children, whose dollars are increasingly going to pay for deluxe dorm rooms and student exercise facilities, and football coaches salaries that act as training facilities for privately owned NFL football teams that suck money out of local economies through publicly funded billion dollar stadium projects, while actual teaching of knowledge takes a back seat, and teachers are largely replaced by cheap contractor “lecturers” from among the ranks of excess PhD students, who get paid less than minimum wage when you consider all the prep-work they wind up doing, and at the same time, projects such as Coursera and Khan academy and soforth are actively trying to eliminate the teaching component of large universities anyway, so that they can become pure economic signaling institutions predicated predominantly on amplifying the effect of SAT scores and legacy connections…

      and on the other hand, if we have the ability and knowledge to do good science and to help create and improve real science-based health or engineering or economic policy practices… and we don’t out of disgust for the rest of the industry, are we in fact helping that industry do its corrupted thing? Can we afford as a society to bury real science under a craptastic load of polished dung?

      • Martha (Smith) says:

        Lots of good points in this rant.

      • Shravan says:

        I see this rant as also implying that more competent people should try to actively enter the relevant field and try to influence the direction things are going in. Maybe Daniel should consider doing that.

        Similarly, do statisticians sit on the editorial board of Lancet and other such journals? Why don’t they have statisticians checking every paper? That would already be a start.

        As a reviewer, I notice that the way funding agencies distribute money (US, Europe) is influenced to a disturbing degree by the fame level of the researcher (a famous scientist can get millions, sometime multiple times for the same idea), the perceived newsworthiness of the research (will it look good as a news item?), and personal connections. Part of the reason this happens is that there is no transparency. E.g., try finding out which projects were submitted vs funded by an agency. And try finding out why. If funding agencies were forced to report all submitted proposal titles and abstracts in the public domain, they might get a bit more careful about their decisions. Right now it’s a secret society.

  9. Ian Fellows says:

    Thank you for sharing this. I’m surprised at the strength of the negative reactions here. Yes, it’s a bitter screed. I don’t think the author would disagree. The hope and high minded ideals that he strived for with his recently deceased colleague have fallen flat. The very real danger that the truth of science can be twisted or drown out by the incentives of the market is terrifying to him. Indeed, it is terrifying to me.

  10. Jon says:

    I question whether this was worthy of publication. The anecdotes are disturbing, but we don’t have the answers from people on the other side and one cannot tell how pervasive this is.

    He cites no examples of bad decisions or even questionable decisions that illustrate the consequence of the problem. For all I know he may have rotten-cherry picked the worst moments of his career and ignored many more positive experiences.

    • Ian Fellows says:

      Worthy of publication? It is not a research article, but part of a commentary series commemorating a dead colleague. I hope when I die my friends and colleagues can write what they want in their obituaries to me.

      http://www.sciencedirect.com/science/journal/08954356/73/supp/C

    • Martha (Smith) says:

      Jon,
      It sounds like you are not familiar with Ioannidis’ work. His main focus is on improving the quality of medical research; to do that, it’s necessary to focus on the worst cases — those are the ones that most require improvement. Focusing on his positive experiences would be neglecting his purpose. (And bear in mind, as others have pointed out above, that this particular article is part of a collection in tribute to D. L. Sackett — see http://www.jclinepi.com/article/S0895-4356%2816%2900158-X/abstract .)

      • Jon says:

        No I am not familiar with his work; but a piece with such broad generalities I don’t find very helpful.

        • Rahul says:

          EBC (Evidence based criticism) still remains an unmet goal, worthy to be attained.

          • Keith O'Rourke says:

            Very good point and I think a good example from JI here http://biostatistics.oxfordjournals.org/content/early/2013/09/24/biostatistics.kxt036.full

            There does seem to be this academic myth that JI never bothers to try to get actual evidence.

            By the why crappy evidence is worse than no evidence.

            • george says:

              A bit more on this story;

              http://simplystatistics.org/2013/09/26/how-could-code-review-discourage-code-disclosure-reviewers-with-motivation/

              Based on the supplemental material JI seems, to me, more interested in pushing his own agenda (and in this case killing the work of others) than fairly representing the facts. Goodman and Greenland’s critique of JI’s 2005 paper (see http://biostats.bepress.com/jhubiostat/paper135/) supports this view, I think.

              An academic version of “truthful hyperbole”, perhaps?

              • Andrew says:

                George:

                The Goodman and Greenland article seems very reasonable to me. I think that the framing of research results as “true” or “false,” as in the Ioannidis article, doesn’t typically make sense. Is the “air rage” paper, for example, true or false? Different flying conditions must have some effect on passenger behavior, but the causal claims in that paper are produced by the usual method of finding statistically significant patterns in noise. That doesn’t mean its claims are “false”; indeed it’s not really clear what those claims are, in that they are presented in terms of direction and magnitude. I think it’s much more helpful to say that the directions are basically random (that is, a type S error rate of close to 50%) and that the magnitudes are massively exaggerated (that is, very high type M errors). The false-poaitive, false-negative framework doesn’t really work for this and so many other studies; it’s just a mathematical structure left over from many decades ago.

                At tha same time, I’m supportive of Ioannidis’s general point that, in some literatures, the empirical results of most published papers are close to meaningless. Hence I agree with Goodman and Greenland that Ioannidis was pointing in a useful direction even though his particular analysis seems inappropriate to the situation at hand.

              • Martha (Smith) says:

                I consider Ioannidis’ 2005 paper as a “back of the envelope” calculation, which I’ve never taken as giving (nor intending to give) a definitive result, but instead as giving plausible evidence for scientists to question “the way we’ve always done it.” To the extent that it has done that, it has been an important contribution to the literature.

              • Keith O'Rourke says:

                George:

                With regard to link on more on this story, I do not think it is productive (and perhaps a bit insulting) to guess reviewer/commentator’s motivations and even claim to have clear evidence of them e.g. “The motivated discussant wanted to discredit our approach. … included their email exchange with the editor about the bug and this quote … the statement is clearly designed for the sole purpose of embarrassing us (the authors) and discrediting our work”.

                It would be much more productive (as one of my past directors explained) to pretend no matter hard that the reviewer/commentator was actually trying to help with the work (after of course taking a day to rage about how evil they really are). To me, the _motivated_ discussant found a bug in their code worried that it might have a big effect on the results but also it suggested possibly more bugs and perhaps some sloppiness – which I think are responsible worries until they were ruled out.

                And there was this on an earlier version of that paper “that you may have just biased every statistical reviewer that we might get in the future with one fast and loose post. Jeff” http://statmodeling.stat.columbia.edu/2013/01/24/i-dont-believe-the-paper-empirical-estimates-suggest-most-published-medical-research-is-true-that-is-the-claim-may-very-well-be-true-but-im-not-at-all-convinced-by-the-analysis-being-used/#comment-132708

                As for the Goodman and Greenland’s critique of JI’s 2005 paper, when I discussed this with Stephen Goodman just after he wrote it I think, I suggested it was somewhat overly critical of a back of the envelope calculation.

    • Shravan says:

      Sounds like you’ve been the fortunate position of not having much contact with the medical world. Everything I have experienced as a transplant patient and now as a dialysis patient is consistent with what he says. Evidence-based medicine suffers from similar manipulation and vested interests as the kinds of research coming out in social psych and other areas, the kind of stuff Andrew criticizes. When Amy Cuddy makes millions/builds her reputation selling her shaky ideas, the harm caused is “just” a waste of a lot of time and money. In medicine people will suffer physically.

  11. Alex Gamma says:

    Rahul says “entrenched corporate interests” and “market pressure” worry him less than researchers’ perverse career incentives. But one can and should worry about both. An area where corporate interests have perverse effects on medicine is advanced-stage cancer care. It is the logic of profit that dictates that companies keep flooding the market with highly expensive me-too cancer drugs (10’000 $ per month of treatment [1]) with marginal survival benefits (2-3 months on average [2]) and an effective decrease of quality of life due to severe side-effects (including things like nausea, vomiting, diarrhoea, fever, headaches, fatigue, loss of appetite, mouth sores and others).

    Regulators, physicians and patients have so far simply accepted this as the way things are, and have therefore allowed pharma to implicitly define the standard of care: patients go through rounds of aggressive treatments using the latest corporate offerings, they suffer from the side effects, and then usually die. The de facto treatment ideal behind this is to prolong the patient’s life no matter the cost. But it is not an ideal that anyone has chosen after careful deliberation of risks and benefits, as it should be. It is simply the result of current incentive structures in the drug market and health care providers and patients aligning with them for various reasons.

    But clearly, it should be society and not corporations that set priorities in end-of-life care, based on moral considerations, especially patient preferences. Patients should arrive at informed health choices based on shared-decision making, where they are equal partners to doctors. Emphasis should be placed on palliative care accompanying patients early-on, helping them understand not only available treatment options but also the end-of-life issues facing them from a patient perspective [3]. There is already evidence that if patients were better informed, many would prefer measures that improve the quality of life without prolonging life to such that only prolong life without any symptom relief [4].

    Why is this not happening?
    One important reason are severe deficiencies in patient-doctor communication. A recent study in 178 patients with end-stage cancer and a life expectancy of of less than 6 months found that only 5% had been fully informed about their status as terminally ill [5]. In other words, 95% had never been told they were about to die. The reason: doctors were very reluctant to communicate the facts clearly. They said things such as “your tumor has only grown by 2 mm” or “some of your tumors grew, but others didn’t”, suggesting that the situation is not hopeless, but really meaning nothing of that sort [6]. Maybe they have some misguided sense of “sparing patients the worst”, or maybe they fear their reputation might suffer as they could be seen as being “pessimistic”.

    Similarly, a recent literature review concludes that “[t]imely and effective communication about serious illness in primary care is hampered by key clinician barriers, which include deficits in knowledge, skills, and attitudes; discomfort with prognostication; and lack of clarity about the appropriate timing and initiation of conversations.” [7]

    How this failure feels from the patient/family perspective is impressively recounted by the husband of a woman who died of cancer (published as a perspective piece in BMJ [8]):

    »Our physician referred to each new round of therapy as the “gold standard,” meaning the approach that scientific evidence shows works the best. What patient in a difficult situation could resist faith in the gold standard? What we did not appreciate then was how little evidence there is that Elaine’s gold standard treatments produce outcomes that actually matter to patients and families…

    The symptoms can be so debilitating that they steal from life all of the pleasures of seeing family and friends, of being engaged, of looking forward to the remaining moments…. While her disease presented tremendous challenges, the interventions made in the name of prolonging her life were by far the greater source of her miseries.

    Oncologists have told me that patients in this situation are desperate for solutions and want every possible chance. I don’t doubt this. But I also suspect that most cancer patients’ outlooks are based on incomplete information about the ordeal ahead and their odds of success.«

    There is the factor of human suffering in this, which is bad enough, but there is also the waste of resources where billions go into the development of unneeded drugs. Even just a fraction of that money spent on re-structuring the system around humane values, re-educating physicians, and reinforcing palliative and hospice care could easily do more good than the whole cancer drug industry has done so far.

    [1] Mailankody S, Prasad V. Five Years of Cancer Drug Approvals. JAMA Oncol. 2015 Jul 1;1(4):539.
    [2] Fojo T, Mailankody S, Lo A. Unintended Consequences of Expensive Cancer Therapeutics—The Pursuit of Marginal Indications and a Me-Too Mentality That Stifles Innovation and Creativity. JAMA Otolaryngol Head Neck Surg. 2014 Dec 1;140(12):1225.
    [3] STAT News. A doctor focused on dying finds lessons for better living. https://www.statnews.com/2016/07/06/bj-miller-doctor-end-of-life-lessons/?s_campaign=stat:rss
    [4] Silvestri G, Pritchard R, Welch HG. Preferences for chemotherapy in patients with advanced non-small cell lung cancer: descriptive study based on scripted interviews. BMJ. 1998;317(7161):771–5.
    [5] Epstein AS, Prigerson HG, O’Reilly EM, Maciejewski PK. Discussions of Life Expectancy and Changes in Illness Understanding in Patients With Advanced Cancer. J Clin Oncol. 2016. Available from: http://jco.ascopubs.org/cgi/doi/10.1200/JCO.2015.63.6696
    [6] STAT News. Cancer patients keep getting aggressive end-of-life treatment, despite lack of benefit. https://www.statnews.com/2016/06/06/cancer-patients-end-of-life/?s_campaign=stat:rss
    [7] Lakin JR, Block SD, Billings J, et al. Improving Communication About Serious Illness in Primary Care: A Review. JAMA Intern Med. Published online July 11, 2016. doi:10.1001/jamainternmed.2016.3212.
    [8] Klepper B. The Gold Standard for Current Cancer Treatment. JAMA Intern Med. Published online June 27, 2016. doi:10.1001/jamainternmed.2016.2932.

    • I’ll corroborate the general lack of end-of-life realism. When my 92 year old grandfather died in early 2015, my sister, a nurse practitioner, and I both had primary concerns for his comfort, and suggested to doctors that they discuss his end of life options as he entered the hospital. Instead, after a week or so in end stage heart failure, his cardiologist breezed in for a few minutes suggested that he was fit as a fiddle and simply needed an ablative surgery on a portion of his heart so his pacemaker would take over, assigned a replacement surgeon to do the surgery and then breezed out on her way to a pre-scheduled vacation. The surgery was performed at a cost to insurance of about $100k dollars, he was placed in a nursing facility where he declined for about 3 days and then 15 minutes before he died they finally called me and suggested “hospice care”. By the time they returned with a dose of morphine for him he was already dead.

      That surgery made someone a lot of money no doubt.

      • Shravan says:

        This is really sad, Daniel. Is it possible to discuss alternatives to surgery in the US with the doctors, when the patient is at such a late stage in life? Or do you have to do what they say?

        • In the end, it was my grandfather’s call. In general the patient makes the decisions, but the doctors provide the information that they use to make them. And more treatment equals more billing. It varies from doctor to doctor, but most are very reluctant to discuss end of life even with a 92 year old.

          • Robert Matthews says:

            All this is chillingly similar to the methods of financial advisors: the field is complex, they’re the “experts”, we’re supposed to trust them – but they clearly have conflicts of interests.

            • Shravan says:

              What worked for me was an independent financial advisor. I used to consult my bank’s “advisor” for several years, until I finally realized his main goal was to convert his experience and my money to the bank’s money and my experience. Can’t one call in an independent consultant to do the same in medicine?

              • Shravan says:

                But I must add that I have so far failed to find an independent consultant for my medical problem of deciding whether to get a kidney transplant or not. It’s obviously not a thing, unlike in finance (at least in Germany, one can find a certified financial analyst without any affiliation).

            • The thing about Financial Advisors is that for the most part things are really simple, and the advisors are there trying to make it seem complex so you’ll rely on them and buy their stuff. Since the advent of Exchange Traded Funds in around the year 2000, there’s no excuse for the existence of financial advisors really.

              https://www.ishares.com/us/

              You can build a fully balanced portfolio by looking at just HDV, IWV, IWM, LQD, CIU, ICF

              I’m not saying you can’t do a little better by knowing a little more, but you could do a lot worse than just buying some mix of those based on your age, and ignoring everything every professional has to say about what special funds and instruments they offer.

          • Martha (Smith) says:

            Yes, it does vary from doctor to doctor.

            An example where the situation went in a direction that seems to me much better: A friend’s mother had congestive heart failure. When living alone got too difficult, she moved into an assisted living facility, where she got more assistance as she needed it (initially, just things like meals, housekeeping, and assistance with oxygen at night; gradually increasing to help with bathing and other things, to the extent that her long term care insurance covered everything.) After moving into assisted living, she had far fewer trips to the emergency room, and, until the last one, none requiring hospitalization. The doctor said after the last one that there was no more he could do for her heart, just give medication to make her more comfortable. So after one night in the hospital, she went back to her assisted living facility, which was at that point her home, and lived for a few more weeks — collapsing after dinner on her birthday (90 or 91), after a day filled with phone calls from relatives and friends wishing her a happy day. She had signed a Do Not Resuscitate order, so that was the end — a pretty good way to go, in my opinion.

  12. Thomas says:

    Funding is one thing but the real smoking gun is the lax regulation in federal law as “enforced” by the FDA. A text analysis of federal regulations by government agency (published in a paper by Goldschlag and Tabarrok titled “Is Regulation to Blame for the Decline in American Entrepreneurship?” Ungated copy available here http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2559803) documents the dearth of regulatory stringency at the FDA (see Table 1, the FDA has the lowest level of stringency of any agency).

    This is a tribute to the success of pharma industry lobbyists in gutting federal statutes aimed at reining in industry malpractice.

    • Keith O'Rourke says:

      Thomas:

      This seems to me to be a very poor measure of what happens in regulatory authorities – “To measure
      regulatory stringency, Al-Ubaydli and McLaughlin (2015) comb the CFR and
      count the number of restrictive terms or phrases including “shall,” “must,”
      “may not,” “prohibited,” and “required”. “

      This does though seem like an interesting data source for text mining ” RegData provides an annual industry-level
      measure of regulation that is based directly on the text of the Code of Federal
      Regulations.”

  13. Keith O'Rourke says:

    I should thank Andrew for generating all the interesting comments and views generated on Ioannidis’ letter.

    Ioannidis of course is biased as I will be in what I add to this discussion. I do think he painted a slightly overly bleak picture – but only slightly.

    I worked with Dave Sackett early in my career and one of my biggest disappointment was to get an email from him to hold off on making flight arrangements to be the statistical collaborator in residence for first retreats in 199 held at Trout Research & Education Centre at Irish Lake, Canada, for aspiring clinical epidemiologists. Unfortunately that fell through. I also worked with Iain Chalmers (who later co-founded the Cochrane Collaboration) when I was in Oxford.

    Largely I think Ioannidis was just describing how people behave (badly) under difficult constrained conditions when there is little to no real oversight. The comment attributed to Chalmers does sound like the sort comment he often made (glass half full) but I doubt he made it about Ioannidis.

    The bad behavior Ioannidis discussed applies to most groups, and I think very much to the Cochrane Collaboration. I am sure Chalmers is aware this as I know he intervened when the editors of the Cochrane Handbook and the Statistical Methods Group repeatedly edited out any references to my publications even when others were suggesting they be included. He insisted that this one 1989 publication of mine “Meta-analysis in medical research: Strong encouragement for higher quality in individual research efforts” be referenced. (it was discussed here http://statmodeling.stat.columbia.edu/2012/02/12/meta-analysis-game-theory-and-incentives-to-do-replicable-research/ ).

    It was nice to see Ioannidis comment on retraction watch that he was hopeful as we should not lose sight that there is a percentage out there are are persevering with great effort to do and encourage good research. “JI: The right ideas are there, and there are many superb scientists and clinicians who want to do the right thing, so I am always cautiously hopeful. We should keep trying.”

    It was even nicer to see these comments he also made there “Systematic reviews may sometimes be most helpful if, instead of focusing on the summary of the evidence, highlight the biases that are involved and what needs to be done to remedy the state-of-the-evidence in the given field.”

    That was the main thesis of that paper of mine in 1989 and though I am arrogant enough to point that out, I will not be as arrogant as Ioannidis and draw attention to the fact that his comment in 2016 is evidence that my 1989 paper has yet to have had any real influence justifies my declaring myself a failure ;-)

Leave a Reply