Sander Greenland points us to this news article by Christina Jewett, “National Academies’ Report Took Pharma-Friendly Stance After Millions in Gifts From Drugmakers”:
To several U.S. senators, it looked wasteful, even outrageous. Every year, taxpayers pay for at least $750 million worth of expensive pharmaceuticals that are simply thrown away. Companies ship many of the drugs in “Costco”-size vials, one lawmaker said, that once opened usually cannot be resealed or saved for other patients. Yet pharma gets paid for every drop.
So Congress turned to the prestigious National Academies of Sciences, Engineering and Medicine for advice, given its reputation for “independent, objective reports” on such matters. The national academies’ influential report, released in February, struck physicians who’ve tracked the issue as distinctly friendly to Big Pharma. It advised against an effort to recoup millions for the discarded drugs. It concluded that Medicare should stop tracking the cost of the drug waste altogether.
Yet the report left out a few key facts, a KHN investigation has found.
Among them: One committee member was paid $1.4 million to serve on the board of a pharmaceutical corporation in 2019 and in 2020 joined the board of a biotechnology company that lists government “cost containment” efforts as a risk to its bottom line.
Another committee member reported consulting income from 11 to 13 pharmaceutical companies, including eight that Medicare records show have earned millions billing for drug waste. His pharma ties were disclosed in unrelated publications in 2019 through this year. . . .
But:
In a statement emailed to KHN, the national academies said the two members with undisclosed board and consulting roles had “no current conflicts of interest during the time the [drug waste] study was being conducted” from January 2020 through February.
What??? The committee member joined the board of a biotechnology company that lists government “cost containment” efforts as a risk to its bottom line—-and that wasn’t a “current conflict of interest”? Ahhhh, maybe I see. The study was conducted in Jan-Feb 2020, so if the board member didn’t join the company until later in the year, it’s not a “current” conflict. In the other direction, the member served on that phrarma board in 2019, but that was before 2020. So, massive conflict before, massive conflict after, but no conflict during those two months.
Not a good look, National Academy of Sciences.
And this statement from a spokesperson for the academies:
Protecting the integrity, independence, and objectivity of our study process is of the utmost importance to the National Academies.
I don’t think so, and I’ll get to why I don’t think so in a bit.
But first some more background, again from Jewett’s article:
Before and while [an influential 2016 paper on drug waste] was making waves, physicians who would eventually be on the national academies committee were forging alliances with the pharmaceutical industry.
Dr. Kavita Patel reported earning a speaking fee in 2015 from the Pharmaceutical Research and Manufacturers of America, or PhRMA, of $5,001 to $15,000. She also accrued assets valued at more than $50,000 for her role as a pharmaceutical company board member, according to 2015 and 2018 disclosures filed with the Government Accountability Office.
Dr. Anupam Jena, who also served on the committee, wrote a 2018 article with staff members of PhRMA arguing that medications should be valued not for their actual benefit, but rather for the potential for innovation that comes with making new therapies.
What jumps out at me here is that neither half of the corruption equation seems objectionable on its own. I’ve earned between $5,001 to $15,000 as a corporate speaking fee, I’ve done corporate consulting. And, on the other side, the position that medications be valued for their potential for innovation is not unreasonable.
That said, I agree with the critics that there’s a big ethical problem here. So where is it coming from. I see two issues. First, as noted in the linked article, the conflicts of interests are hidden. The authors of this report being paid big bucks by pharma companies—that should be stated right up front, not kept as a secret to be found by enterprising reporters. Second, the involvement of the National Academy of Sciences, a quasi-governmental organization that is some sort of representative of prestige academia.
My connections
This story interests me directly for several reasons:
1. Pharmaceutical support. Over the years, I’ve received several thousand dollars of research support from pharmaceutical companies. I’ve spoken at pharma companies for free and for pay and I’ve worked with colleagues in the industry. I get as annoyed as anyone about the worst abuses in pharmaceutical research (fake experiments, hidden data, and all the rest). So it’s complicated. It’s an industry that does lots of things, including bad things that make lots of money.
2. Corporate support more generally. I’m in the habit of taking cash where it comes. I say no sometimes, but I say yes a lot too. Here’s a partial list. I feel pretty strongly about my own integrity, but if you don’t want to trust some of my views because of conflict of interest, that’s fair enough. It’s a price I pay for the $.
3. The National Academy of Sciences. From my perspective as an outsider, the NAS seems to me to have three functions:
(a) It produces the journal, PNAS.
(b) It’s a society that honors prominent scientists.
(c) It provides occasional advice to the government.
My connections are: (a) I’ve published in the journal, criticized papers in the journal, and a few years ago I got them to fix their slogan; (b) I know lots of people who’ve been honored by the academy; (c) I’ve given advice to some NAS panels.
It’s no surprise to me that NAS members have corporate support. Free money is something that comes to you when you become more prominent—it’s a gravitational kind of thing—so it makes sense that many of the country’s most prominent scientists have been, like me, willing to take some of the money that has been offered to them. I used to think I’d always want money and I’d always want to be on TV. I no longer want to ever be on TV again but I’m still accepting cash, so I can easily believe that others would too.
What I don’t fully understand is how the three functions of the NAS fit together. I get that NAS members are involved in the PNAS process, but a lot of this must have to do with the management of the journal. Thirty years ago, PNAS was pretty obscure. But somewhere along the way, it became a top journal. And it serves a useful role, no joke. In addition to being a repository for prominent scientists to promote their favorite junk research, it’s also place for publishing interdisciplinary work that could be hard to get published elsewhere, as with our paper on penumbras which borders political science and sociology. I don’t really know how PNAS achieved its prominence. I’m guessing it had some enterprising editors—but that just seems like a journal thing, not particularly connected to the National Academy.
Finally, I don’t think the three roles of the NAS—journal publisher, honorary society and advice-giver—go together so well. From one perspective, sure, if the government is getting advice from scientists, why not take advice from the best? The trouble is, the NAS is not the best scientists; rather it’s the most honored, which is not the same thing at all! I say this as a much-honored scientist myself. These organizations operate in the manner of links on a chain, with people nominating their friends or people they’ve heard of. So you’ll get some well-connected scientists, that’s for sure.
But that’s fine too: no filter is perfect, and it’s not so bad for the government to be getting advice from well-connected scientists. My problem is that the whole point of the honorary society is to say how wonderful its members are, which is strongly in opposition to the goal of a scientific journal, which is to explore uncertainty, and the goal of an advice-giving society, which is to . . . explore uncertainty.
The NAS’s contradictory roles
To me, the problem of that controversial NAS report was not the involvement of corporate-funded members—in this day and age, this would be hard to avoid—but rather that a key role of the NAS is to extol its members. The extolling role leads to PNAS publishing papers that understate uncertainty while favoring NAS members’ favorite theories, and it leads to NAS reports that dismiss concerns of corruption and bias. To even bring up the possibility of bias is a contradiction of the NAS role of saying how goddam wonderful its members are.
Countervailing power
The bad news is that the NAS produced that report full of unacknowledged conflicts of interest. The good news is that the above-linked investigative article by Christina Jewett came out. I don’t know how much attention it received, but it did at least reach Sander Greenland, and then me, and now all of you. Jewett’s article was published by . . . KHN. What’s that? I clicked to find out:
KHN (Kaiser Health News) is a national newsroom that produces in-depth journalism about health issues. Together with Policy Analysis and Polling, KHN is one of the three major operating programs at KFF (Kaiser Family Foundation). KFF is an endowed nonprofit organization providing information on health issues to the nation.
KHN reports on how the health care system — hospitals, doctors, nurses, insurers, governments, consumers — works. In addition to this website, our stories are published by news organizations throughout the country. Our site also features daily summaries of major health care news. . . .
Kaiser is a large health maintenance organization . . . ummm, according to Wikipedia, it’s an “integrated managed care consortium.” Anyway, here’s my point. Pharma is big business. Health care is big business. In many ways, pharma and health care are natural allies, but one place where they differ is drug costs. The big business that is Kaiser is a countervailing force to the big business that is pharma.
Countervailing power is a good thing. It would be bad news if the folks at the Kaiser foundation were like, “Hey, we’re rich businesspeople, pharma’s rich businesspeople, let’s not rock the boat, let’s focus on the common enemy and fight to lower taxes and business regulation.” Or if they were like, “Hey, health care is a science-based field, the NAS promotes the prestige of science, so let’s not rock the boat, let’s work together to increase science funding and not do anything to make scientists look bad.” OK, maybe they do this sort of thing sometimes, and that’s cool too, that’s part of their story. But not all the time. Kaiser is willing to rock the boat, they’re annoyed enough about overpaying for drugs that they’re willing to blow the whistle, they’re annoyed enough about scientists shilling for shillings that they’re willing to call them on it.
Remember that famous Walter Matthau saying that poker “exemplifies the worst aspects of capitalism that have made our country so great”? That’s kind of what I think here. It would be so easy for the Kaiser foundation to play by gentleman’s rules and not do anything to upset their fine colleagues in pharmaceuticals or the esteemed members of the National Academy of Sciences. But instead they are motivated by . . . not greed, I think, maybe “annoyance” is a better word . . . they let their annoyance get the better of them and they lash out. They have a little tantrum, and in doing so they play an absolutely necessary role, a source of countervailing power. They’re comfortable enough not to have to do this, but they don’t like overpaying for drugs and they don’t like it when a bunch of stuffed shirts with Ph.D.’s try to justify it. Good for them, and it’s a good thing that there’s some economic competition in the world, as well as a stable political system by which they can offer such criticism without fear of getting bombed or sued out of existence.
Let me emphasize . . .
I don’t, repeat don’t, think of the pharmaceutical industry or the National Academy of Sciences as the bad guys. I use pharmaceutical industry products all the time, I’ve worked with many colleagues from different pharma companies, I’ve even spoken to pharma groups for free. Similarly, I know lots of excellent NAS members, I’ve published in PNAS and learned a lot from articles by others published in that journal. It’s not at all a criticism of these fine people and organizations to say that these institutions, like any other, can have bad sides, hence the need for internal and external checks on their power and activities. The Kaiser foundation serves as one of these checks, and that’s a good thing. And I also appreciate the aspects of our economy and society that allow these checks to work, which can enable all the good people in pharma and science to do even better work.
P.S. John Kenneth Galbraith doesn’t get full respect nowadays but I’m with Mankiw that he’s one of the great economists of the 20th century. As well as having many good ideas (such as countervailing power, the topic of the above post), Galbraith was also involved, less successfully, in politics and punditry, and it seems that when he wrote about the academic field of economics he did not show sufficient respect for some of the other work in that field. I can see how this could’ve annoyed many of his colleagues, but from the current perspective, many decades later, I feel like we can look past the various lowlights of Galbraith’s career and focus on his many positive contributions. In any sensible world, the concept of countervailing power would be considered much more important than, say, the concept of rational addiction.
P.P.S. It seems that I got the KHN story wrong. There is no ongoing relationship between Kaiser Permanente (the health-care consortium) and KFF (the foundation that funds Kaiser Health News) and there hasn’t been for decades, so for better or worse this does not seem to be an example of countervailing power in economics but rather a more direct example of the value of the free press, in this case funded by a nonprofit foundation rather than from the usual subscriptions-and-advertising model. Overall I think I’d be happier if this had been an example of countervailing power in a capitalist system, because that would seem more scalable and robust than relying on the good fortune that the Kaiser foundation got organized to do this sort of thing. Also I’m bummed because the HMO-vs.-pharma angle was cute, and now I’ll have to retract that Galbraith praise. Countervailing power is a pretty idea, but maybe plain old political ideology is what matters now.
Note that Kaiser Health News and Kaiser Permanente (the “integrated managed care consortium”) are not the same thing. When I received an interview request from KHN, it came specifically with the caveat “Kaiser Health News is a national nonprofit news service affiliated with the Kaiser Family Foundation (and no relation to Kaiser Permanente).” They were founded by the same rich industrialist, but no longer share leadership or board members. So KHN’s reporting is not motivated by Kaiser Permanente’s annoyance at medication prices.
Alex:
Sure, but I was assuming there’s still some connection with regard to worldview?
Kaiser Health News does a fair amount of good investigative reporting on health issues. I read their stories regularly, and don’t see a health care industry worldview.
John:
OK, maybe I should celebrate the countervailing power of the nonprofit and journalism sector of our society. One think that strikes me sometimes is how thin this sector can be. Often an important story is only told by one organization or one news organization. If there were no Kaiser Family Foundation, would the above conflict of interest ever have been reported? It seems pretty damn clear that the National Academy of Sciences wouldn’t have disclosed it on their own.
Wikipedia:
“Kaiser Permanente is one of the largest nonprofit healthcare plans in the United States”
“KFF (Kaiser Family Foundation), also known as The Henry J. Kaiser Family Foundation, is an American non-profit organization” [KHN is part of KFF]
Both organizations are non-profits and while they are no longer associated with one another, they share the same roots and to a first glance appear to have specifically been founded as “countervailing power” to the for-profit health care industry.
But while it’s always sensible to eliminate waste, tracking $750M waste in prescription drug costs **is** worth questioning. In 2020 Americans spent $360B on prescription drugs – thus 2% is wasted. That’s equivalent to 10 minutes in an eight hour day: an extra trip to the restroom and a short chat with a colleague.
There’s lots of “Kaiser” stuff around, all of it tracing back to Henry Kaiser but now independent.
There’s a rather odd National Park — the official name appears to be the Rosie The Riveter Home Front National Park — that sort of winds its way along the waterfront in Richmond, California. Among other things, there’s a waterside path that connects a park to the small but nice Rosie the Riveter museum. At the park and along the path there are interpretive signs that tell what work and home life were like during WWII when the whole area was occupied by the Kaiser Shipyards, which made Liberty Ships and then Victory Ships. Every now and then there’s a mention of some social innovation (or at least unusual feature at the time) introduced by either the Kaiser corporation or by Henry Kaiser himself: free day care (or maybe it was $1/day or something) so moms could work at the shipyard, hiring black workers to work alongside whites, free health care for workers injured on the job, and so on.
It seems possible in principle that all the great stuff about Kaiser has some sort of unsavory basis, like Kaiser Permanente donated a bunch of money to the park or something, but I really don’t think so. Everything I’ve seen about Henry Kaiser just makes him seem like a heckuva guy, always (or at least sometimes) thinking about the welfare of his workers and others in the community and putting some thought, time, and money into them rather than just trying to maximize profits.
No particular relevance to this post, just thought I’d pass this along.
There is a reason that your mother told you not to take candy from strangers. Now clearly, the vast majority of people are good and would enjoy giving a piece of candy to a kid, but….Pharma companies are sophisticated for-profit organizations. A large cap ETF from Vanguard has grown 15% over the last ten years; an expenditure by a corporation must strive to beat that return. Promotional things like dinners, swag, speaking fees, and conference trips to resorts are undertaken to beat the returns on some passive investment like an ETF. Yes, I went to dinners and conferences and accepted fees for speaking, but I became very wary. The reps are very charming, but they are not my friends. Dollars spent by corporate entities are intended to improve the bottom line. Money buys influence, and I was vulnerable to the dollars and the flattery that comes with it, and so is everybody else.
I had the opportunity in a couple of different lawsuits to read the internal messages and slides of pharmaceutical rep teams. Ugh. They seem more brainwashed than openly venal, but, ugh.
NAS/NRC committees recruit rather broadly. I served on one in the 90s, and I’m about as unconnected as humanly possible. I could just as well live on a desert island. I appreciate that their mandate, respected at least some of the time, is to assemble as diverse a panel as possible.
From the diversity point of view, I think it’s OK to have *some* pharma representation on a panel like that, but of course it should be absolutely transparent. This is important, i.a., for the passages in the report that reflect minority viewpoints; you want to make it clear where they’re coming from. And if it isn’t transparent there is less defense against having a sectional perspective (e.g. what’s good for pharma) outweighing the more general public one.
I have had so many experiences with the grad students actually doing the work and writing papers saying “they did what they needed to survive”, I consider these industry conflicts of interest a joke. There is always a conflict.
Then there is the peer review problem. When someone well-connected and prestigious publishes, the work is actually more likely to be fatally flawed than a publication from a nobody. Simply because they are less likely to get called out on it.
If you are going to use these heuristics (conflicts, prestige) instead of critically assessing the science (which is what you should be doing… nullius in verba and all), you are better off using the opposite.
Anoneuoid –
> When someone well-connected and prestigious publishes, the work is actually more likely to be fatally flawed than a publication from a nobody.
I’m wondering if you have any way for me to evaluate that claim other than just taking your word for it.
It is well known amongst people who actually work in research and try to replicate these studies. You can check out the various reproducibility projects (e.g., psychology, cancer, spinal cord injury) where the rate of replication for highly influential papers is worse than people coming up with an idea than deciding the result via a coinflip.
For people unable or unwilling to critically assess the evidence and logic, the solution has always been to wait for a surprising prediction coming true, or impressive feat preformed that you can verify yourself.
This goes back to Archimedes pulling a ship out of the harbor on his own to prove he knew what he was talking about. Probably back earlier, that is just the earliest example. Another example of that is Halley’s comet, which was what finally lead to widespread acceptance of Newtonian mechanics.
https://www.hellenicaworld.com/Greece/Technology/en/Syracusia.html
> . You can check out the various reproducibility projects…
I’m aware that some of those projects show high profile studies by well-known researchers that have been found to have fatal flaws.
But I’m not aware of research that shows some kind of systemic, positive correlation between greater “connection and prestige” on the part of authors and higher likelihood of fatal flaws (or a positive correlation to the fatality of flaws, however that might be measured).
I’d appreciate any links you might offer.
I didn’t understand your reference to Archimedes and Halley’s but I also don’t think that’s its particularly relevant to my question.
To go on my own tangent – while people talk a lot about a “replicability crisis,” and while I don’t doubt that the existence of perverse incentives creates pressure to publish material of questionable value, I haven’t seen quantified evidence that a higher % of publications recently are flawed. More flawed studies in number doesn’t really seem like a “crisis” to me if the number of flawed studied hasn’t grown in greater proportion than the number of studies published overall.
In regard to your tangent. I’m not sure what “higher” or “grown” means in this context, but 83% of multi-study articles related to Psychology in Science being flawed feels like a “crisis” to me:
https://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0114255
Greg –
> I’m not sure what “higher” or “grown” means in this context,
It seems to me that for it to be a “crisis” the situation has to be quickly growing worse than it’s been in the past.
How do we evaluate whether that’s true in this context? Is the scientific information that we’re depending on becoming significantly more unreliable than it’s been in the past? As a result, is there some discernible impact on society?
My general sense is that as a society we are increasingly benefiting from the industry of scientific research. Even in psychology, despite much erroneous science, we know more than we did in the past, so I don’t have a sense of “crisis.” That isn’t to say that there aren’t identifiable problems. Nor, does that in itself say whether we’ve hit some kind of deflection point where we’ve reached a point of diminishing returns. But I’m curious how to measure the costs of erroneous science against the benefits, or even whether there’s some kind of evident trend in the ratio.
> but 83% of multi-study articles related to Psychology in Science being flawed feels like a “crisis” to me:
That’s an interesting paper, with concerning implications. I see that there’s some criticism out there of this particular application of the “excess success” paradigm, but I’m curious more generally of a critique of the paradigm. From my viewpoint, while certainly problems have been highlighted by the initiatives to evaluate the state of science through replication projects, there’s also methodological problems inherent in those projects. The TES seems to be an interesting alternative way to gain some insight but (not being statistics literate) I have no way of assessing its usefulness.
But I thought the replication rate was statistically indistinguishable from 100%
Just ask him for the clippings!
This is just a toy simulation which is probably not what you are looking for: https://royalsocietypublishing.org/doi/10.1098/rsos.160384
The short version of the paper is: “As in the real world, successful labs produce more ‘progeny,’ such that their methods are more often copied and their students are more likely to start labs of their own. Selection for high output leads to poorer methods and increasingly high false discovery rates.”
Al –
Thanks. Looks worth spending some time with.
Back in the 90’s I did some consulting work for a startup that was developing a new medical device. They were honest people–they never pressured me. My contract stipulated that I did not have to submit my publications to them for prior review. But they paid me handsomely, wined and dined me, and gave me travel opportunities to nice places. About a decade after that relationship came to an end, amicably, I had occasion to review the article I had published about the work I did for them. It was a cost-effectiveness analysis. Cost-effectiveness analyses have highly ramified gardens of forking paths that biomedical and clinical researchers cannot even begin to imagine. I saw that at virtually every decision point in designing the study and in estimating parameters, I had shaded things in favor of the device. Not by a large amount in any case, but slightly at almost every opportunity. The result was that my “base case analysis” was, in reality, something more like a “best case” analysis. Peer review did not discover any of this during the publication process, because each individual estimate was reasonable. When I wrote the paper, I was not in the least bit aware that I was doing this; I truly thought I was being “objective.”
It was a sobering experience for me to revisit that paper. I have taken no private consulting money related to things I do in my day job since then. It is fine for people to work for industry. It is fine for other people to have careers in policy analysis and technology assessment. It is not at all fine for the same people to do both. We need a firewall between the entrepreneurs and the evaluators.
The involvement of those people with obvious conflicts of interest, including the ones with “past” and “future” conflicts of interest, in the NAS study is simply inappropriate. It is unethical behavior on the part of those individuals, and a grotesque failure of judgment, diligence, or both, on the part of the NAS study to have allowed them to participate. Perhaps they could consider adopting standards similar to what the National Institutes of Health applies to grant proposal reviewers. https://search.nih.gov/search?utf8=%E2%9C%93&affiliate=nih&query=conflict+of+interest+reviewer&commit=Search. Those standards, although they are, in my opinion, pathetically lax, are clearly light years ahead of the NAS.
This business of self-dealing by professionals is not a trivial matter. Apart from the direct consequences it has in policy making, the public has become aware that our elites are corrupt. It exacerbates all of our country’s social and political problems because nobody believes anybody else about anything, and there isn’t much reason that they should.
I think you got this just right! Getting to like your clients puts you on the slippery slope.
Thanks for sharing this.
Clyde –
Thanks for your honest insight.
> It is fine for people to work for industry. It is fine for other people to have careers in policy analysis and technology assessment. It is not at all fine for the same people to do both. We need a firewall between the entrepreneurs and the evaluators.
To play devil’s advocate, I suppose there might be a counterargument that the entrepreneurs represent an important (highly motivated) group with a unique perspective and perhaps skillset.
Do you think it’s possible that crossover has some value, but that it needs to be in a modest proportion overall so as to not have too much influence?
Clyde:
Worked on conflict of interest for NEJM paper in the mid-1990s. The senior author of that paper (Allen Detsky) mentioned he had updated his views (that disclosers would suffice?) and I think this the paper on the updated view which is behind a paywall for me :-(
From the $80 hamburger to managing conflicts of interest with the pharmaceutical industry – https://www.bmj.com/content/365/bmj.l1939
But the excerpt might suffice “During the 1980s and 1990s AD engaged with drug companies as a paid consultant and speaker at symposiums and advisory board meetings. The endeavour was intellectually and financially rewarding, and to him the conflict of interest seemed manageable. This changed in the late 1990s when he embarked on a programme of research that led to some of the earliest reports documenting how the drug industry influences the interpretation of medical evidence and guidelines. Since that time he has had no relationships with industry. Moreover, he now “sees” industry influence in almost all facets of patient care, medical education, clinical research, and even certification exams (in which the correct answers are based on pharmaceutical funded guidelines).”
RE: “Moreover, he now “sees” industry influence in almost all facets of patient care, medical education, clinical research, and even certification exams (in which the correct answers are based on pharmaceutical funded guidelines).”
—-
Astounding to learn in mid 90s. Is there a place for conflict of interests free experts? I gather the Lown Institute is venue for such experts.
Clyde:
This is an interesting story. I don’t think this particular issue has arisen in my business consulting because the work I do is generally for internal decisions and not to influence external policy. But similar concerns arise in my non-compensated research work! If I help a researcher out on an analysis, and I end up liking the person, then, yeah, there’s lots of motivation to get things to work out. I agree that profit and $ add another dimension, but it seems to me that just about any collaboration has the concern that I can influence the results to go a certain way.
Clyde’s recommendation that NAS adopt NIH conflict-of-interest standards reinforces the problem Andrew identified. Congress wanted a scientific report, but instead of going to an organization that does science, they went to an organization of scientists. There’s a reason the Pentagon doesn’t contract with, uh, whatever the national society of aeronautical engineers is, to design fighter jets. I mean, imagine if Congress asked the ASA to put together a coherent, conclusive report on p-values!