A tale of two epidemiologists: It was the worst of times.

A couple of commenters pointed me to the story of John Ioannidis and Gideon Meyerowitz-Katz. David Gorski tells the tale. Ioannidis still seems to be dealing with the after-effects of his extrapolation last year that there might be 10,000 coronavirus deaths in the United States. This was just a back-of-the-envelope calculation, and as Gorski says, there’s no shame in being off by “by a factor of ten or even 50.” The problem is that Ioannidis didn’t seem to be able to let go of his initial downplaying of the risk. I agree with Ioannidis that so many of the deaths are elderly people so the loss of qualys is much less than from a flu that kills the same number of younger people, but still it is a lot of mortality and morbidity, not to mention all the disruption of people’s lives (much of which cannot be attributed to “lockdowns,” given that at this time last year people were pulling their kids out of schools, cancelling trips, etc.). It seems in retrospect that we could’ve saved a lot of lives by just all wearing masks, but it’s not like Ioannidis said, “only 10,000 deaths if we all wear masks”; no, it was the opposite, he said that if we’d just been living our lives normally we’d barely have noticed coronavirus at all.

This 10,000 extrapolation was based on an underestimate of the fatality rate of coronavirus and also a guess that only 1% of the U.S. population would get infected. The odd thing is, the very same March 2020 article with that calculation also said, “we don’t know if we are failing to capture infections by a factor of three or 300,” and a month later Ioannidis was a coauthor of a papers estimating that some counties in California already had infection rates over 2% in early April. I had skepticism about the 2% estimate—not that I thought it was too high, necessarily, just that I thought the estimate was less certain that the authors of that study had claimed—but, in any case, there was a lot of incoherence in the position that (a) infection rates were much higher than reported, but (b) only 1% of the U.S. population would get infected. I think we can all agree it’s been more than 1% by now.

Again, getting a forecast wrong is fine. It’s the nature of probabilistic forecasts that there will be uncertainty. We forecasted that Joe Biden would win between 259 and 415 electoral votes. He actually got 306, so that was in our range, but it was lower than our point prediction. I can’t fault Ioannidis for considering 10,000 deaths as a possibility; as Gorski says, the problem came later with a failure to fully reassess his models in light of the forecast’s (inevitable) errors. This is not a problem unique to any particular epidemiologist; similar issues arose in the other direction with Imperial College epidemiologist Neil Ferguson. Again, the point is not that Ferguson was unreasonable, just that making a prediction in a time of uncertainty is step 1 of a two-step process. Step 2 is going back afterward and assessing how your forecast did.

I wonder if some of these problems can be attributed to Jensen’s inequality. When forecasting the death toll from a pandemic, there are literally orders of magnitude of uncertainty. Some of this relates to the feedback in the stochasticity of the spread—an epidemic can die out or it can explode, and if it explodes it can become uncontainable—and some of it is just uncertainty about how fast it will spread and how many people it will kill. Suppose the forecast of deaths for a particular pandemic is in the range {100, 1000, 10,000, 100,000, 1,000,000}, with the lower and higher ends being less likely. To fix ideas, suppose the probabilities of these 5 outcomes are 5%, 20%, 50%, 20%, and 5%, respectively. Due to the practical difficulties of diagnosis and reporting, it might be that the three outcomes on the low end will be hard to tell apart, but let’s not worry about this here. In any case, the point is that the best estimate or most likely outcome here is 10,000, but the expected number of deaths is 75,000. And major economic and social damage could occur at the very high end. So it can make sense to prepare for a million deaths and to forecast an expected 75,000 deaths, even if 10,000 is the most likely. And all of that is with the probabilities known. With unknown probabilities, everything is that much harder. And then, after the fact, if the death toll is only 10,000, lots of people can run around accusing the “75,000” people and the “1,000,000” people of being unnecessary alarmists. My point here is not that it was wrong for Ioannidis and others at the beginning of the pandemic to consider low-end possibilities. Rather, I just want to emphasize the inherent difficulties of handling this sort of variation over orders of magnitude, and the importance of going back and understanding where past guesses were wrong.

P.S. There’s also some discussion about Ioannidis criticizing Meyerowitz-Katz for wearing big glasses and having a cat. That wasn’t cool for Ioannidis to do that, but I think that’s pretty much separate from the issues of statistics and epidemiology discussed above. We all do things that aren’t cool sometimes; Ioannidis just had the misfortune not to have an editor on that article who could’ve suggested he remove those passages.

237 thoughts on “A tale of two epidemiologists: It was the worst of times.

  1. It’s somewhat worse than you depict. Immediately when that first article came out, I had an email exchange with an epidemiologist pointing out that JI had combined a low-ball IFR that required assuming a high cryptic infection rate with an ultra-low limiting infection rate, the opposite limit. Either one could conceivably have been right, but the combination was essentially impossible.(You make the same point above.) Combined with other tendentious parts of the same paper, it’s clear this was not just an ordinary erroneous estimate. JI had some unstated agenda.

    • There’s more that goes beyond merely falling victim to orders of magnitude in uncertainty and lacking an alert editor.

      The worst, imo, is a fundamental epidemiological and scientific and logical error: Ioannidis went on national TV to promolgate an generalized IFR estimate that was derived by extrapolating from non-randomized and unrepresentative convenience sampling without even adjusting post-stratification for common predictors of health outcomes like race/ethnicity. I have yet to have someone explain to me how that is even remotely acceptable epidemiological practice.

      There’s more, that I’m not entirely sure is less aggregious. For example, when talking about the uncertainty in quantifying COVID deaths, Ioannidis focused on uncertainty in one direction only – the potential mixing of “died with” vs. “died from” COVID – and ignores the uncertainty that might have led to undercount. Another example is when talking about the Danish mask study, he ignored the significant limitations in the finding (see Andrew’s post on the topic) to focus only on a conspiratorial framing of the publication process.

      With Ioannidis’ science on COVID, there has been a uniform alignment in relation to the uncertainties – towards limiting the seriousness of the pandemic. I wouldn’t assume an “agenda,” as that would require mind-reading – but at some point when viewed in the context of more than one projection that were off by orders of magnitude, there is a systemic pattern in his direction of error that implies more than just a random distribution within the range of uncertainty.

      Then you mix that with his obvious and explicit affiliation with a particular group of policy advocates, and a consistent leveraging of political media to push his policy preferences.

      And then you add in his bizarre use of the scientific literature to launch a cheap personal attack. In the very least, this all undermines his self-cleansing claims of “pure” science focus and no political interest, and gives his editorial framing more than a tinge of sanctimony.

      • Hi Joshua,

        RE: “There’s more, that I’m not entirely sure is less aggregious. For example, when talking about the uncertainty in quantifying COVID deaths, Ioannidis focused on uncertainty in one direction only – the potential mixing of “died with” vs. “died from” COVID – and ignores the uncertainty that might have led to undercount.
        —–
        John does go into more detail in his July 2020 article, that I cited earlier, as to why quantifying COVID deaths has been so challenging. He has noted, for example, that there can be substantial error rate. Heck even Sander Greenland has a post on this blog about this problem.

        https://pubmed.ncbi.nlm.nih.gov/1871957/

        A subset of medical experts have pointed to overdiagnoses and other such issues in medicine, the concerns of which have been aired in so many science and medical journals. John Ioannidis has presented his perspectives in many scientific fora. So ok he has a cautionary approach to projections made. That is one approach. The other side of it is that there have been many doomsday projections as well. Our propensity to align with one approach is determined by many heuristics and biases that we acquire through our lives. A good number of physicians have written about their disillusionment with medical treatments. It’s worth reading such critiques. The Evidence Based Movement has continued to echo degrees of disillusionment and frustrations.

        We may not ever come close to calculating an accurate IFR, unless we devise a testing surveillance public health program. Now that may be a possibility as a conwquence of Michael Mina’s efforts.

        The state of knowledge is itself in flux in so many endeavors. Also we know far less than we want to.

        • Hi Andrew,

          I was a member of Right Care Alliance, a project of the Lown Institute. Shannon Brownlee was its founder. I did not have an occasion discuss the article nor anything else about John Ioannidis at the time b/c I was too busy with some other projects. I was surprised that they knew each other as well as they did. I did know that John Ioannidis had given a keynote at the Lown Institute which was well received and that Shannon had co-authored an article which I haven’t read yet.

          I have not been actively involved with the Right Care Alliance for a year and a half.

          John has loyal friends and colleagues, as you and others who post here do. So I didn’t view the defense of John as a political maneuver but as one of enduring loyalty.

          It’s not my way to besmirch people with whom I disagree with or even dislike. I saw too much of that and it strikes me as toxic.

        • Sameera:

          I think loyalty isn’t what’s at question here but full disclosure by interested parties. If I write a defense of a long time collaborator readers ought to know this fact.

        • Sameera:

          Tha authors of that article misrepresented what I wrote and they misrepresented the reporting of Stephanie Lee, and then when I contacted them to ask them to correct these misrepresentations, they refused. Then a few months ago they did it again. Perhaps they could find a way to be loyal to their friends and colleagues while still writing the truth about others. Or perhaps they do not have the ability to thread this needle, and to them loyalty is more important than truth. In that case I think they’d do better to just not write about the topic at all, if they would otherwise find it necessary to misrepresent the actions of others in order to make their point. In any case, I’m glad that Scientific American called them on it. That showed integrity on the part of Scientific American’s leadership.

        • Hi Andrew,

          Sorry to hear that you were not able to resolve the misunderstandings. I know that can be very. frustrating. The progression of the situation seems so complex. Without John directly addressing the critiques, some of us can only second guess the behind the scenes dynamics.

          My brief experience of Shannon Brownlee [ don’t know Jeanne Lenzer] is that she is a straightforward person. I strongly doubt that she intentionally hid that the fact that she and Jeanne co-authored articles with John. In reviewing John’s keynote at Lown Institute, I would guess that Shannon and John share similar viewpoints about the challenge of conflicts of interests and constraints posed by the lack of reliable data in so many cases.

          .

          Scientific American added that disclosure right off the bat. So good for the magazine.

          I will reread that UnDark article if it is still available. I never heard of UnDark until the controversy erupted.

          And it’s unfortunate when people misquote or misinterpret our statements.

        • That Scientific American article was awful. I am the MPH student that the authors quoted out of context. JI argued in a European Journal of Clinical Investigation article, a day after his infamous STAT news Op-ed, that the US did not have enough reliable data to enact “draconian” public health measures. I simply thought we did with the closed dataset of the Diamond Princess Cruise Ship and the fact that Italy, a country one fifth the US, already had 10,000 COVID-19 deaths. That was on March 18, 2020!

          https://pubmed.ncbi.nlm.nih.gov/32293030/

          I am certain the Ioannidis defenders (Scientific American article authors) did not like that I said I thought JI’s position was irresponsible, reckless, and could cost thousands of lives, if policy makers considered it when making public health decisions.

        • Jim:

          The good news is that if you’re quoted out of context early in your career, you’ll know in the future not to trust someone just because they talk smoothly and are associated with a respected institution such as Scientific American or MIT or whatever. It’s awesome that Scientific American looked into it and flagged that misleading article. I’m still disappointed that Undark didn’t do the same.

      • To the extent that Ioannidis has an “agenda,” my assumption is that it is to do good science and save lives.

        It isn’t his agenda that I question. I question his science. And I question the degree to which he has allowed “motivated reasoning” to affect his science.

        Another example – one of many: his comparison of COVID to the seasonal flu: It’s almost as if he doesn’t understand the basic dynamics of compounding growth – which isn’t plausible.

        Let’s say that he’s right about the IFR of COVID. So then he says that it’s not that far away from the flu (in absolute terms). But the problem is that a small difference in IFR (coupled with a somewhat small difference in R0 in absolute terms), and you wind up, after months of unchecked growth, a huge difference in the number of deaths.

        But then we need to go further. From what I’ve seen Ioannidis almost always concentrates on deaths. He ignores the hospitalization rate, which is obviously important – and he ignores the much higher infection hospitalization rate from COVID when he compares COVID to the flu.

        As I mentioned above, he only talks about the uncertainties that might lead to an overcount and ignores the uncertainties that could lead to an undercount.

        He makes facile statements attributing negative outcomes to COVID without speaking to how such an assessment is based on counterfactual assumptions (that things wouldn’t have been worse absent interventions) for which he has very little evidence.

        There’s a long list of examples for how poorly he’s conducted science related to support his COVID activism. I don’t begrudge him wanting to be an activist. I wouldn’t expect someone to sit on their hands in such an important situation.

        But I’m rather shocked at how bad his science has been.

        • +1 and to a lot of the commentary below.

          Uncertainty in multiplicative risks, ipso facto supports stronger precautionary approaches than uncertainty in additive domains. This is the core insight of Taleb – although far from unique, and as others point out, he gets a lot of unnecessary mileage for making the same point in various ways. But it is still absolutely correct! And Ioannidis, for all his heavyweight background, still either doesn’t get it, or doesn’t want to.

          Moreover, like Joshua I fault Ioannidis for going out pre-emptively to press with his mistaken analyses (e.g. that Santa Clara study that took many of us 5-10 minutes to show its analytical flaws), and even sought out contrarian Covid Truther podcasts to share his point of view.

          I don’t think he has malign intent. I think he had a gut intuition that he couldn’t let go of, and runs in some Stanford circles who lean politically independent to conservative. Those guys (not necessarily him) got politicized and it may have leached off on him, but this is going into realms of speculation that are not necessarily helpful. So I’ll stop there :)

  2. >>> “My point here is … to emphasize the inherent difficulties of variation over orders of magnitude, and the importance of going back and understanding where past guesses were wrong.”

    OK, so looking back 13+ months at official US government guesses on coronavirus mortality — were they wrong ?
    If so, why ?

    And what is the margin of error (if any) in officially designating Covid-19 as the cause of death for hundreds of thousands of deceased people ?

    • Kam:

      Oh, yeah, the U.S. government response was terrible. See the above-linked article by Lawrence Wright for lots of details. To get more statistical, here’s an example of a government forecast that was clearly ridiculous at the time it was produced. What was particularly bad about this case was that the government official in question attacked his critics for not having “peer-reviewed scientific work and academic appointments.” As if having a government job, peer reviewed papers, and a fancy academic chair is more important than getting things right. I can’t stand that swaggering attitude. If these dudes had spent less time preening and more time telling us all to wear masks, maybe we wouldn’t be in the bad place we are today.

    • And what is the margin of error (if any) in officially designating Covid-19 as the cause of death for hundreds of thousands of deceased people ?

      We know George Floyd met the criteria for a covid death.

      Lots of people died early on due to inappropriate medical interventions, maybe they still are. I am not up to date on what they are doing to covid patients these days.

      Also stress and poverty are no longer considered to increase mortality rates apparently.

      Finally, I don’t know where Ioanndis got his 1% infected assumption, but I am sure he did not account for sending the covid patients into nursing homes.

      • Anon,

        Let me take these in order:

        1. As far as I can determine, George Floyd did not “meet the criteria for a COVID death”. https://www.medpagetoday.com/blogs/working-stiff/88141 for example. And https://www.cdc.gov/nchs/data/nvss/coronavirus/cause-of-death-data-quality.pdf I think you’re just wrong about this one. By extension, I think you’re wrong about what I think you’re implying, which is that there’s a significant overcount of COVID deaths. If you think COVID deaths are significantly overcounted — by enough to change the way we should think about Ioannidis’s projections and his certainty in them, which would require a huge, huge overcount — then I suggest you say “I think US COVID deaths are over-counted by more than 400,000 [or whatever you think the number is], and here’s why I think this:” and then list your reasons.

        2. “Lots of people died early on due to inappropriate medical interventions, maybe they still are. I am not up to date on what they are doing to covid patients these days.”

        When you say “lots”, I’m not sure whether you’re talking about a percentage or an absolute number. If 50K people died due to inappropriate medical interventions, that’s a lot in absolute terms — the number of U.S. combat deaths in Vietnam — but it’s less than 10% of official US COVID deaths at this point. Do you really think the number of COVID deaths due to inappropriate medical interventions is so large that it accounts for a substantial part of the difference between Ioannidis’s projections and the actual course of the pandemic? That seems to me to be a ridiculous suggestion, that something like 90-95% of COVID deaths are due to that. I don’t even think 20% would be a reasonable estimate.

        I realize there’s some interaction with hour first point: maybe you think COVID deaths are over-counted by a factor of 3, and of the remaining third, you think 80% of them are “due to inappropriate medical interventions”, so when you put these together, the count of COVID deaths that aren’t due to inappropriate medical interventions is only 40K or whatever. This still seems ridiculous to me. Is this really what you believe, and if so, could I trouble you to explain why?

        3. Stress and poverty… I have no idea what you’re saying there. It’s widely recognized that stress is harmful, and that poverty is associated with poor health and shorter lifespans for a variety of reasons. When you say these things are “no longer considered”, by whom do you think they are no longer considered? Or are you just saying “poverty” is not listed on death certificates, or something like that? It never was, as far as I know.

        4. “Finally, I don’t know where Ioanndis got his 1% infected assumption, but I am sure he did not account for sending the covid patients into nursing homes.”

        I agree that Ioannidis probably didn’t think in detail about what might or might not happen with elderly sick patients in hospitals and where they might be sent as the hospitals filled up with patients. For one thing, it seems that he didn’t think hospitals anywhere would be overwhelmed with seriously ill people. Indeed, his failure to imagine such a scenario is of a piece with his dismissal of the suggestion that things might get quite bad. But even if Ioannidis had considered the possibility that there might at least be local saturation of critical care capabilities, he was just trying to get an order-of-magnitude estimate and I doubt he would have thought things through in so much detail. So, as a criticism of Ioannidis this comment seems beside the point: the problem with his estimates, and his presentation of his thoughts, is not that he failed to consider how some locales might respond if they had more patients than they could handle.

        But I don’t think your comment is intended to be a criticism of Ioannidis. I’m not sure what it’s intended to be. To try to be extremely generous to you, I’ll assume you are trying to contribute to this discussion and you’re pointing out the difficulty in predicting how governments, hospital administrators, nursing home managers, etc., might respond to a pandemic. If that’s your point then: yes, sure, when Ioannidis was making his predictions a year ago it was very unclear how all aspects of society might respond to the coronavirus outbreak. But, logically, that should have made Ioannidis _less_ sure about his projections.

        Andrew generally discourages personal comments on this blog, but I’m going to put one here anyway:

        Anon, I think I am not the only one who finds your comments frustrating. You sometimes make genuine contributions to this blog and its discussions — I learned some things from you about Vitamin D, for example — and I hope you continue to do that. But often…far, far too often…your posts just seem pointlessly snarky. Here I’ve pretended that your snarky comments are intended to contribute to the discussion of how hard it is to forecast something like the course of a pandemic, and how to handle the resulting uncertainties, but I don’t think that’s actually the case. I don’t think you were trying to contribute in any way. Sometimes you just seem mad at the world and your response is mockery and derision. Hey, it’s the Internet, I’m well aware that there are always going to be trolls who want to get a rise out of people. It usually doesn’t bother me, but with you it does bother me because I know you’re capable of doing so much better.

        • Start with #1:

          DR. DEBORAH BIRX: So, I think in this country we’ve taken a very liberal approach to mortality. And I think the reporting here has been pretty straightforward over the last five to six weeks. Prior to that when there wasn’t testing in January and February that’s a very different situation and unknown.

          There are other countries that if you had a preexisting condition and let’s say the virus caused you to go to the ICU and then have a heart or kidney problem some countries are recording as a heart issue or a kidney issue and not a COVID-19 death. Right now we are still recording it and we will I mean the great thing about having forms that come in and a form that has the ability to market as COVID-19 infection the intent is right now that those if someone dies with COVID-19 we are counting that as a COVID-19 death.

          https://m.youtube.com/watch?v=blZpgra3XbU

          So, you are wrong. And if this policy changed at some point then all these charts looking at death rates over time are misinformation.

        • Anoneuoid –

          Intersting thst you’re taking the word of a politically appointed public health official as gospel. I wouldn’t expect that from you.

          Regardless, given that uncertainties about “COVID deaths” run in both directions – is your argument that in the US, overall there is a significant overcount?

          Part of the rhetorical playbook of many people who think that interventions have cost more lives than they’ve saved has been to address uncertainties related to counting “COVID deaths” in a very one-sided manner (mixed in with promoting conspiracy theories about how medical professionals have been falsifying deaths to hurt Trump and profit financially). Ioannidis has treated the uncertainties very selectively.

          No doubt, I’d guess that some number of people have been counted as dying from COVID where that designation could be fairly questioned. But I’d say the salient question here is how big that number is in comparison to the number running the other way – such as those who died at home or on the streets or in a nursing home but never had an attending physician, never took a COVID test, etc.

          I thought that Phil was asking you to be more specific related to that comparison.

          (Andrew – I’m hoping that’s not overstepping you’re request that I not belabor points of disagreement with Anoneuoid)

        • Anon,
          The only statements I can find about Floyd and COVID say exactly the opposite of what you’re saying: they say that the coroner determined that although Floyd tested positive for COVID it was NOT a contributing factor in his death. You may be right that Floyd’s death certificate lists COVID as a cause but you’ve presented no evidence that that’s the case, and it contradicts what I’ve seen.

          As for how COVID tests are counted: If you are genuinely interested in this issue, which now seems to me unlikely, you might start here: https://www.aamc.org/news-insights/how-are-covid-19-deaths-counted-it-s-complicated and follow the links.

          Here’s one section:

          The CDC uses different sources to post slightly different fatality figures.

          Death certificates provide the data for the agency’s daily updates of COVID-19 deaths. In mid-February, the total stood at 462,000.
          At the same time, the agency’s COVID Data Tracker reported 486,000. That data comes from the National Notifiable Diseases Surveillance System (NNDSS), which gathers information from state and local health departments when a disease is diagnosed in someone in a health care setting. (Estimates vary among media outlets and other organizations because they use various sources, including the CDC.)

          Why use two counting methods? The NNDSS provides “real time” awareness of fatalities linked to a disease, Robert Anderson, PhD, chief of the Mortality Statistics Branch at the CDC’s National Center for Health Statistics, says via email. He explains that it takes about two weeks longer for death certificate information to work its way up to the daily updates.

          “Ultimately, the official numbers will be based on the death certificates,” Anderson says.

          If you care to read about this, you’ll find that some states initially counted everybody who died with COVID as a COVID-related death, but then changed that policy…and went back and changed the numbers so that the time series would be correct. Here’s Colorado, for example: https://www.coloradoan.com/story/news/2020/05/16/colorado-changes-how-coronavirus-deaths-state-counted/5198485002/

        • You may be right that Floyd’s death certificate lists COVID as a cause but you’ve presented no evidence that that’s the case, and it contradicts what I’ve seen.

          I never said this. I said he met the criteria of dying with covid.

        • Anon, you said “We know George Floyd met the criteria for a covid death.”

          Yes, which was announced to world as dying with a positive covid test. Personally, I think it was unlikely he got included in the stats, but I have no idea how to tell.

          What states report and what the cdc reports is different, as you say they are changing the criteria and past numbers *somehow*, etc.

          Return to the original post:

          And what is the margin of error (if any) in officially designating Covid-19 as the cause of death for hundreds of thousands of deceased people ?

          We know George Floyd met the criteria for a covid death.

        • If you care to read about this, you’ll find that some states initially counted everybody who died with COVID as a COVID-related death, but then changed that policy…and went back and changed the numbers so that the time series would be correct.

          I hope readers will focus on this. You don’t even realize you are making my point for me.

        • You are asking us to focus on the point that states like Colorado went back and made their numbers more accurate?

        • Anon,
          I find it pretty much impossible to tell what your point is with any of this stuff. This is why I suggested that, instead of making a cryptic and apparently erroneous statement about George Floyd, you tell us why you think there is a significant over-count of COVID deaths, if that is in fact what you believe. Ditto for all of your other statements.

          Or, you can just keep being snarky and argumentative without contributing anything. You’re allowed to do that. I just wish you wouldn’t.

        • you tell us why you think there is a significant over-count of COVID deaths, if that is in fact what you believe.

          I never said this!

          This strawman stuff is getting so extreme on this blog.

          Can you share sources that account for these other factors?

          The answer is no. They all just set them to zero like the one andrew linked to in the OP. I am sorry that science is becoming uncomfortable for people but your choice is science or ignoring inconvenient info. Choose the latter and more people will die.

        • Choose the latter and more people will die.

          Not you of course, or me. We are largely irrelevant. But the collective choice.

          If you want to save lives, encourage sharing of accurate information so people can make informed cost-benefit based decisions.

        • No more. I appreciate discussions, but this is just making the blog less useful when you fill up the comment thread like this.

          Sorry about that, I do understand and didn’t see your other comment before all those posts.

        • Btw, this longstanding concern has finally been confirmed:

          Altogether, our results suggest that vaccinees probably do not elicit an early humoral response detectable at mucosal surfaces. They strengthen the hypothesis that some vaccines may not protect against viral acquisition and infection of the oral–nasal region, but may prevent severe disease associated with viral dissemination in the lower respiratory tract. Our results are in line with those obtained in nonhuman primates, in which vaccinated and then challenged animals display detectable viral loads in nasal swabs but not in lower airways37.

          https://www.nature.com/articles/s41591-021-01318-5

          So the vaccines do not induce mucosal immunity.

        • The paper you linked states:

          Our results indicate that B1.351, but not B.1.1.7, may increase the risk of infection in immunized individuals.

          I believe it’s been known for a while that there is no sterilization of the virus, just decreased load and, most importantly, protection from hospitalization and death.

        • Here we demonstrate that two recently emerging mutants in the receptor binding domain of the SARS-CoV-2 spike protein, L452R (in B.1.427/429) and Y453F (in B.1.298), can escape from the HLA-24-restricted cellular immunity. These mutations reinforce the affinity to viral receptor ACE2, and notably, the L452R mutation increases protein stability, viral infectivity, and potentially promotes viral replication. Our data suggest that the HLA-restricted cellular immunity potentially affects the evolution of viral phenotypes, and the escape from cellular immunity can be a further threat of the SARS-CoV-2 pandemic.

          https://www.biorxiv.org/content/10.1101/2021.04.02.438288v1

          Here they come…

          Still, I am pretty sure it can’t be as bad as last year unless some new poorly thought out intervention is added.

        • Let us go to point #2:

          When you say “lots”, I’m not sure whether you’re talking about a percentage or an absolute number. If 50K people died due to inappropriate medical interventions, that’s a lot in absolute terms — the number of U.S. combat deaths in Vietnam — but it’s less than 10% of official US COVID deaths at this point. Do you really think the number of COVID deaths due to inappropriate medical interventions is so large that it accounts for a substantial part of the difference between Ioannidis’s projections and the actual course of the pandemic? That seems to me to be a ridiculous suggestion, that something like 90-95% of COVID deaths are due to that. I don’t even think 20% would be a reasonable estimate.

          Fine, what is the percent due to that? Currently it is set to zero in most discussions, eg: https://sciencebasedmedicine.org/what-the-heck-happened-to-john-ioannidis/

        • Btw, I don’t mean forum speculations like 10-20% based on unclear methods. I mean is there a single published paper that tries to account for this?

          Can you share a single one?

  3. Joshua- We’re in agreement. But IIRC in that first paper JI did adjust for some characteristics of the cruise ship population, in particular age. So that lowered the estimated population IFR. He didn’t adjust for other variables, like not being too frail to go on a cruise, that would have raised the estimated IFR. Completely tendentious, as we both agree. Again, it was obvious at the time.

    • Michael –

      > But IIRC in that first paper JI did adjust for some characteristics of the cruise ship population, in particular age.

      Sure – but the very idea of extrapolating from a cruise ship population – and living conditions on a cruise related to spread of an infectious disease – to a general population in normal living conditions seemed like a scientific parody to me.

      > He didn’t adjust for other variables, like not being too frail to go on a cruise, that would have raised the estimated IFR. Completely tendentious, as we both agree. Again, it was obvious at the time.

      Yup. Yes, they did adjust for age and I’m not saying don’t look at the DP and calculate the IFR, but trying to say such a situation is generalizable, even with post-stratification adjustments for age, looks to my non-PhD-in-epidemiology-eyes as bizarre.

      But then he later extrapolated from the Santa Clara – obviously non-random – sample w/o even adjusting for such an obvious correlate with health outcomes as race/ethnicity? I mean what? And then went on a national TV campaign to promote that IFR and ridicule other scientists (for promoting “science fiction and projecting from inadequate data) as he said that COVID was basically as much of a threat to public health as the seasonal flu?

      • Hypothetically, had one of us had to issue our best forecast on the day JI did, how would you have gone about it?

        I would love to hear the alternative analysis with the constraint that you should put yourself ( best as you can) at that point of time sincerely: you are not allowed to even implicitly color your forecast with anything you know since then.

        I think this will be an interesting thought experiment.

        • Rahul:

          Speaking just for myself, I would’ve found it very difficult to issue a forecast, even a probabilistic forecast, given my state of knowledge at that time. And, as I wrote in my above post, I don’t hold it against Ioannidis that his published forecast (or estimate, or projection, or hypothetical scenario, or whatever you want to call it) was way off. It’s the nature of highly uncertain outcomes that point forecasts can be way off. I do think that Ioannidis could’ve done better at assessing where and how his forecast went wrong. For example, his projection of only 1% of the U.S. population ever being exposed to the virus was pretty much directly contradicted by the high-profile claims of the Stanford study he was involved in the very next month. It would’ve been appropriate for him to acknowledge that contradiction, work through its implications, and learn from the discrepancy.

        • Let’s assume no forecast is not an option on the table.

          Just for Fun, imagine your state of mind a year ago: what number would you have picked for say total deaths had you been forced to.

        • At the time, I would have looked at the estimates of IFR, assumed they were probably somewhat high due to under-reporting of mild/asymptomatic cases, compared that to past pandemics, and said (as I did say in April) that this would probably be comparable to 1957/1968, maybe a bit worse, but clearly less severe than 1918.

          Adjusting 1957 death rate to current population numbers would suggest about 200,000 deaths in the US and maybe 2.5 to 10 million worldwide.

          This is a very unscientific way of doing it, but doesn’t seem to have been much farther off than the serious professional estimates! (Too low for the US – though maybe not for the world depending on how quickly vaccination spreads, and whether the low per-capita deaths in the Old World tropics are real vs. an effect of underreporting.)

        • Andrew –

          > For example, his projection of only 1% of the U.S. population ever being exposed to the virus was pretty much directly contradicted by the high-profile claims of the Stanford study he was involved in the very next month.

          His mention of 1% was probabilistic. It’s hard to tell from that article how likely he felt that % might be. I think it’s important to be clear about that. Since Ioannidis always stresses one end of the spectrum of the probabilities (e.g., in April 2020 he said that we had probably peaked in COVID deaths in Europe and the US) the temptation is to peg him as not actually considering a range. One problem with Ioannidis, IMO, is that he treats the probabilistic aspects in such a fashion. He did that when he said that the IC modelers “predicted” 2 million deaths (and that such a prediction was “science fiction”). It bugs me when people like Ioannidis ignore the probabilistic aspect of the Imperial College projections.

          But in the climate change world, there’s a pattern where scientists make probabilistic statements about uncertainty, consumers of that information then ignore the probabilistic aspect, cherry-pick the low or high end of the range as suits their “motivations,” and then shout that the scientists were wrong, and complain that the scientists are over-certain and don’t properly treat uncertainties.

        • Yes, he did mention that the high end (which he didn’t believe) was something like 40 million deaths worldwide.

          The problem I have is that the 1% infection is incompatible with the low IFR assuming high rates of undetected infection, so multiplying those two doesn’t work.

          It would have made sense to give *separate* estimates for those *different* scenarios (say 1% infected at 1%-2% IFR; 20%-33% infected [comparable to flu pandemics, IIRC] at 0.3% IFR).

  4. To clarify the dissuasion with respect to COVID19 and consider the role of Ioannidis, I would distinguish between:
    1. The various stages of statistical intervention
    2. The methods and tools for statistical intervention
    3. The role of the statistician and biostatistician

    My take, in considering the role of Ioannidis in all this, is mixed. I cannot avoid the thought that attracting media attention seems to have played a major role in terms of motivation.

    Here is a brief of what I experienced in Israel, with some generalization.

    The Covid-19 pandemic is still considered as an emergency. However, one cannot really talk about an emergency, when a health situation lasts for more than one year and includes an evolution of policy strategies, population behavior and scientific knowledge. The role of a medical researcher, in general, has been evolving from handling a first panic phase (March 2020), when the search for diagnostic tools, medication approaches and hospitalization procedures was almost emotional and based on trial and error strategies, until today, when the focus is on understanding the vaccine effect and on planning the best vaccine strategies. In parallel, the role of statisticians and biostatisticians has been evolving to face different challenges and needs of biomedical research, in the hope of mitigating the dramatic impact of COVID-19 on society, the health and the economic systems.

    Phase I: Data collection and storage
    During the first phase (February 2020-June 2020) statisticians working in hospitals have been involved in designing and building a biobank for storing COVID 19 data. Initially, this was complicated by the lack of accuracy of diagnostic tools. No unique criteria were provided for defining variables (even the definition of “positive” was not unique and comparable). At that stage, statisticians were coordinating data collection and storage trying to ensure some quality control. In Israel, Prof Gamzu was appointed as COVID19 project coordinator in July 2020. One of his first steps was to reorganize the ministry of health data base. Because of politics, Gamzu was not able to implement differential restrictions (the red light system) and restrictions were applied universally, if needed, or not. Overcontrol was also prevalent due to lack of awareness and methods for process control methods such as change point detection.

    Phase II: Data analysis
    During a second phase (June 2020-November 2020), statisticians have worked on analysing clinical data and data from basic research. Basically, this was a very important phase where many statisticians tried to find a modelling approach, and extrapolate from information on epidemiology parameters such as mortality, fatality rate, incidence rate and predictors of disease evolutions. The search for appropriate statistical approaches, in dealing with highly correlated covariates and variables, has been a primary goal to help biomedical research identify COVID19 risks factor. In Israel, physicists and computer scientists lead modeling efforts. A call by the Israeli National Statistician, and general manager of the central bureau of statistics, to run a national serological survey to map infection rates was largely ignored. In fact, Seigal Sadeszki, who was the director of public health in the Health Ministry, stated on TV that testing is not useful as it does not cure people.

    Phase III: Design and surveillance
    In a third phase, statisticians have been involved in study design (planning new prospective studies) and surveillance studies. This includes i) handling issues on sampling (that is designing a representative statistical sample to follow longitudinally evolution of immunity related factors) ii) identification of individual predisposing factors to define not only surveillance procedures for frail groups of infected people but to develop vaccination strategies and iii) to identify priority groups for vaccination. In Israel, with a world record deployment of vaccinations (Pfizer), a major study by the Clalit healthcare system assessed vaccination efficiency.

    Again, considering such phases can help assess the writings of Ioannidis.

    This blog, although not specifically addressing epi and medical topics, seems to have done much more….

  5. I think the attacks on the grad student are related, given that the overall message of Gorski’s article is that Ioannidis seems to be suffering from a bad case hubris.

  6. Not commenting on Ioannidis but your comment in the paragraph starting with “Jensen’s inequality” lines up a lot with what Taleb has argued about the pandemic. In an exponential or multiplicative situation, even small errors in a parameter estimate can lead to large differences in projected outcomes.

    • Which may be true but ( like most points Taleb makes) so what? Is it actionable in any way?

      How do we use that fact to make better forecasts or even to remotely help our fight against covid in any way. All that says is forecasts will be difficult.

      To me this is a recurrent theme with Taleb: apparently profound but eminently useless.

      • It was very much actionable at that time! Ioannidis was suggesting that absent better quality data, the lockdown measures were an overreaction (we couldn’t be sure it was bad enough to warrant a lockdown given the trade-offs). Taleb agreed with the point that there was a lot of uncertainty, but argued that uncertain situations are precisely those in which it is crucial to be cautious. Caution consists in taking the action that minimizes the possibility of the worst outcome, ie lockdown.

        • But doesn’t it have another more significant meaning in the context of the time and for the future?

          If your data are highly uncertain your model doesn’t tell you anything. You can’t use it to justify any action. Its outcome is meaningless.

        • I don’t see why you can’t use uncertainty to justify action. If you are uncertain about the depth of a fast flowing river, don’t try to cross it. When there was massive uncertainty about the coronavirus, the prudent thing to do was take preventative action by locking down. From a decision theoretic point of view, the expected value is so massively dominated by worlds in which the worst things are true that one needs to act so as to avoid catastrophe in those worlds.

        • +1

          One thing this pandemic underscores is what a difficult time some people have evaluating “far tail” risks in the face of high uncertainty.

        • “what a difficult time some people have evaluating “far tail” risks”

          Yes, and some people have a difficult time making any useful analysis whatsoever because they’re so busy relying on cliches about analysis.

        • “if you are uncertain about the depth of a fast flowing river, don’t try to cross it. ”

          I made no such statement nor anything similar.

          Through this entire pandemic there is not one single data analysis that I ever saw that could reasonably contribute to decision making. The data analysis were entirely useless.

  7. >>> The problem is that Ioannidis didn’t seem to be able to let go of his initial downplaying of the risk.

    “For whoever exalts himself will be humbled, and whoever humbles himself will be exalted.” Matthew 23:12

  8. It is a little strange that Ioannidis is writing papers like this. After all, it isn’t neccessary to downplay the effects of the virus in order to oppose lockdowns.

    That said, I take issue with the “From my reading, it was a solid meta-analysis and review” of the Meyerowitz-Katz paper. I have a blog post from a few months ago that looked at this paper at https://categoricalobservations.com/2020/10/26/replication-when-it-matters/, and I don’t think the meta analysis is particularly good. Then again, back then I set out to do it better myself with Bayesian techniques and quickly gave up because I couldn’t find the relevant data.

    • > Gideon Meyerowitz-Katz co-authored a paper which came up with a mainstream estimate, as I understand it…

      There are others. The “mainstream estimate” I’d say comprises a number of meta-surveys in addition to thst one.

      In all fairness, Ioannidis’ meta-survey “global” estimate includes very large populations of much younger median age (like all of Africa and India) and as such, that estimate shouldn’t be considered applicable to the US.

      That said, he himself has been very inconsistent in how he has applied his IFR estimates – sometimes applying a single estimate to widely varying populations. Of note, he did that early on to promote an of IFR around 0.23% (as I recall) and imply during his TV publicity campaign that number applicable for the US generally, which seems clearly implausible for the whole population given it would mean that some 80% of Americans have been infected, and obviously more are going to die. And he did that back when, given the data he was using, the IFR should have been even higher than it would be now, since treatment has improved.

      • Joshua,

        Improved treatment led to fatality decrease among hospitalized patients, but that doesn’t mean IFR increased, decreased or stayed the same as we don’t know the ‘dark data’ at any given time (true number of infected people, which varies region to region).

  9. I don’t think the current kerfuffle has much, if anything, to do with Ioannidis’s original article which included 10,000 as a possible lower bounds on the deaths the US would experience.

    It centers around his continued publishing of papers which some find poorly done which purport to show that the IFR of covid-19 is, in actuality, no worse than a bad seasonal flu.

    The latest paper which includes the personal attacks mentioned above concludes that global IFR is 0.15%. This point estimate applied to the confirmed covid deaths in the US implies that virtually everyone in the US has had the disease at this point. It is nowhere near the mainstream estimates of about 0.5%-1%, a range that is consistent with mainstream estimates that 20%-30% of the US at this point has had either an asymptomatic or symptomatic case of covid-19.

    Gideon Meyerowitz-Katz co-authored a paper which came up with a mainstream estimate, as I understand it, and in the past has publicly criticized Ioanndis’s lowball IFR numbers on his popular twitter account.

    Anyway here’s Gideon Meyerowitz-Katz’s twitter thread on the subject.

    https://twitter.com/GidMK/status/1376304539897237508

    • > The latest paper which includes the personal attacks mentioned above concludes that global IFR is 0.15%. This point estimate applied to the confirmed covid deaths in the US implies that virtually everyone in the US has had the disease at this point.

      To be fair, that 0.15% is “global” and the US population is quite different from the global population (16% and 8% over 65 respectively).

        • “In all fairness, Ioannidis’ meta-survey “global” estimate includes very large populations of much younger median age (like all of Africa and India) and as such, that estimate shouldn’t be considered applicable to the US.”

          It is primarily US policy that he’s been focused on influencing.

        • dhogaza –

          > It is primarily US policy that he’s been focused on influencing.

          No doubt. But that doesn’t mean he’s arguing that the 0.15% IFR would apply for the US.

        • I’ve never seen him argue that the IFR in the US is in the range of 0.5-1% either …

          Didn’t the original draft of the Santa Clara study come up with a point estimate of something like 0.17% or the like?

        • Not in defense of Ioannidis’ low estimates for the US (eg Santa Clara): but the much lower per-capita fatality rate in the Old World tropics, vs Europe, US, or the New World tropics, *could* mean that the global IFR is indeed much lower than US/Europe IFR – as a large part of the world’s population is in South Asia, Southeast Asia, and Africa.

          (It could also mean that the infection rate there is surprisingly low.)

      • To be extra fair, how on earth is Ioannidis estimating the global IFR of Covid when we know that lots of places on earth don’t even have good data? Ioannidis is a contrarian who came up with a bad estimate early in the outbreak and has dug in based on even weaker arguments.

        • Steve,

          John Ioannidis has continued to highlight that the data is not reliable. I think that it is pointed out in that article as well. The quality of the data has been a problem all along. Why? The experts did not see the need to test asymptomatics, as John Ioannidis maintained from the start of the pandemic.

        • “John Ioannidis has continued to highlight that the data is not reliable.”

          But in ways that are favorable to his assertion that the IFR is close to that of the seasonal flu and favorable to his positions on policy.

  10. Andrew writes: “My point here is not that it was wrong for Ioannidis and others at the beginning of the pandemic to consider low-end possibilities. Rather, I just want to emphasize the inherent difficulties of handling this sort of variation over orders of magnitude, and the importance of going back and understanding where past guesses were wrong.”

    I think that you are too charitable to Ioannidis. He and others were clearly engaged in advocacy not armchair science. If a fire alarm goes off, and I tell everyone that I’ve been a fire warden and 90% of the time those alarms are false, just sit put, I’m sure it will shut down soon. If people die, I am morally and perhaps legally responsible. In the face of assymetric risks, it is immorral to advise people to take the least safe path. On the one side, the pandemic can explode (in January, you could already see in China’s data that deaths were doubling every 3 days). On the other, you have an economic slow down that can be immediately reversed if suddenly its discovered that Americans are magically immune to a virus that kills other humans. This was not a guy who said, “We have estimated the risk and it is much lower than other estimates, but our data is too uncertain to be used for policy decisions and the risks if we are wrong are exponential.” He was saying quite the opposite.

    • “If a fire alarm goes off, and I tell everyone that I’ve been a fire warden and 90% of the time those alarms are false…”

      Off topic, but oddly enough I had that experience on an upper floor of a business hotel in the DC area. Breakfast room full of Colonels and Lt. Colonels in uniform every morning, on their way to meet with Congressional staff etc.

      Anyway first morning I was awoken at sunrise by a fire alarm. Looked outside and could see fire trucks on the way, put a wet towel under the door jam, waited. They stood around, looked, did nothing else, alarm went off, they drove away.

      I was there for an entire business week and the same thing happened every morning …

  11. Last summer, when it was clear he was very wrong, I had an email exchange with Ioannidis about his estimates, and his whole “aggresively optimistic” view and approach (lots of interviews, op eds, etc…).

    In our personal exchange, he seemed very forthcoming, and openly admitted he got it way wrong.

    I guess it’s a lot easier to do that one-to-one than go on a big mea culpa tour on t.v.

    Of course, he was perfectly willing to go on a big t.v. tour when he was spouting media-friendly, attention-grabbing, contrarian opinions.

    Not sure what my point is here – just that humans are complex. He certainly seems like a very decent and smart dude from all of his work and my limited interactions. I don’t know how well I would react to a massive public humiliation like this either.

    • John has admitted he has made mistakes in several public interviews. I think that John’s being featured on Fox news raised angst But the authors of the Santa Clara Study did incorporate its critiques subsequently.

      • Sameera –

        They never walked back extrapolating an IFR from non-random convenience sampling and compounding that error by extrapolating from their methdologlically flawed estimate based on the Santa Clara data to a more broadly applied IFR without adjusting those data to make them representative.

        That’s the kind of science that led Ioannidis to compare COVID to the flu.

        Do you find that justifiable?

        Errors are one thing. Bad science is a other.

        • Timeline error – he compared covid-19 to a bad seasonal flu season first, Santa Clara followed and was meant to support the assertion.

        • Hi Joshua,

          As I have suggested, that comparison to the flue was context specific to the Diamond Princess Case, Each consecutive month since March, John Ioannidis has continued to say that COVID19 is more lethal especially as it affects vulnerable and elderly population. In short there is a steep age gradient.

          Moreover, John has continued to highlight that we are drawing hypotheses from unreliable and little data. I don’t think there is real disagreement on that point.

          Finally, even good science can be misapplied to a specific situation. It’s a complex epistemic environment.

          Addendum: I don’t agree with John Ioannidis always. I am much less of a proponent of Statistical Significance, as one example.

  12. just some wild thoughts. Is it possible that Dr. JI has been so carried away by his years of being different that his mindset became fixed to be different from others? True, his many articles criticized wrong practices in medicine and epidemiology, which I agreed with him most of them.

    I was very optimistic when I learned COVID in late Dec 2021(yes, the underground Chinese media were already circulating bad news before the official announcement). I didn’t believe there would be 100,000 deaths in the US, and I thought the Imperial College estimation is a bluff to paint the worst case scenario. I believed it would go away by May or June, just like SARS in 2003. If checking the history, virus jumps onto human beings all the times, and small outbreaks of new virus (or new variants) occur every few years. Most epidemic will die away without the public knowing it. The public health department will handle it quietly and the reports remain internal, and even if made online (or to the public), nobody cares.

    But reality kicked in and I was puzzled by the erratic behaviors and beliefs of the public, the government, and worst of all, the CDC and other acclaimed experts. Not wearing masks, deriding lockdown, and so on are unthinkable. These traditional NPIs worked since 1918, and nothing is new. However, watching the number of deaths and hospitalizations going up continuously is very saddening and painful. It is a sign of failure in epidemiology and public health system and also an indication of selfishness of many Americans.

    • Xyu:

      One thing that bugs me is that the anti-lockdown people aren’t more vocally recommending masks. I can see that anti-lockdown can be a reasonable position, but if you’re not going to have restrictions on how people interact face to face, then masks are even more important, right?

      • There seems to be something even sicker than that. E.g. a while ago Martin Kulldorff was arguing that herd immunity was the way to go. His argument was indifferent to human life but at the time not as obviously wrong technically as some of the other arguments. Now he’s urging everyone who is not at high personal risk not to get a vaccine. What happened to promoting herd immunity? It seems like the performative callousness is a more consistent thread than any particular technical claim.

        • The irony is that since March until December, some rate of herd immunity has been progress. Paul Offit, Michael Osterholm, and others have guessed that upwards of 100 million in US has had COVID; most perhaps asymptomatic. I think even CDC Director Walensky put out a figure of 20 % of population has had COVID19. There may be an undercount of COVID infected pop. Some may suggest that John Ioannidis was instrumental in forging COVID policy under Trump administration. But he did insist that asymptomatics get tested. That was not heeded. Why we have the case of the elusive COVID19 denominator.

          In regard to Who should or should not get vaccinated. There is intense disagreement. I do think that we could saved many more lives had we tested Americans back in April or May. Not with the PCR; with the Rapid Antigen tests for infectivity.

        • “In regard to Who should or should not get vaccinated. There is intense disagreement. I do think that we could saved many more lives had we tested Americans back in April or May.”

          For that to be effective you have to be willing to take action on the information that is made available. That wasn’t going to happen in the US at the federal level, certainly, and in many states.

        • Absolutely agreed re rapid antigen tests. It’s still not too late for them to be somewhat useful.

          Don’t just about everybody agree that ~100M have had Covid in the US?

          I found my old email from 3/17/20″ “a neat trick: get a low CFR by assuming many undiagnosed cases and a low infection plateau by assuming very few undiagnosed cases.”

          You’re bending over backward to give JI credit for saying in passing “we need more data”, as if that weren’t on the stationary letterhead for any scientist. His overwhelming message was: meanwhile, no biggy, don’t worry, do nothing.

        • Hi Michael,

          I have watched nearly all of the interviews with John Ioannidis and read a good number of his articles. I will add that John is most understandable when he is lecturing or being interviewed. The articles can get a tad too technical. I have to rely on commentaries to make sense of the technicalities.

          Yes, I do give John ioannidis credit for highlighting the complex epistemic environment environment in medicine. I have followed the evidence based medicine movement since the 90s. I’m disappointed to find that it has been co-opted in ways that translate into marketing products and treatments that of marginal value. It takes considerable courage to convey this to the public and to the medical establishment.

          BTW, here is the article that seemed to have caused such controversy. When John mentioned the 10,000 figure, in what context did it come up? I think that readers have taken the 10,000 out of context. My interpretation was that John was suggesting that even the Diamond Princess projections were unreliable due to scant data. I took the trouble to ask some of my friends to read the article. Even they did not think that John Ioannidis was making a categorical claim that the IFR was as low as the flu. He couldn’t have makde that claim given the totality of the argument he made.

          https://www.statnews.com/2020/03/17/a-fiasco-in-the-making-as-the-coronavirus-pandemic-takes-hold-we-are-making-decisions-without-reliable-data/

        • Hi Sameera- I read that article at the time, with great alarm. At every stage where there was any uncertainty he always chose the lowball number, even when that required inconsistent assumptions. He never gave a hint that for exponentials early action is cheap and effective while late action is expensive and ineffective. It was a complete hack job, decorated with a few scientific bromides, like how more data would be nice.

          This is not Monday morning quaterbacking. I wrote people on 3/17/20.

          One important thing to do under uncertainty is to ask whether the second derivative of the utility function is positive or negative wrt estimation error. JI implicitly got the sign wrong.

        • Sameera –

          > Even they did not think that John Ioannidis was making a categorical claim that the IFR was as low as the flu. He couldn’t have makde that claim given the totality of the argument he made

          2:36 in this video… “the same ballpark as the seasonal influenza.”

          https://youtu.be/jGUgrEfSgaU

        • And anyone can be wrong.

          The problem is the methodology behind why he was so wrong. He got it so wrong because he extrapolated from unrepresentative and non-random and ill-suited and insufficiently comprehensive data.

          That isn’t just missing on pegging the uncertainties. That’s bad science. What’s odd is that (1) he warned about that kind of bad science vis a vis COVID and, (2) he made his stellar reputation by criticizing bad science, exactly like the bar scoence he’s done with COVID.

        • Good Morning Joshua,

          The video that you posted doesn’t capture the progression of John Ioannidis’ hypotheses. The video below is clearer, I think.

          https://www.youtube.com/watch?v=QUvWaxuurzQ&t=1s

          What is the thesis of the original article which has resulted in such expert angst? It’s in answering that question that lends itself to appropriately contextualizing the 10,000 estimate associated with the Diamond Princess Cruise situation.

        • How is the 10,000 estimate associated with the Diamond Princess Cruise situation?

          The main problem with the 10,000 estimate is the assumption that 1% of the U.S. population gets infected which is introduced without any justification.

        • as of beginning of March, my own data on seroprevalence of SARS-CoV-2 infection is 20-27% in the community, depending on the samples. My estimation of overall infection in the US as of March, 2021 (when US reached 500,000 deaths mark) is 23%.

        • Sameera:

          What do you mean by some rate of herd immunity? We definately don’t seem to be at it? If you mean we are 30% there, sure.

          Second, 100 million versus 20% of US population is a sizable difference. In any case neither of those numbers seem anywhere close to herd immunity.

          So basically I struggle to understand your claim.

        • Hello Rahul,

          To clarify, earlier I meant to qualify the characterization ‘herd immunity’ to natural infection. By that I mean that herd immunity can also be reached when a sufficient number of people in the population have recovered from a disease and developed antibodies against future infection.

          Note what I posted earlier:
          “he irony is that since March until December, some rate of herd immunity has BEEN IN PROGRESS. Paul Offit, Michael Osterholm, and others have guessed that upwards of 100 million in US has had COVID; most perhaps asymptomatic.”

          So I was inferring from the opinions based on those COVID experts that roughly a 1/3 of US population had been infected. Rate is a ratio.

          In this case calculating an accurate denominator has been a challenge. Now CDC director Walensky estimates 20 %. Back in the fall, The previous CDC director estimated that there were between 90 to 105 million.

        • Sameera,

          There is a 100-fold variability in antibodies among those previously exposed to covid. True, for some people on the higher end of the spectrum, the vaccine won’t add much, but without complicated and expensive tests we can’t know where any of us stand.

          The chance for a second infection and long covid in those previously infected is a real danger. There were cases of second infection fatalities in Italy.

          Whether vaccinate or not, 1918 flu is a pretty good indicator. I believe you mentioned your reaction to vaccine some time ago, and in those cases probably best to wait for a milder one, which may come soon, but don’t let your n=1 sway your opinion in general.

          AFAIK, there are no preservatives, alluminum, eggs, mercury and other stuff in these new ones, so you may as well be ok with it (depending on your age and if you need it in the first place, of course).

        • AFAIK, there are no preservatives, alluminum, eggs, mercury and other stuff in these new ones, so you may as well be ok with it (depending on your age and if you need it in the first place, of course).

          The main risk along those lines is anti-PEG antibodies:

          Despite this success, an emerging body of literature highlights the presence of antibodies produced by the immune system that specifically recognize and bind to PEG (anti-PEG Abs), including both pre-existing and treatment-induced Abs. More importantly, the existence of anti-PEG Abs has been correlated with loss of therapeutic efficacy and increase in adverse effects in several clinical reports examining different PEGylated therapeutics.

          https://pubmed.ncbi.nlm.nih.gov/27369864/

        • Sameera:

          So I hear that fact about herd immunity often but don’t understand why that matters.

          Sure, you can reach herd immunity by natural infection. If you wanted to go that route we could all shed masks, stop social distancing and go about life as usual. Maybe even congregate around super spreaders and our herd would reach immunity faster!

          Problem is, natural infection carries with it a mortality / morbidity risk which is way higher than building up your immunity by using a vaccine.

          That’s where vaccines have a huge advantage over the natural infection route. A secondary point is that the antibody titres post vaccination seem higher and sustained than post natural infection ( on average).

          Finally, even taking your estimate that 30% of the US population has been infected, it’s still a far cry from herd immunity and vaccines are the only credible way to get there, IMO.

        • A secondary point is that the antibody titres post vaccination seem higher and sustained than post natural infection ( on average).

          Most studies claiming this compare antibodies at 1-2 weeks post second vaccination in healthy people to convalescent antibodies at some random timepoint after infection in whatever blood samples are sitting around.

          Can you share one that does not suffer from this bias?

          It is also not at all clear that high levels of monoculture antibodies towards one specific spike protein sequence (that does not even exist anymore: https://nextstrain.org/ncov/global) are preferable to lower levels of more diverse antibodies.

          Why do you think you need a booster dose? It is because your immune system knows to wait for multiple exposures before dedicating itself to a full response. That way it gets an idea of the spectrum of mutations that may show up, making OAS less likely:

          https://en.wikipedia.org/wiki/Original_antigenic_sin

        • +1 I hate to play armchair psychologist, but on this blog Andrew often talks about a repeating narrative where ordinary people are dumb and irrational. There is a ton of these sort of narratives throughout academia, economics etc. I think people have internalized these “people are irrational, but not me” stories. It is such a part of their identity that they felt compelled to point out people were freaking out when toilet paper was flying off the self as if they had a thorough knowledge of the TP supply chain. They want to be intellectually superior, and you just can’t do that if the correct answer is “I don’t know, it could be really bad. Maybe your natural instincts that resulted from millions of years of evolution are more useful than my degrees.”

      • I agree with you that many loud people showed illogical think during the pandemic. When assessing various NPIs, I recalled one comment from the Atlantic magazine. People are thinking in a “binary” way (either this or that), and also I think “reductionism” is not appropriate here.

        Long term lockdown definitely hurts economy and society, especially those poor people. But anti-mask and now anti-vaccine are unthinkable during the pandemic.

        One principle of public health intervention during the emergency is we don’t wait for evidence, we act first and modify along the way. This is written in the teaching materials by CDC. This is the same as in the ICU room. When a patient has fever, physicians will prescribe antibiotic treatment immediately before the lab identifies the type of bacteria. Most likely, the tentative antibiotics are correct, that is what the years of clinical training are made for.

        • If you recall the message from the establishment was:

          a) surgical and other low-quality masks are ineffective and shouldn’t be used;
          b) N95 and high quality masks should be the exclusive right of medical professionals.

          This position wasn’t unique to the CDC nor did it originate there. It was widespread among epidemiologists, many who otherwise provided excellent guidance on the likely outcome of the pandemic (e.g., Michael Osterholm).

        • Jim:

          Again, I recommend the above-linked New Yorker article by Lawrence Wright. It’s horrifying to read the story, kind of like watching a vase fall off the table onto the floor and not being able to stop it. The mix of ignorance and politics . . . ugh. And lots of other countries did as bad or worse.

          I remember at the time feeling so ignorant. Back when there was all this emphasis on handwashing, I had no idea that masks were where it’s at.

        • I can’t bear to read the whole thing. I’ve been through this stuff so many times.

          I don’t feel like I was ever ignorant. In Osterholm’s interview with Joe Rogan last march, he said “in some cases you can get it from just talking to people” – i.e., airborne transmission. When the “Choir Case” from Mount Vernon, WA, surfaced it was clear that the primary mode of transmission is airborne – in a kind of invisible fog. Not droplets, not surfaces. Shortly after watching Osterholm’s interview with Rogan in late march last year, I bought a pack of KN95 masks off eBay and have been using them ever since. They wash well.

          The unwillingness of the establishment to recognize the obvious is disturbing, but not surprising. American students are trained to give the right answer, not to solve problems. They can hardly reason at all.

          Once during my undergrad I was studying with a group of students when a controversy arose over some study question. I explained the answer, but the others insisted my answer was incorrect “because Dr. Johnson said….”. I pointed out that what they claimed he said didn’t make any sense, and they probably misunderstood him or he misspoke. But they weren’t looking for sense, just the answer, and what Dr. Johnson said must be the answer. Right? :)

          That’s exactly what happened in this case. No one was solving a problem. Everyone was waiting for the “right” answer, waiting for the square peg to put in the square hole. That’s all they know how to do. This happening in the senior levels of American science. The rot is old and deep.

        • Jim:

          I believe that it was clear to you all the time; it just wasn’t clear to me. And it wasn’t because I was unwilling to recognize the obvious; I was just confused by a multitude of stories floating around.

          Just by analogy, when I saw that silly claim that losing an election for governor cost candidates 5-10 years of life, I could see right away how wrong it was, and it wasn’t hard to take it apart and see what was going wrong. But I think lots of people with statistical training would not have been able to see this (even though they’d ultimately be able to follow my explanation). They lost the forest for the trees. That’s how I’ve been with the pandemic. Confused at each step. Indeed, the contributions I’ve made to the conversation on the topic have largely come from my willingness to accept uncertainty. Which is fine, but willingness to accept uncertainty will only take us so far . . .

        • “it just wasn’t clear to me. And it wasn’t because I was unwilling to recognize the obvious; I was just confused by a multitude of stories floating around.”

          That’s understandable. I wasn’t referring to you as “the establishment” – by that I meant top health officials, both in the US and abroad. What does amaze me is that the press keeps blabbering every word that comes out of WHO like it’s the word of God. They were badly wrong about this and several previous issues. They seem to have an institutional problem.

          My knowledge of 1918 pandemic was probably essential to understanding what was happening. But although there are different areas of expertise in the discipline, many people in epidemiology have that knowledge as well.

        • The idea that two masks between any person’s respiratory system and any other person’s would not hinder transmission of a respiratory disease *at all* never made sense, ignored Asian practices,and naturally quickly proved false. The position was not that masks were ineffective,it was that they were not proven effective. Dichotomous.

        • I watched an interview with Michael Osterholm in March 2020 in which he flatly stated that low-grade masks were ineffective. He and probably others in the epidemiological community seem to have drawn this inference from the ineffectiveness of low-grade masks when used by health care workers attending sick people, and as we know many health care workers in China became ill and died, so their concerns were reasonable initially.

          What they apparently overlooked is that health care workers can be immersed in a super-virus-rich environment for long periods, while people in daily life usually don’t experience that kind of immersion and thus low grade masks are effective. I think initially it was a reasonable inference to draw, but somehow people got dug in to that position and rather than admit that any mask would have at least a marginal benefit, then they started seeking more justifications for rejecting mask use.

        • Indeed.

          In England face masks and coverings for the general public in public places have not been mandated beyond public transport and hospitals. Wearing a face mask or covering in the UK has had very low uptake (~25%, late April 2020). The lack of clear recommendations for the general public and low uptake of wearing face masks and coverings may be attributed to: (i) over-reliance on an evidence-based medicine approach and assertion that evidence was weak due to few conclusive RCT (randomised controlled trial) results in community settings, discounting high quality non-RCT evidence. There have been no clinical trials of coughing into your elbow, social distancing and quarantine, yet these measures are seen as effective and have been widely adopted; (ii) inconsistent and changing advice from supranational organisations (WHO, ECDC) and other nations with variation in policy even within the UK; (iii) concern over the applicability of findings across multiple settings (health care versus general public, other pandemics and countries), yet many ‘lessons learned’ from previous pandemics, including public wearing of face masks and coverings, repeat themselves during COVID-19; and, (iv) mix of supply concerns of PPE shortages of surgical face masks with recommendations for face mask and covering wearing for general public.

          There was fighting between EBMers and SBMers. Despite the efforts David Gorski and other SBM advocates have put in over the years it seems this kind of stupidity is still a problem in medicine.

        • xyu said,
          “One principle of public health intervention during the emergency is we don’t wait for evidence, we act first and modify along the way. ”

          +1

        • Certainly the operative principle for anyone who has either the knack or has otherwise learned how to lead in a crisis. Not a “paper crisis” (my god! are we going to have retract that claim because figure xx and table yy are in disagreement?). Think of the evacuation of Dunkirk. How did they assemble all those miscellaneous seagoing vessels in such a hurry, without having first an expert consensus, quantifying — among other things — the risk of depressing the Kentish fishing economy?

  13. RE: There’s also some discussion about Ioannidis criticizing Meyerowitz-Katz for wearing big glasses and having a cat. That wasn’t cool for Ioannidis.

    Ya never know what someone is saying behind your back. lol It’s like high-school in some quarters.

  14. Andrew says “so the loss of qualys is much less than from a flu that kills the same number of younger people”

    Just curious, do we know this for certain now:

    Covid kills older and flu kills younger?

    • “Covid kills older and flu kills younger?”

      Seasonal flu primarily kills older people.

      But the Spanish Flu pandemic killed a great many younger healthy people 20-40 as well as 65+ people. This is one of the things that makes it so interesting.

    • I may be wrong but as I recall, the age stratification of the fatality isn’t that different until you get to quite young ages.

      But fatality isn’t the only relevant measure. I believe that COVID is much more likely to hospitalize younger people or cause other important problems, healthcare and otherwise. When you’re considering the impact of the flu and COVID respecdtively, focusing only on deaths would seem sub-optimal, IMO.

      • Normal flu, yes, flu pandemics I think the mortality curve is vastly more different. 1918 had a very weird 3-peak mortality curve (youngest, ~30, oldest) and I think 2009 had a relatively flat curve (though deaths were low at all ages), though I don’t have/remember a source for the latter. Not sure about 1957/68 – and 1918/2009 were both H1N1 so might have been different.

    • Seasonal flu seems to kill slightly younger than covid. At least if we look at absolute numbers, in the last decade 6.6% of the estimated deaths in the US were 18-49 (and 0.6% younger) while for covid it’s 4.3% (and 0.0%). The number of 0-17 deaths by covid is comparable to the average year for flu, in the 18-49 it’s 10 times as much and 15 times as much for the 50+ (50-64: x16.5, 65+: x15). The 65+ represents the large majority of deaths in any case: around 80%.

      Maybe that “much less” is about “a [non-seasonal, pandemic] flu that kills the same number of younger people”.

  15. I’ll see if it will post again.

    Carlos, I may not full understand your question. So apologies if so.

    RE: How is the 10,000 estimate associated with the Diamond Princess Cruise situation?

    The main problem with the 10,000 estimate is the assumption that 1% of the U.S. population gets infected which is introduced without any justification.
    ——-
    The association is implicit b/c people inferred that John Ioannidis used the Diamond Princess as basis for his estimate. But I contend that there is little in that controversial article to suggest that he was making a categorical estimates based on the Diamond Princess as he emphasizes its thin dataness and

    Friends [non-experts] [who have no direct stake in the debates] but who have mathematical or engineering backgrounds saw it similarly b/c John continually noted that data was unreliable.

    Now as to where he got the !% I do not know. But I did read this article previously. John is mentioned in it. He might have gotten it from one of the other researchers of the Diamond Princess case.

    https://www.nature.com/articles/d41586-020-00885-w

    I have also read Hilda Bastian’ rebuttal.

    https://hildabastian.net/index.php/covid-19/88-addendum-to-fiasco-rebuttal-what-about-that-0-05-0-1-case-fatality-estimate

    My view is that much of the discussion on Twitter, at least, is that people cherry pick and take out of context statements. So it’s refreshing to have a forum where we can all be accountable for our mistakes.

    • Sameera –

      It all fits together as a package. Santa Clara, the Diamond Princess, 10,000 deaths, 40,000 deaths, 1% infected, the peak probably reached in April 2020, the low IFR, the extraordinarily high prevalence estimates, the extrapolating from non-random and non-representative sampling, the publicity campaign on rightwing TV, the selective treatment innrhebincdrosntirs about attribution of deaths to covid, the weird discussion of the Danish mask study, likening COVID to the flu, the bizarre personal attack on the appendix of a published paper, and most importantly imo, comparing across contexts without good control for confounding variables and talking about deaths due to “lockdowns” without being explicit about definition of terminology or explicating assumptions about counterfactuals.

      • Re counterfactuals: there’s something I’ve been wondering about for a while here.

        “In theory” it’s clearly possible for life to “go on as normal” with no major disruptions while a pandemic is happening, as that’s what happened in 1957 – COVID is worse than that (at least in the US: world deaths per capita may very well be basically the same) but not totally out of scope.

        But I don’t think just different government policies could have gotten to that outcome, given the very different nature of media/communication now, and (IMO) society being far more risk-averse in general.

        (I am not saying that that would have been the right thing to do – just talking about the range of possibilities.)

        • I don’t know! That’s why I qualified it with “IMO” :)

          But I was thinking of things like the lack of environmental and worker-safety regulations, much less safety rules in general (e.g. cars were a lot less safe, I believe), and so on.

          In the 50s restaurants in Las Vegas had balconies where people could eat while watching the mushroom clouds of above-ground nuclear tests go up!

          I was reading a book about the X-15 program titled “At the Edge of Space”. It said that some of the X-15 pilots had previously been involved in a program spraying chemicals on forests in Oregon — and (IIRC) about a third of the pilots in the program died in crashes in, I believe, one year (8 out of 23 or something like that). And it was just mentioned as “by the way, this happened, flying is dangerous” not as something that was especially shocking.

        • Recognizing different risks doesn’t mean less risk-averse. I wouldn’t be surprised if they were more concerned about the risk of walking under ladders or putting a hat on the bed than we are.

        • Maybe?

          But I don’t think they didn’t *recognize* workplace fatalities or car crash deaths or airplane crash deaths (environmental risks, yeah, I think were genuinely not understood).

          I actually think a lower risk-tolerance today would be almost unavoidable, given how much safer life-in-general is.

          Adults in the 1950s had grown up before antibiotics and before most vaccines. I don’t think they didn’t *recognize* the 1957 pandemic; it just didn’t seem particularly frightening compared to familiar disease risks. Whereas, with the exception of HIV/AIDS, most modern American adults under 80 or so have never had to worry about infectious disease at all.

        • “A very impressive peer reviewed publication history.”

          We all know this. It doesn’t exonerate his track record on covid (which continues down contrarian lane, apparently driven by political ideology). It makes it worse, actually.

        • It’s sort of embarrassing the lengths John Ionnidis is going to in order to defend the cruddy 0.15% IFR. He’s pushed through this review of 6 systemic evaluations (one of them was his) through peer-review that was essentially a hit-piece meant to directly attack (and punch down) at specific authors of the other analyses (worth mentioning JI was previously editor-in-chief for the journal publishing the article-in-question). Some quotes (real nice guy):

          “In multiple main media interviews and quotes Meyerowitz-Katz is presented professionally as an “epidemiologist”, but apparently he has not received yet a PhD degree as of this writing and he is still a student at the University of Woolongong in Australia. Neither he nor his co-author of the evaluation (apparently another PhD student) had published any peer-reviewed systematic review or meta-analysis on any topic prior to the pandemic”

          And:

          “For students performing their first evidence synthesis ever, choosing a topic that requires advanced expertise due to unusual cross-design features, difficult methodological challenges and convoluted and often erratic data, a highly-flawed final product should not be surprising.”

          Geez, gatekeep much?

          It’s astounding to me this type of commentary (in the Appendix) was accepted by the editor and peer reviewers.

          Of course, he used the article to double (tripled? quadruple?) down on the bogus 0.15% IFR claim that’s remain unaltered for the past year, despite contrary evidence.

          Conclusions: “Acknowledging residual uncertainties, the available evidence suggests average global IFR of ~0.15%”

          Highlights: “Global infection fatality rate is approximately 0.15% with 1.5-2.0 billion infections as of
          February 2021.”

          Just awful stuff, coming from the guy who helped write the PRISMA guidelines.

        • Leaving aside the comments in the Appendix that have caused such angst, I am in the process of reading as many other authors of IFR estimates. It is an extremely challenging project given that there was limited testing last spring and summer. I thought that this NYT article was quite excellent. It notes the studies that have attempted to calculate herd immunity. The range of estimates is wide, to say the least.

          https://www.nytimes.com/2020/04/13/opinion/coronavirus-immunity.html

          Specifically it echoes the view that I have expressed as to the need to capture the elusive denominator. Mark Lipsitch conveys:

          “It is possible that many more cases of Covid-19 have occurred than have been reported, even after accounting for limited testing. One recent study (not yet peer-reviewed) suggests that rather than, say, 10 times the number of detected cases, the United States may really have more like 100, or even 1,000, times the official number. This estimate is an indirect inference from statistical correlations. In emergencies, such indirect assessments can be early evidence of an important finding — or statistical flukes. But if this one is correct, then herd immunity to SARS-CoV-2 could be building faster than the commonly reported figures suggest.”

          To return to John’s estimate, I came across review by John Ioannidis from October 2020 that goes into even more detail than the article published in March 2020.

          Global perspective of COVID‐19 epidemiology for a full‐cycle pandemic

          https://onlinelibrary.wiley.com/doi/full/10.1111/eci.13423

          I’m emphasizing that it is important to be more mindful of the contexts in which statements were made.

        • Sameera –

          This is an example of what I find in Ioannidis’ work on COVID –

          > Limited testing still leaves some COVID‐19 deaths undocumented. Conversely, many countries may count some spurious COVID‐19 deaths. Death certificates are notoriously error‐prone in general1 and may be even more error‐prone with COVID‐19. Adherence to stringent clinical case definitions plus imaging/pathology documentation for SARS‐CoV‐2 causal impact is often lacking.2 In high‐income countries, almost all the deceased have known comorbidities, raising causality debates on whether some deaths are with rather than by COVID‐19.3 Deaths in people without documented comorbidities are more frequent in low‐ and mid‐income countries,4 but perhaps comorbidities remained undetected in resource‐poor settings.

          One sentence about possible under-counting and a much more detailed description of possible over-counting. That seems odd to me given that from what I’ve seen, most of the examination of the accuracy of death counts (many comparing COVID numbers to excess mortality, P-scores and Z-scores the like) suggests that undercounting is more likely than over-counting, in general.

          But maybe John is less biased and/or less wrong than most others looking at the issue. Maybe I’m nit-picking. Maybe there’s no particular reason he should have a similar amount of discussion of under-counting as over-counting. But the problem there is that I’ve seen him focus on the problems with over-counting many times, and this pattern repeats where when he discusses the uncertainty he certainly seems to lean in much heavier on the uncertainty in the one direction rather than the other – with no full explanation for why he’d do that.

          And then there’s this:

          > More importantly, differentiating between COVID‐19 deaths and those due to harmful response measures is challenging.5, 6 Some of the deaths due to the measures taken happen acutely (eg, due to people with acute myocardial infarction not coming to the hospital for care),7, 8 but the majority may accrue over longer periods of time (Table 1). There is strong evidence on the adverse effects of unemployment, financial crises, depression, and social isolation on long‐term morbidity and mortality,9-14 but caution is needed to extrapolate this evidence to the current situation which is unprecedented in terms of the acuteness and massive impact of the measures taken. Some projections have been made for these excess deaths, and evidence is already accumulating for some of these excess death causes.7, 25 Putting projections together, the excess deaths from the measures taken is likely to be much larger than the COVID‐19 deaths, for example, disruption of tuberculosis programmes alone is expected to cause 1.4 million extra deaths over the next 5 years and the death toll from famine can be even more staggering. However, the exact impact of these major problems has very large uncertainty, and some projections may be exaggerated (as was the cause also for COVID‐19 projections).26 Their excess death toll will likely depend on our ability to address these problems early on and to avoid recurrent lockdowns and other draconian measures.

          Here there is zero discussion of how he determines why to attribute indirect deaths and other harms, like unemployment or famine, to interventions as opposed to the pandemic itself. No mention of how and why he rests his advocacy on an unproven counterfactual assumption that all of those negative outcomes wouldn’t have been the same or perhaps even worse absent the interventions. I’ve never seen him once address that issue comprehensively. Even to the point of addressing why those negative outcomes don’t seem to be more prevalent in a place like Sweden than in a place like India or China which had much more stringent interventions. Or even in the other Nordic countries, which had at least somewhat more stringent interventions. Perhaps you have?

          But IMO, he should do it each and every time he addresses that issue. Of course, comparing across countries is ripe with the potential for uncontrolled confounding variables, but he does that a lot anyway, and I think it’s extremely important that he should address his counterfactual assumptions, and I don’t understand why such an important scientist with such a heavyweight status treats this issue as he does.

          And as I’ve mentioned to you many times, as I see it, his one-sided treatment of the uncertainties has been systematic and consistent (in one direction only) throughout his work on COVID. Now maybe I’m being unfair and it is my own biases that explain my confusion. Perhaps his treatment of the uncertainties is exactly what we should expect of a scientist of his stature and there’s a scientific explanation for the imbalance in his approach. Or maybe any imbalance is my own invention or an artifact of my own subjective perspective. I can’t rule any of that out – but it sure would help if John at made a more explicit attempt to engage with, what seem to me to be, obvious counterarguments to the arguments he’s making.

        • “I don’t understand why such an important scientist with such a heavyweight status treats this issue as he does.”

          Because the axe he is compelled to grind has supervened all other considerations; and because he has transformed into a caricature of himself.

        • From October 2020…

          > Median IFR across 51 locations is 0.23% for the overall population and 0.05% for people Moreover, multiple studies have identified pre‐existing cellular immunity that may be effective against SARS‐CoV‐2 in 20%‐50% of participant samples.55-57 If so, the proportion of people who need to be infected to reach herd immunity may be much lower than originally estimated.

          Here he certainly seems to be talking about T-cell immunity, and to be suggesting that it provides immunity from infection. But from what I’ve seen, the majority of scientists who are examining T-cell immunity with respect to COVID say that it likely *doesn’t* provide immunity from infection, although perhaps it may help to provide an immunity that would reduce the severity of infection. Thus, there is much question as to whether it will reduce the % of infected need to reach a herd immunity threshold (although reducing severity of infection might reduce the probability of someone being infectious). But once again, his discussion is one-sided. He discusses the possibility that T-cell “immunity” might reduct the HIT, but doesn’t discuss the related uncertainties and caveats need to examine that question more comprehensively.

          It’s like this with each paragraph in John’s article, and indeed in each paragraphs in all the articles I’ve seen him write on COVID.

        • oops. Messed that up.

          This was the quote about T-cell immunity:

          That first sentence before the “Moreover” belonged with another quote

          The quote I wanted to respond to was…

          > Median IFR across 51 locations is 0.23% for the overall population and 0.05% for people <70 years old. IFR is larger in locations with higher overall fatalities. Given that these 82 studies are predominantly from hard‐hit epicentres, IFR on a global level may be modestly lower.

          His numbers here suggest that at least 80% of the US population has been infected, because in having the 14th highest per capita death rate, I'd say that the US would qualify as a hard-hit epicenter. I dunno, 80%+ of the population having been infected seems pretty implausible to me (and pretty far outside the range of what the CDC estimates). But even if it weren't, elsewhere in the paper he talks about reaching a HIT at maybe 60% population infection (the highest number he references) or even as low as 10%-20%. His selective treatment of the uncertainties leads to that kind of incoherent argument and internal contradictions.

        • Here, Scott Atlas promotes misinformation (stated as fact that T-cells provide immunity to infection from COVID, actually he said, a much higher % of immunity than antibodies have provided) at a presser, where Atlas assured us its the government’s best science by name-checking John Ioannidis.

          https://twitter.com/justin_hart/status/1308904131130781697?s=20

          This is direct evidence of the significant impact of John’s unfortunate, selective attitude towards the uncertainties related to the pandemic.

          It’s quite possible that if John had talked to Atlas about the full range of what was known about T-cells, Atlas wouldn’t have given misinformation to the public in his role as a spokesman for the coronavirus task force. It seems that instead, John gave Scott the same shallow sort of treatment of the uncertainties he included in that paper I excerpted.

        • Not at all in defense of Ioannidis’ approach in general – but those quotes do say *global* IFR. And I don’t think 0.15% globally is ridiculous.

          *Likely* too low, but not *crazy* as it would be for the US or Europe. Tropical Asia and tropical Africa represent much more of the world population, and these areas have relatively low per-capita death rates.

          I am not sure a weirdly low IFR is any less plausible than any other explanation for this. Age distribution, climate, and underreporting don’t seem to explain why the numbers should be so radically different from tropical America, and the huge diversity of government types and available resources IMO means that it’s not very plausible that they would *all universally* be way better at measures.

        • Now that I’m reading a wider range of authors, the challenge has been calculating a more accurate denominator. I see very little disagreement on this point, with very prominent experts echoing the need to gather reliable data. I posted a couple of articles that I think go deeper into assumptions made.

        • Yeah. I just think the US-specific statements by Ioannidis are much better to criticize, since we have much better data showing the US IFR isn’t anything like 0.2% or whatever the Santa Clara study had.

          I’m not sure 1.5 billion infections worldwide is really any harder to believe than that India’s only had say 30 million infections (about what you’d get extrapolating from 163K deaths and an IFR of 0.5%-0.6%) out of a population of 1.3-1.4 billion.

        • Confused

          RE: Yeah. I just think the US-specific statements by Ioannidis are much better to criticize, since we have much better data showing the US IFR isn’t anything like 0.2% or whatever the Santa Clara study had.
          ——
          I gather that the authors of the Santa Clara Study did incorporate the critiques of the study and then was published.

          RE: I’m not sure 1.5 billion infections worldwide is really any harder to believe than that India’s only had say 30 million infections (about what you’d get extrapolating from 163K deaths and an IFR of 0.5%-0.6%) out of a population of 1.3-1.4 billion.
          ——-

          My impression is that the quality of data has been an issue. So projections will be a work in progress and determined by the extent to which methods can be gleaned transparently.

        • It’s not the number. There’s a lot of uncertainty.

          It’s the science, and the rhetoric and the level of engagement in the public discussion.

        • Several of my friends and acquaintance have followed the Twitter debates even before COVID19. We all seem to select phrases and sentences out of context sometimes. This is what puzzles me, at least, given the high level of education among experts. Given that there seem to be allegiance bias that is not even subterranean, the tendency to select specific sentences out of context is also increased.

          It takes a high level of self awareness to see more dimensions of different biases.

        • Sameera –

          I agree. It takes diligence and a commitment. And even then you won’t get it much or perhaps most of the time. Maybe there’s hope not to get it wrong all of the time.

          I don’t know if you were following the whole Slate Star Codex brouhaha, but there’s an intersting and related discussion here about a “rationalist” approach (and a view of Bayesianism” and being “less wrong”) that is related and which you might find interesting (I listen as a podcast rather than watching the video) :

          https://bloggingheads.tv/videos/61230

        • BTW, Joshua,
          I look forward to reading those references. Thank you.

          I’ve long admired the views of Stuart Firestein, who has made the scientific process accessible to wider public. His book Ignorance is geared to it.

        • Hi Unanon,

          RE: Of course, he used the article to double (tripled? quadruple?) down on the bogus 0.15% IFR claim that’s remain unaltered for the past year, despite contrary evidence.

          —-
          Just curious to whom or which study do you refer as providing contrary evidence?

        • Sameera,

          Well, gosh, O’Driscoll, Brazeau, Basu, Levin, quite a few others!

          I’m curious if you have anything to say about John Ioannidis’s lack of professionalism in his latest piece? I’m also wondering if you’ve given thought to Joshua’s thoughtful comments from March 30th toward the top of the thread.

        • Hello Unanon,

          Thanks for those references. As I wrote earlier, I am in the process of reading the studies by other experts. Some though are behind a paywall. I like to read each study several times as it is so easy to misinterpret or misread out of context. So I will have to request the studies from the authors directly. And will read them and answer your question more fully.

          In reviewing one of O’ Driscoll’s and one of Nicholas Brazeua’s commentaries briefly I don’t see any really major deviations from John Ioannidis’ assumptions. All three emphasize the steep age gradient, estimate roughly the same range of IFR estimates, and point to the difficulty in making comparisons across and within countries.

          John Ioannidis has been advising WHO and I see that Nicholas Brazeau work is in collaboration with WHO.

          Specifically which potentially deviance from John by among these authors that you list stands out?

          As for the appendix comments, well they are most unfortunate. But to be honest, these types of comments, in my experience, stnndard fare in expert circles. The sociology of expertise is a fascinating subject. It was Kingman Brewster once echoed that politics are rife in universities. I confided in him way back in the seventies over my father’s career obstacles. There was perhaps more professionalism in a superficial way. Now social media has expanded the sociological boundaries.

          Frankly I have known perhaps 15 academics who have been exceptionally above the fray. Nearly every expert seems to have allegiance which actually constrain them from stating what they truly believe. Their institutional affiliations demand it. I think Deborah Rhone had made that point in her book about academia.

          As for Joshua’s comments. I do plan to respond when I have more time. I will say that it’s hard not to conclude that Joshua has a consistently critical view of John. My position is that one can be focused on perceived flaws of any work.

          I have more of Stuart Firestein’s perspective about scientific enterprise. It is misperceived by the public which is why it is important to revisit our education. And looks as if it is also misperceived by experts too. It’s not a linear or systematic endeavor most of the time.

        • Sameera –

          > I will say that it’s hard not to conclude that Joshua has a consistently critical view of John. My position is that one can be focused on perceived flaws of any work.

          I certainly have a consistently critical view of his COVID-related science, because as near as I can tell it’s been pretty poor. And I have a consistently critical view of the rhetorical approach he has used in his very public policy activism. Am I focused on the flaws I perceive in his work? Absolutely. Is there a reason I shouldn’t be?

          Sure, there are flaws in everyone’s work. If I give greater priority to the flaws in his work than I do to the strengths in his work, or even if there’s some sort of objectively defined imbalance or disproportionate distribution in my focus in that regard relative to a comparative measure of the flaws and strengths of his work, it doesn’t mean per se that my criticisms are invalid or unimportant.

          I guess you could consider as you say that I have a consistently critical view of him, but I try to avoid going there because I don’t know him. I assume his motivations are like those of most people discussing this topic – a goal of minimizing the negative outcomes of the pandemic.

        • Sameera, I believe any of the other aforementioned studies’ global IFR estimates are well above JI’s estimate, some by nearly an order of magnitude. I think Joshua did a fantastic job describing JI’s analytical and policy-related bias toward COVID; the problem isn’t just the estimate alone (although it’s problematic) but it’s all of the other COVID-related topics involving JI as well.

          I think you might be too quick to dismiss, with the blog equivalent of a handwave, JI’s lack of professionalism. I feel the ad-hominem in that Appendix was better suited for the dumpster fire of Twitter–not a peer reviewed journal. I don’t read as many peer-reviewed Epi journals as some on this board (maybe 3-5 articles a week for the past 3 years, so maybe only 600 articles), but I don’t think I’ve ever read such a direct, open attack on other researchers, where each of non-PhD status, lack of publication history, and competency were all called into question, in such a condescending manner. At the very least, authors often display some level of decorum and only passively-aggressively call into question others’ competence. :-) I just don’t understand the blind defense of JI, but maybe he runs in your circles, or is a personal friend, or something. The world is a confusing place.

      • Kant: “Hence people discuss all sorts of inane questions …. it often happens that of the disputants one appears as milking a he-goat, and the other as holding the sieve under.”

        Kant: [elsewhere] “For if a question is absurd in itself and calls for an answer where none is required, it not only brings shame on the propounder of the question, but may betray an incautious listener into absurd answers; thus presenting, as the ancients said, the ludicrous spectacle of one man milking a he-goat and the other holding a sieve underneath.”

        Lucian: [quoting Demonax]: “On seeing two philosophers very ignorantly debating a given subject, one asking silly questions and the other giving answers that were not at all to the point, he said: ‘Doesn’t it seem to you, friends, that one of these fellows is milking a he-goat and the other is holding a sieve form him!’ “

        • Prof. A: Because there have been no double-blind rcts to develop evidence (one way or the other) apropos wearing of masks in an epidemic of contagious respiratory disease, we have insufficient reason to act at all?

          Prof. B: Let us endeavor therefore to develop a double-blinded rct to develop said evidence!

          Prof. A: That’s the spirit: ‘Coniunctum est quod adhuc orbis et orbis erat!’

          C [a layman]: Now that they’re hard at work on the problem, I may put my worries to rest.

    • Sameera –

      > “Cruise ships are like an ideal experiment of a closed population. You know exactly who is there and at risk and you can measure everyone,” says John Ioannidis, an epidemiologist at Stanford University in California.

      I think that’s an odd statement. I can think of little that is less suitable for extrapolating health outcomes than a population of cruise passengers living on board a cruise ship. The sample is not at all representative and the living conditions are completely unrepresentative.

      • Joshua:

        Internal validity and external validity are different things. I take Ioannidis as saying that a cruise ship is like a lab experiment. Lab experiments have value because of their strong internal validity, even though they have external validity problems.

        • Sure. But what is less useful than an internally valid paradigm that has zero external validity?

          There can be judgment calls. This, I argue, is the exact problem with Ioannidis COVID science. You have to be explicit about the assumptions you’re making, and make post-stratification adjustments.

          If you want to talk about covid outcomes on a cruise ship, that’s fine, and make it clear that you AREN’T saying findings should be extrapolated more widely.

        • You can’t use the “lab-experiment” to say anything about outcomes such as IFRs (due to limited external validity, and the particular setup that might impact R, medical treatment etc.), but you can use it to show mechanisms, e.g. that airborne transmissions exist – the “lab-experiment” showed substantial evidence for that.

    • If “people inferred that John Ioannidis used the Diamond Princess as basis for his estimate” is because he said so:

      > If we assume that case fatality rate among individuals infected by SARS-CoV-2 is 0.3% in the general population — a mid-range guess from my Diamond Princess analysis — and that 1% of the U.S. population gets infected

      Personally, I don’t think the main problem of the estimate was the 0.3% (even though I would have found 0.6% or 0.9% more credible). It was the 1%. No effort whatsoever was made to justify why that optimistic assumption was more adequate that considering the potential wide spread that some epidemiologists had been warning about for weeks.

      • Thanks for pointing that out Carlos. I can see how different audiences would interpret that statement differently. As I posted before, John does a better job of explaining himself in person, lecture, or in an interview. I, e.g. viewed that estimate as a kind of hypothesizing and not a categorical claim.

        Yes, the estimate did not lend itself to support. I see that. But that has not been unique to John. Just about all have made one or more claims without support in their work. It’s a complex epistemic environment, with visibly mash up of arguments, theories, and rules of thumb translating into degrees of confusion for people like myself, a member of the public.

        My broader point is that John Ioannidis has called into question some of the data. And frankly, the authors of the Santa Clara Study did incorporate the criticism of some of the errors. That to me is an exercise in science.

      • Right. I think 0.3% was well within the range of plausible estimates in March, but it assumed a lot of undetected cases, which is *incompatible* with 1% of the US being infected.

        If you assume a lot of undetected cases, it would be more consistent to assume ~20% infected (like 2009) or 33% infected (rough estimate for 1918) = 200,000 to 330,000 deaths in US.

      • LOL, I totally get what you mean. I have been following the subject diligently; the deeper into the biology of the virus and kinetics of testing, the more like I have a ? on my forehead rather than a V [for vaccinated]. But I’m working up some courage to get an appt. As u picked up in my earlier postings, I think I contracted in about 10 months ago.

  16. From the start (I mean PLOS Med 2005) my problem has been that when I compare JI’s presentations in microscopic detail against the literature he discusses, I’d find biased selection of assumptions and evidence geared toward swaying opinion toward a foreordained conclusion* (rather than explaining assumptions and arraying all evidence so the audience can see for themselves if they agree that the conclusion is compelling in light of what is known so far). Thus the presentations looked like carefully crafted, highly technical analogs of the rhetoric found in high quality legal briefs, aimed at the court of policy opinion. But they were offered as if they were neutral presentation of evidence that everyone should trust for forming their own inferences and policy preferences; and until the pandemic provoked closer scrutiny, most observers took them that way, I think because they conformed to common beliefs (despite their being billed as “contrarian”, which before 2020 they were not).

    Now disillusioned JI fans like David Gorski are starting to recognize that JI’s presentations have been crafted with goals of advancing policy preferences rather than scientific knowledge (although the two goals are not independent). The biggest problem for me has been that sometimes my preferences are not far from JI’s; yet my deeper commitment is to improving our evidence base and its interpretation. That commitment stems from curiosity about what is really going on, understanding that as Feynman said of science “The first principle is that you must not fool yourself, and you are the easiest person to fool” and that we need to supply everyone with all the evidence if we are to avoid that trap. I fear that (until the pandemic provided a Waterloo) JI was demonstrating how, with skillfull rhetoric and sleight of hand, one can fool most scientists most of the time – at least if the message is one they were already disposed to believe (including flattery).

    *As an illustration of such a detail here is a Twitter thread I posted on May 5th 2020 in response to Daniels:
    https://twitter.com/SPilleron/status/1257628335963697153
    which I copy here without the numbering or breaks:
    … let’s consider Ioannidis using Switzerland as the comparator to Sweden here:
    https://www.youtube.com/watch?v=T-saAuXaPok&t=244s
    That 1:1 country comparison was the data presented for his statement that “they [the Swedes] have done fairly well”.

    [But] Sweden’s adjacent Nordic neighbors have had about a fifth the reported covid death rates. I think none of those Nordic countries are mid-Winter vacation spots for more Southerly Europeans. In contrast, Switzerland sits atop Lombardy, the hardest hit region in Europe, and where the open Swiss transalpine highways terminate. Until mid-March Switzerland had borders and skiing ful open. It is filled with famed ski areas; you can ski between some Swiss ski resorts and resorts in France, Italy, Austria (eg Ischgl, covid ground zero in Austria).

    So, which comparator should I choose to judge Sweden’s success so far? It’s Nordic neighbors or Switzerland? If I want to claim that Sweden is doing fairly well, I’d take Switzerland. If I want to guess the impact of its policy relative to lockdown, I’d take its neighbors.

    The punch line of sorts: I’m sympathetic to Ioannidis’s points about lockdowns; I just don’t trust his analyses. Whether comparing people, studies, or regions, credible effect estimation demands detailed mechanistic information to evaluate the comparisons, presented in full. That is just epidemiology 1A, evaluate bias sources. We should not draw read too much into raw comparisons of a few countries, yet our gut may do just that. We need instead slow critical deliberation of details, and wariness of persuasive delivery which glosses over that.

    • Sander as always offers a valuable critique. To be honest, I don’t think that John’s views are so ‘contrarian’. In listening to students of and colleagues of David Sackett and other gurus of the Evidence Based movement, back when I lived in Massachusets, I gathered that the state of science was precarious in many respects. I grant that I haven’t read every one of John’s articles. But with reasonable certainty, I can conclude that John echoes much of what has been in circulation for over 40 years, notwithstanding his support of Statistical Significance.

      I come at all this as a hobbyist, as Nasem Taleb has characterized me. Maybe I’m just lazy. LOL. Seriously though, I come at this by attempting to place myself in the patient position and consumer of medical research. Plus I was influenced by Serge Lang and Jerome Bruner.

      BTW, I just started Jonathan Howard’ss Cognitive Errors and Diagnostic Mistakes. Thought you might also be interested in it.

      Finally, it would be really great if John can respond to critiques.

    • On the contrary, please pen your thoughts on vaccines!

      I’m curious to see if there’s an anti vaccine position here that coming from you maybe more scientifically defensible than the usual layman crap.

      • Why aren’t the stats people demanding that all cause mortality and morbidity in the first week after the first dose get reported, particularly in nursing home patients?

        That hasn’t been reported anywhere. How is that possible in an honest investigation of the effects?

        And as for lockdowns, what is going on in New Zealand. Their hospitals are overrun

        https://www.nzherald.co.nz/nz/nz-hospitals-in-crisis-after-biggest-january-february-on-record/7ICBYUJ6KVNKYSP7DXMZNBOTDY/

        • Why does this matter? I don’t see the connection.

          Similarly for the NZ situation. That’s not covid related. So what’s the connection?

        • Last response in this thread- per andrew.

          It matters because it was reported Pfizer vaccine induced lymphocytopenia for that first week, which is immunosuppression. Then in the RCT there was 40% more cases of covid-like illness in that week (or maybe it was first two), then since then there have been half a dozen observational studies reporting ~40% increased rate of covid cases in those first weeks.

          Immunosuppression is going to do more than make you more susceptible to covid. So what about other infections, etc? It is very possible that tens of thousands of people have died from this already. But out of sight, out of mind.

          The reason for hospitals overrun in new zealand is unclear (why are there so many more sick people than usual?), but given the point of all these measures was to prevent that from happening why is it happening anyway?

          There are already covid strains that go undetectable on the standard PCR test btw, which will then easily bypass pcr-based border restrictions.

          People simply are not being informed about the actual risks and benefits, and that is just what we know to look for. The other missing data is PEG antibody levels and ADE after waning against new strains.

        • Anoneuoid –

          > The reason for hospitals overrun in new zealand is unclear (why are there so many more sick people than usual?), but given the point of all these measures was to prevent that from happening why is it happening anyway?

          Are you suggesting it’s because of vaccinations?

          Have you looked to see how many people have been vaccinated there?

          https://www.nzherald.co.nz/nz/covid-19-coronavirus-vaccine-tracker-how-many-kiwis-have-been-vaccinated-and-how-do-we-compare-with-the-rest-of-the-world/ENMCOHM5QW6W3UN6MRMCOQKO2U/

          So 0.3% of the population. If their hospitals are being overrun because of 0.3% of the population being vaccinated, I would think that places like Israel or the UK or the US, with much higher %’s vaccinated, would be completely shut down. Are you suggesting there’s something special abut Kiwis, or the vaccine there?

          And the article you linked (from March, 25) talked about hospitals being overwhelmed (by a few hundred patients over the previous year) in January and February – how many of those extra patients would have been because of vaccinations?

        • Sorry, that 0.3% would be fully vaccinated, not given one jab only (more like 1.4%)…which would be the measure of concern for your argument.

        • Yikes –

          That “few hundred” was poorly remembered from the following article (caveat lector), in reference to a hospital in Christchurch, not the country as a whole. The article also suggests that the current crisis may be at least partially explained by needed medical workers not being able to get into the country due to border restrictions, on top of longstanding, systemic issues:

          https://www.wsws.org/en/articles/2021/03/30/nzhe-m30.html

        • “And as for lockdowns, what is going on in New Zealand. Their hospitals are overrun”

          Well, the article you cite makes no connection with covid, pointing out that the problem has been building for years. Short staffing and a lack of capacity and what would appear to be poor planning.

          i.e. “There’s not enough staff to open up the new areas where new departments have been built” would indicate that staffing increases haven’t kept up with plans to increase capacity.

          and “College of Emergency Nurses chair Sue Stebbings said the crisis point had been building for years.”

          The only possible connection with covid is buried here:

          ‘”So we’ve got a chronic problem exacerbated by an acute problem relating to getting enough doctors … and then we’ve got people delaying being seen because of Covid and associated issues around access,” Dr Baddock said.

          “You’ve ended up with this tsunami of patients needing to be seen.”‘

          Without diving in deeply, it appears that New Zealand may be struggling with the costs of health care, like so many countries including ours.

          I doubt that there’s any way to tie this into your anti-vax position given that NZ isn’t vaccinating many, or to your anti-lockdown position since NZ’s been mostly back to normal for ages now.

        • Never said I had an anti-vax position. I have a pro-science position.

          Also never said that was due to vaccinations, which wouldn’t make much sense unless it is a variant that came from an immunosuppressed person that got vaccinated elsewhere.

          Something is going on leading to many more patients than usual and overrun hospitals which is what lockdowns, etc were supposed to prevent though.

        • “Never said I had an anti-vax position.”

          Pfft. All we hear from you is that vaccines are probably causing 10s of thousands of unrecorded deaths, it’s being intentionally swept under the rug, etc.

        • Just look at you …

          “It is very possible that tens of thousands of people have died from this already.”

          Speculation.

          “But out of sight, out of mind.”

          And you of course assume your speculation is truth, and that authorities are simply ignoring the tens of thousands that you claim have died.

          “I have a pro-science position.”

          That seems very unlikely.

        • An answer was provided in the response above. You should try reading it. If you still don’t understand, try reading it again until it is clearer.

        • He’s not anti abortion. He’s just pro life. :)

          Andrew, I am sorry for continuing but how can you be ok with this? These people are smearing someone who wants to see the data that exists on what happens the first week after first dose as anti-science.

          Just asking to see data that has already been collected is bad now.

        • Hi Anoneuoid:

          I do not like commenting directly on other commentators, and Andrew can get mad at me and take what actions he likes, I am fine with that. Perhaps Andrew’s comments to you come from the fact that you do not just make a comment to make a point, and perhaps one or two responses, you respond over and over and over again as if if you keep responding you will cudgel people into submission and into agreeing with you. And you do so not just on this topic, and maybe one or two others, but on almost any topic I can think of on this blog. Not only does it totally dominate and distract the timeline of the discussions, but in my view it takes away from the points you are making. i think some of the more ad hominem responses you see to your comments are in response to this. And I doubt Andrew is fine with these other responses, but I understand why he has reached out to you (among others).

          Something to think about.

        • you respond over and over and over again as if if you keep responding you will cudgel people into submission and into agreeing with you

          Why do I need to cudgel people on a data/stats blog to want to see data? Not a single person has ever responded:

          “Yes, it would be interesting to see what happens in that first week after first vaccination”

          The responses all discourage that.

        • “Why do I need to cudgel people on a data/stats blog to want to see data? Not a single person has ever responded”

          When data is not available, you put forth speculation, such as in the example I posted above, then expect people to treat your speculations as fact.

          When called out on your misinterpretations of papers you cite, you claim that the authors don’t understand the implications of their own work, and that one should reject the conclusions reached by the authors and accept yours instead.

          You expect us to believe that you know more about epidemiology, virology, and infectious disease than the experts in the field, and that you’ve got insights that all others have either missed or are dishonest about. That the mainstream view that data shows that mRNA vaccines for covid are safe and effective is wrong, and that they’re actually dangerous. That covid is no big deal at all, and that media articles on the pandemic foment unwarranted hysteria on the part of the public and government.

          Ditto climate science.

          And recently, ditto high-energy physics.

          You know more than all the specialists. Which makes you rather special … though possibly this has more to do with your evaluation of your personal talents and intelligence rather than any real world metric.

          Just sayin’…

  17. Sameera –

    Only about a year too late. If we’d had this month’s ago, this discussion of Ioannidis would be moot – and he could have kept is unsullied status as an icon.

    https://twitter.com/michaelmina_lab/status/1377454976885219330?s=19

    -snip-

    BIG NEWS!!

    Rapid Antigen “Paper Strip” Tests get over-the-counter use from @US_FDA!

    • NO prescription or doctor needed
    • NO CLIA waiver

    Simple, streamlined rapid tests will be available in US

    First companies:
    @AbbottNews BinaxNOW
    @QuidelDX Quickvue

    https://t.co/X4FB68amla

    • Hey Joshua,

      GREAT NEWS. I invited Michael Mina in September to present to my education task force, as an effort to introduce in school COVID19 testing, a project that I thought was critical to prevent viral spread. I offered testimony last July to the DC Council. To my surprise, the school administration followed up on the feasiblity of antigen testing.

      Now I kid Dr. Mina, by saying ‘we knew you back when’.

  18. I don’t think problems can be attributed solely to Jensen’s inequality. I think the main cause if the treatment of the pandemic issue as an estimation (where we might be interested to know where central moments lie) rather than a risk management problem (where tail risks is what interests).

    • I don’t know. Ideally, yes, but … in a situation where due to politics it’s likely that an overreaction with serious economic impacts would totally destroy the credibility of public health for at least a generation, I’m not sure the tail risks are all in one direction.

      If COVID *had* really turned out to have an IFR of 0.2%, what would have been left of the CDC by January 20, 2021?

  19. Some interesting data here:
    https://jamanetwork.com/journals/jama/fullarticle/2778234

    So 500k excess deaths. 350k attributed to covid, 30k to heart disease, 20k to “unintentional injuries”, 15k diabetes, and 10k each for stroke and alzheimers. That leaves another 65k not accounted for in that top-12 list.

    Early misuse of ventilators probably accounts for 20% of covid deaths (I am assuming the practice of early intubation has since been discarded everywhere…), then HCQ overdoses another 10% or so. That would be about 100k covid deaths due to medical error, leaving 250k.*

    There could be other harmful interventions that were used on covid patients (eg, reperfusion injury from suddenly exposing tissue that has been hypoxic for days to high oxygen), but even ignoring those we are already at ~50% of the excess deaths are not directly due to covid. Instead they are due, in one way or another, to the hysteria and response.

    * Ignoring sending covid patients into nursing homes, which happened in a half dozen states. That could be other 10-50k deaths due to the response.

    So far this has yet to get much attention, but it is looking quite possible, even probable, that the response killed more people than the virus in 2020.

    Questions regarding the most effective interventions to reduce the spread of the virus—for example, more testing, requirements to wear face masks, and stricter and longer lockdowns—become widely discussed in the popular and scientific press, informed largely by models that aimed to predict the health benefits of proposed interventions. Central to all these studies is recognition that inaction, or delayed action, will put millions of people unnecessarily at risk of serious illness or death.

    However, interventions to limit the spread of the coronavirus also carry negative health effects, which have yet to be considered systematically. Despite increasing evidence on the unintended, adverse effects of public health interventions such as social distancing and lockdown measures, there are few signs that policy decisions are being informed by a serious assessment and weighing of their harms on health.

    https://www.bmj.com/content/371/bmj.m4074

    • Positive cases were detected 7 days (1), 14 days (10), 20 days (12) and 23 days (3) after the first vaccination (Figure 1). Only 3/26 (12%) residents were symptomatic at the time of diagnosis while 12/26 (46%) positively tested residents developed symptoms in the further course. Overall case fatality rate was 9/26 (35%). Of note, 5 of the 9 patients with fatal outcome were diagnosed on day 20 after vaccination. All three residents who refused to receive a BNT162b2 vaccination were tested positive, but showed mild courses of the disease. No new cases of SARS-CoV-2 infection were detected 23 days after the first and two days after the boosting dose administration.

      https://academic.oup.com/cid/advance-article/doi/10.1093/cid/ciab299/6213878

  20. “Ioannidis just had the misfortune not to have an editor on that article”

    If John Ioannidis does not have professional and well funded PR-consultants, im eating a broom.

  21. Rahul: Thanks for raising your views.

    RE:

    Sure, you can reach herd immunity by natural infection. If you wanted to go that route we could all shed masks, stop social distancing and go about life as usual. Maybe even congregate around super spreaders and our herd would reach immunity faster!

    Problem is, natural infection carries with it a mortality / morbidity risk which is way higher than building up your immunity by using a vaccine.

    That’s where vaccines have a huge advantage over the natural infection route. A secondary point is that the antibody titres post vaccination seem higher and sustained than post natural infection ( on average).

    Finally, even taking your estimate that 30% of the US population has been infected, it’s still a far cry from herd immunity and vaccines are the only credible way to get there, IMO.
    —————

    I believe you are reading more into my raising the herd immunity estimates. My point was to convey what current and past CDC director estimates. Granting that X percent was infected and some portion of those recovered, I think it is important to get data on who and why they recovered. In other words, we focus excessively on those who have died or have had severe symptoms. It skews our thinking. Unfortunately, we don’t have the data we needed in March or April. Testing would have been useful. Even conducting serological testing, which, at the time, was questioned for its utility. Hopefully these rapid antigen trials in NC and TN can cull reliable data as to whether the at home antigen tests can reduce spread.

    Moreover, I follow the experts very closely. It is a bit disappointing to hear that the current vaccine may not address or more variants; AND perhaps make subsequent vaccine adaptations to variants even less effective; this concern was raised back in September too. But somehow the subject was off the radar b/c of the urgency of getting the vaccine distributed.

    My sense is that we really won’t know the effectiveness of our strategies for several months at the earliest. This is simply speculation on my part. And really we are all rendering opinions, some of which are sensible and quite accurate and other opinions that are steeped in uncertainty and lack of knowledge.

    • So why is it a surprise that vaccines may not be guaranteed to address all variants? We know that from flu already!

      Let’s say for arguments sake that we know for sure that next year’s mutants will need a new vaccine. Should it have modified our response in any way? How?

      Let’s say that the long term response is a new covid vaccine every year isn’t it still the best response we have?

      It’s one thing to just point out flaws of the status quo solution but unless you have an actionable strategy which would be better it’s all academic isn’t it? All approaches have flaws, yes, that’s hardly surprising.

      My problem is there’s a lot of muddying of the waters, speculation etc. out there which then gets subverted by the anti-vax / anti-science / conspiracy theory cohort.

      Idle speculation in this case doesn’t remain innocuous. It has real public health consequences.

  22. Hey there Rahul,

    Apologies, I don’t want to persist in this subject b/c as I mentioned earlier, reliable data is not in the offering on some dimensions of the pandemic. But let me address a few of your claims:
    ——
    RE:

    1. My problem is there’s a lot of muddying of the waters, speculation etc. out there which then gets subverted by the anti-vax / anti-science / conspiracy theory cohort.
    —-

    I refer to this as the fundamental attribution error. It ‘is the tendency for people to under-emphasize situational explanations for an individual’s observed behavior while over-emphasizing dispositional and personality-based explanations for their behavior.’ Wiki

    2. It’s one thing to just point out flaws of the status quo solution but unless you have an actionable strategy which would be better it’s all academic isn’t it? All approaches have flaws, yes, that’s hardly surprising.
    ——–
    This is a strawman’s argument. Wiki

    The concerns expressed by very intelligent experts is not about me. They are not anti-vax or vaccine hesitants. They are actively involved in trials, public health departments state and federal, etc. These questions about the current vaccine emerge b/c these variants are now in the US. And we don’t know yet if the current vaccine can address the variants adequately. At this time, the best that we can do is to launch a large scale testing effort to tamp down spread. CDC and NIH have launched in TN and NC.

    3. Let’s say that the long term response is a new covid vaccine every year isn’t it still the best response we have?

    I don’t know.

    4. Idle speculation in this case doesn’t remain innocuous. It has real public health consequences.
    —–

    I believe Stuart Firestein would disagree with your framing of the claim. Much of science has been rooted in speculation idle-wise or deliberatively. I think speculation is the driver of science.

  23. Rahul,

    RE: Problem is, natural infection carries with it a mortality / morbidity risk which is way higher than building up your immunity by using a vaccine.

    That’s where vaccines have a huge advantage over the natural infection route. A secondary point is that the antibody titres post vaccination seem higher and sustained than post natural infection ( on average).

    ———
    Can you give me references to studies which make these claims?

    • Joshua:

      I followed the link, and here’s what was added to replace the removed section about the cat photo etc.:

      Note added by the author in proof: An earlier pre-proof version of the paper contained an additional supplementary appendix that I have asked the journal to remove. The appendix was an earnest effort to explore in more depth whether technical competence issues or confirmation biases could explain why two overviews had strong, unidirectional biases that differentiated them from the other 4 overviews. I am grateful to commentators who suggested that this effort might be misinterpreted here. Dispassionate discussion of these technical and bias issues may be pursued in other fora. The notion that other people’s feelings may be hurt causes me more regret than the vilification that I have suffered, because my priority as a physician and epidemiologist has always been to care for other people, even more so in a time of major crisis. Commotion shifts the discussion away from the scientific essence of the main paper, effectively silencing scientific debate. I particularly applaud students who have offered me their wisdom and critique in this occasion, as I consider myself the least knowledgeable student of all.

      I agree with Meyerowitz-Katz that it’s good that Ioannidis removed the earlier section. Disparaging remarks about cat photos have no place in science. But I’m kinda baffled by the paragraph that replaces it, for two reasons:

      1. Ioannidis writes, “Commotion shifts the discussion away from the scientific essence of the main paper, effectively silencing scientific debate.” But scientific debate wasn’t silenced, “effectively” or otherwise. See the P.S. from my above post. It’s not at all hard to separate the scientific debate from discussion of preferences regarding cat photos.

      2. He concludes, “I consider myself the least knowledgeable student of all.” What does that mean? He’s published lots of scientific papers with policy implications! You wouldn’t do that if you consider yourself the “least knowledgeable student,” would you? I feel like there’s something I’m missing here.

      • Andrew –

        He has called different opinions about COVID outcomes “highly irresponsible” and “science fiction” That doesn’t track with wanting to avoid commotion. Those kinds of comments will always cause commotion.

        Meyerowitz-Katz also pointed out that it is condescending to describe removing the personal attack as a response to “hurt feelings.” It shows no accountability.

        Carl Bergstrom says it’s highly unusual for a section of a published preprint to just be removed whole like that.

        I’m inclined to not judge Ioannidis’ motivations. But from a sheer logic standpoint, it doesn’t take a genius to know that saying you’re trying to avoid “hurt feelings” would come across as condescending. And saying he wants to avoid commotion after some of the comments he’s made has the same logical problem as his complaining about being seen as political, after co-authoring with fellows of political think tanks and going on publicity tours of political media outlets to advocate for political policies.

        • Joshua:

          I don’t really care about the condescending thing. Hurt feelings are important, but they don’t affect the science. I’m more bothered by his claim that scientific debate is being effectively silenced. I keep hearing statements like this from various people, but I don’t see that debate is being silenced at all.

          From a communications standpoint, I think Ioannidis made a good call to remove those paragraphs, but then he made a mistake to add that new paragraph. Maybe he’d be better off having a blog where he can stash his more controversial takes, such as the idea that he’s the least knowledgeable student . . .

        • It’s not the “hurt feelings thing” per se that’s relevant, imo, but that in making that statement he’s shirking accountability.

          That doesn’t affect his science directly. But it’s relevant to how he’s couched his science in some kind of “above the fray” framing.

          I have no problem with scientists wanting to be advocates. But if they want to do so, they should own it. And they should bend over backwards to be accountable for their errors – which Ioannidis has not done. This is one example but the same pattern extends into his COVID science errors more directly.

          Unaccountable scientific advocacy is commonplace. It’s understandable why it happens. But it gives accountable scientific advocacy a bad name.

        • Andrew –

          > Hurt feelings are important, but they don’t affect the science.

          I don’t understand that response. The resin to remove that section wasn’t because it “hurt someone’s feelings” but because it was fallacious science. Meyerowitz-Katz’s glasses and cat had nothing to do with the science.

          It should have been removed because it should never have been put in – not because putting it in would hurt someone’s feelings (which I doubt it did anyway).

          The intersting question is WHY Ioannidis put it in there and that is what he should have addressed. And you might think that removing it is a good communications strategy but I don’t. I don’t thominknjisf removing it will change anything at all. Because it isn’t being accountable.

          A good communication strategy would have been to apologize in an honest explanation of why he out something in there that had no place being there in the first place. That actually, might have created some increased goo faith exchange among people with different scientific views.

        • Joshua:

          I agree with you that the discussions of cat pictures, hurt feelings, etc., are a distraction. In correcting his paper, Ioannidis removed one distraction (cat pictures etc.) but then added another (hurt feelings). Why he did this, I don’t know, but it’s not uncommon for people to write things that don’t advance their cause, just because they get carried away. It happens to me all the time! One advantage of publishing in a journal is that the editors can sometimes catch these lapses and tell us to remove them. But editors are busy; ultimately it’s the author’s responsibility to communicate appropriately and effectively.

        • Willard,

          Why do you characterize John Ioannidis as a ‘contrarian’? I view him as ‘eclectic’. Nothing he has said has struck me as an outlier of the evidence based medicine movement. What I would speculate is that he has managed to collaborate with many experts across a few different disciplines. Not a bad thing.

        • Andrew –

          Thanks for the further explanation. I got it now.

          Yeah, sometimes people do stuff that doesn’t advance their goals, that is the wrong thing to do.

          I think I did the wrong thing once, but I could be wrong about that.

        • I’m more bothered by his claim that scientific debate is being effectively silenced.

          This has been going on since the ventilators at the beginning. There are probably still places using “early intubation” because there was never a big announcement/paper that stopping that practice reduced mortality by 5-10x, info just kind of spread informally around the critical care community.

          A way to reduce the mortality of the worst pandemic in a century by an order of magnitude (and save tons of money, PPE, and ICU space at the same time!) was met largely with silence. If you can’t see the problem here I don’t know what to say, the malfunctioning system couldn’t be more in your face.

          In general, there has been a systematic suppression of information about the risks and exaggeration of the benefits when it comes to expensive/dangerous interventions. For cheap/safe interventions it has been the opposite.

        • And of course no one has been held responsible for putting that “do not delay intubation” recommendation in the WHO/DoD/etc guidelines based off no evidence.

          If no one is ever held accountable for medical errors that result in the deaths of tens/hundreds of thousands of people it is only going to get worse and worse.

          But I suspect the thought of doing that hits the taboo button for most people here (who are exceptionally educated and intelligent), so we can expect it to get worse.

  24. Why do people like yourself keep misrepresenting what John originally said? He never predicted 10,000 deaths. He listed that as a low end estimate that was almost surely going to be wrong. It was in contrast to the high end estimates circulating at the time. He thought both of them would be wrong – and he was right. It’s dishonest to claim that was his prediction. It was not. And anyone can read it. I know why so many people keep distorting what he wrote, but it’s not a great reflection on these scientists.

    It’s also not true that he “attacked” Gideon Meyerowitz-Katz. On the contrary, he praised him and went on to excuse his sloppy work. Of course, John’s critics have been relentlessly dishonest, so they distorted what he wrote in that appendix. Because the appendix was a subtle critique of those senior critics who threw science out the window and followed an inexperienced grad student doing a complicated meta-analysis and making basic errors. They did this because his meta-analysis matched their ideology. John is criticizing this lack of judgment and that is why these scientists distort what he wrote – they don’t want people to look at the science John presents and realize they abandoned science in favor of ideology.

    • Ryan:

      You say, “people like yourself keep misrepresenting what John originally said.” “People like myself” is a pretty vague category, and I certainly can’t be held responsible for what is done by people like myself who are not actually me.

      So are you saying that I keep misrepresenting what John originally said? If so, I’d appreciate being told exactly what I got wrong. I’m happy to correct myself, but first I need to know what the mistake actually is.

      The 10,000 number was not presented as a prediction; nor was it presented as a “low end estimate.” From the linked article:

      If we assume that case fatality rate among individuals infected by SARS-CoV-2 is 0.3% in the general population — a mid-range guess from my Diamond Princess analysis — and that 1% of the U.S. population gets infected (about 3.3 million people), this would translate to about 10,000 deaths. . . .

      10,000 is not presented as a lower bound; it is presented as a calculation based on two numbers, one of which is characterized as a “mid-range guess.” Nowhere did it say in that article that 10,000 was almost surely going to be wrong.

      And then:

      Some worry that the 68 deaths from Covid-19 in the U.S. as of March 16 will increase exponentially to 680, 6,800, 68,000, 680,000 … along with similar catastrophic patterns around the globe. Is that a realistic scenario, or bad science fiction? How can we tell at what point such a curve might stop?

      The higher number of 680,000 isn’t far from what we have already (currently listed as 562,000 on Google).

      Regarding the comments about the cat photos etc.: I agree with you that jumping on Ioannidis regarding that is silly. He made an error of tone by putting blog-like comments into a published research article, but that has no bearing on the scientific merits of the argument, one way or another. I’d hardly consider a mockery of cat photos to be a “subtle critique,” but really that shouldn’t matter anyway.

      Finally, I don’t think anyone in this discussion has abandoned science in favor of ideology; I think these are difficult scientific questions, the answers have important policy implications that interact in complicated ways with political and scientific ideologies, and people get impassioned and sometimes go overboard in their rhetoric. That’s just the way things go, and we should try not to let that get in the way of scientific discussions.

    • Ryan –

      Going back to that article, here’s an issue I have:

      In the most pessimistic scenario, which I do not espouse, if the new coronavirus infects 60% of the global population and 1% of the infected people die, that will translate into more than 40 million deaths globally, matching the 1918 influenza pandemic.

      The vast majority of this hecatomb would be people with limited life expectancies. That’s in contrast to 1918, when many young people died.

      One can only hope that, much like in 1918, life will continue. Conversely, with lockdowns of months, if not years, life largely stops, short-term and long-term consequences are entirely unknown, and billions, not just millions, of lives may be eventually at stake.

      Ioannidis talks here as if “long term consequences” will only occur as a result of interventions enacted to mitigate the effects of the pandemic.

      That’s a strange and obviously wrong construct, and frankly it’s very surprising to see that kind of simplistic rhetoric from an experienced and respected epidemiologist who, no doubt, has spend years and years dealing with analyses to assess the differential effect of public health interventions.

      In that article and elsewhere, he speaks of the impact of interventions on missed medical care, or other negative outcomes potentially associated with interventions as costs.” And absolutely he is right to call for public health officials to consider those potential costs in their recommendations. But obviously, as surely he knows, determining with whether such costs associated with interventions are caused by those interventions rather than the pandemic itself (or to what extent they are caused by the one mechanism or the other), is an immensely difficult task.

      What I consider very disappointing is to see Ioannidis glide right past the difficulties of that task, in rhetoric that suggests only one counterfactual scenario is applicable; as if there’s a binary choice of risking billions, not millions of lives by enacting interventions, or not taking a risk of unknown short- and long-term consequences through not enacting them.

      >… they don’t want people to look at the science John presents and realize they abandoned science in favor of ideology.

      I don’t throw around accusations such as that, because (perhaps unlike you) I can’t get into people’s heads to mind-read), but I will say that in my opinion, when Ioannidis makes such a public display of scientific analyses that obviously fail basic requirements of the scientific process, he forfeits the right to stand behind a sheiks of “science” as a justification for his work, and from such a safe space launch criticisms on the science of others.

Leave a Reply to Rahul Cancel reply

Your email address will not be published. Required fields are marked *