Skip to content
 

(Some) forecasting for COVID-19 has failed: a discussion of Taleb and Ioannidis et al.

Nassim Taleb points us to this pair of papers:

On single point forecasts for fat tailed variables, by Nassim Taleb

Forecasting for COVID-19 has failed, by John Ioannidis, Sally Cripps, and Martin Tanner

The two articles agree in their mistrust of media-certified experts.

Here’s Taleb:

Both forecasters and their critics are wrong: At the onset of the COVID-19 pandemic, many researcher groups and agencies produced single point “forecasts” for the pandemic — most relied on the compartmental SIR model, sometimes supplemented with cellular automata. The prevailing idea is that producing a numerical estimate is how science is done, and how science-informed decision-making ought to be done: bean counters producing precise numbers.

Well, no. That’s not how “science is done”, at least in this domain, and that’s not how informed decision-making ought to be done. Furthermore, subsequently, many criticized the predictions because these did not play out (no surprise there). This is also wrong. Both forecasters (who missed) and their critics were wrong — and the forecasters would have been wrong even if they got the prediction right. . . .

Here are Ioannidis et al.:

COVID-19 is a major acute crisis with unpredictable consequences. Many scientists have struggled to make forecasts about its impact. However, despite involving many excellent modelers, best intentions, and highly sophisticated tools, forecasting efforts have largely failed. . . . Despite these obvious failures, epidemic forecasting continued to thrive, perhaps because vastly erroneous predictions typically lacked serious consequences. . . .

But Taleb makes a point that the others miss:

What’s relevant is the distributional forecast, not the point forecast. This came up last month, when political columnist John Fund criticized Imperial College epidemiologist Neil Ferguson for getting a bunch of forecasts wrong—but it turned out that what Fund was doing was taking the upper bounds of Ferguson’s forecasts of past public health crises and pointing out that they were overestimates of the actual number of deaths in each case. That was just wrong on Fund’s part: it’s the nature of upper bounds that they will generally too high.

Ioannidis et al. do show problems with the distributional forecasts from the University of Washington IHME, and indeed Cripps and Tanner, the two coauthors of the recent article with Ioannidis, are also coauthors of an earlier paper pointing out problems with the IHME forecasts. The IHME model has lots of problems, and I don’t think it’s right to take its flaws as representative of more serious statistical models for epidemic progression. It’s more fair to take this as a flaw of the news media for sometimes presenting IHME forecasts uncritically.

One thing that bothers me about the Ioannidis et al. article is that it does not at all address the previous statistical failures of Ioannidis’s work in this area.

1. In their above-linked 11 June article, Ioannidis et al. write:

Despite these obvious failures, epidemic forecasting continued to thrive, perhaps because vastly erroneous predictions typically lacked serious consequences. Actually, erroneous predictions may have been even useful. . . .

But just two months earlier, on 9 Apr, Ioannidis said:

If I were to make an informed estimate based on the limited testing data we have, I would say that covid-19 will result in fewer than 40,000 deaths this season in the USA.

That’s fine—Ioannidis was careful at the time to condition his estimate based on the limitations of available data, and you can learn a lot in two months. Still, 40,000 was an erroneous prediction, and at the very least this error should cause him to reassess his assumptions. And if he’s gonna write about erroneous predictions, he could mention his own.

2. In their above-linked 11 June article, Ioannidis et al. list the following problems with forecasts: “Wrong assumptions in the modeling,” “High sensitivity of estimates,” “Lack of incorporation of epidemiological features,” “Lack of transparency,” “Errors”, “Lack of expertise in crucial disciplines,” “Groupthink and bandwagon effects,” and “Selective reporting.”

All of these problems arose with the much discussed Stanford antibody study. Ioannidis was only the 16th of 17 authors on this study, so I’m not blaming him for the wrong assumptions, lack of transparency, errors, etc., but I don’t think he’s disavowed that paper either. My point is not to use this as a “gotcha” on Ioannidis but rather to say that it’s hard to know what to make of these criticisms given that they all apply to work that he stands by. Maybe his article should’ve been titled, “Our forecasting for COVID-19 has failed,” and he could’ve criticized the errors and lack of expertise in that Stanford study.

I don’t think the Imperial College models (for example, here) are so flawed. They’re not perfect—wrong assumptions and high sensitivity are unavoidable!—but they are transparent and I think they’re a way forward. Full disclosure: I’ve worked with the first author of that paper, and my colleagues and I helped him with some of the modeling.

The sad truth, I’m afraid, is that Taleb is right: point forecasts are close to useless, and distributional forecasts are really hard. We have to try our best and use all available resources.

P.S. Maybe we could get law professor Richard Epstein to weigh in on this one. He’s the real expert here.

226 Comments

  1. zbicyclist says:

    I note that the purpose of epidemic forecasting is often to make your predictions wrong — i.e. as a guide to take actions that will make your “predictions” too high.

    Any predictions for COVID-19 are likely to depend on how well we control superspreading occasions, since these seem to be much more dangerous than, say, taking a bike ride.

    • Ryan says:

      Absolutely. In this case with such a strong feedback loop between policy decisions and the outcome of the forecast, I think that forecasts need to carry a tag to describe them as either “forecasts conditional on no policy change” or “forecasts anticipating actual policy changes”. And changes in human behaviour in the voluntary adoption of social distancing measures as the pandemic progresses fits somewhere between those two categories as well.

      There’s been a long list of critics arguing that the early pandemic forecasts in many countries were horribly wrong because some countries (like Australia or NZ) didn’t experience terrible outbreaks despite early forecasts saying they would. But this criticism is rather like saying that your doctor advised you to wear a seatbelt or you might die in a car crash – but then saying that since you wore your seatbelt and didn’t crash the doctor’s advice was worthless. That’s clearly a false statement, and doesn’t invalidate the policy recommendation, though clearly it makes testing the accuracy of the forecast after the fact near impossible because we can’t observe a “no policy change” actual outcome.

  2. A similar point is made here (in that prediction intervals rely on distributional forecasts):
    https://twitter.com/josephnwalker/status/1256767436512129026

  3. Don says:

    Both criticisms still miss another important point. Exponential growth is like chaos: in the long term it is intrinsically unpredictable.

    Nobody seems to be paying attention to this obvious fact (see this preprint
    https://arxiv.org/abs/2004.08842

    • Andrew says:

      Don:

      Yes, good math point. It’s a stochastic differential equation with noisy inputs.

    • Zhou Fang says:

      I do love the pithy abstract on this paper!

      Still I think it’s not quite like chaos in some respects. Specifically, you’d expect to be able to make conditional assertions like “if you don’t social distance cases would increase”. Whereas in a pure chaotic system you wouldn’t even have that monotonicity.

      • Don says:

        Good point. Still, chaotic systems display have bifurcations (kinda like confining until you get R0=1).

        Anyway, I guess that appealing to chaos in that preprint was a matter of analogy.

        By the way they should have used Stan instead of jags :-)

      • confused says:

        You’d think. But in the US at least there seems to be very little correlation between measures in place vs. what the outbreak is doing. The states that look like they’re getting worse right now are mostly neither among the most “open” or the most “closed”.

        Now maybe that’s just because official measures are poor predictors of / ineffective at controlling actual behavior at the individual level.

        • Zhou Fang says:

          That just means there’s a lot of confounding.

          • confused says:

            It might.

            Or it might mean that the dynamics of this disease (highly dependent on super-spreaders, large difference between congregate settings and elsewhere) lead it to behave radically differently in different places that have different behaviors/contact-patterns/living conditions/etc… some of those differences coming from social distancing, but some of them coming from pre-existing differences.

            IE – South Dakota, outside meatpacking plants, might be as “socially distanced” with just banning mass gatherings as a dense urban area with only essential businesses open. This could explain why exponential growth appeared to stop after the initial big outbreak without new measures being imposed, and it now seems to be growing

            There are other states that seem anomalous.

            IE, is it just confounding of a relationship which holds universally, or are pre-existing behaviors/contact-patterns/living conditions/etc different enough across the US that the effects of different levels of measures can’t be extrapolated usefully between dissimilar areas?

            • Anoneuoid says:

              Arent around half the US deaths still in nursing homes? Did we ever find out why nursing homes were ordered by some states but not others to take covid patients?

              Especially in NYC when there were 3 empty hospitals at the time. I doubt any modeller would have included a scenario of sending infected people into nursing homes. Maybe in the future it should be standard to include policies like that.

              • Martha (Smith) says:

                ” I doubt any modeller would have included a scenario of sending infected people into nursing homes. Maybe in the future it should be standard to include policies like that.”

                Good point.

              • Mendel says:

                What *evidence* is there that this is an effect that needs to be modeled?

              • Anoneuoid says:

                What *evidence* is there that this is an effect that needs to be modeled?

                That half the deaths came from nursing homes. And most of the deaths are from states who sent patients into nursing homes. Also, just collective prior knowledge and reason (ie, science).

              • Anoneuoid says:

                Apparently it was NY, NJ, CA, PA, and MI that issued such orders: https://twitter.com/SteveScalise/status/1272661471382700032

                I didn’t find better info, maybe that list is incomplete or inaccurate? For example how long did each state do this for, etc.

                Here is covid deaths according to covidtracking and population according to the R usmap package. You can see that, besides California, all the states are in the top ten in deaths per 100k pop:
                https://pastebin.com/PjBq750F

                I get deaths per 100k pop to be 60 vs 24 in the ones that issued the order vs those that didnt. The total deaths were ~55k for each but the 5 states in question make up only 28% of the population (16% without California, which reported “only” 5 out of those 55k deaths).

    • David Young says:

      Yes Don, It’s a critical point. The range of results for these problems is huge. And it also shows I think that scientific advice in this area should depend more on experience and data about fatality and hospitalization rates than on modeling. In this sense, criticizing the whole modeling enterprise is fully justified.

    • jim says:

      Well we can all be thankful that public officials seem to have given up on forecasting fallen back on talking about how to behave to reduce risk of infection, informed by a more realistic concept of how the virus spreads.

    • M3 says:

      I believe this is exactly Talebs point and why he is using EVT instead of using SIR models. He is actually saying something even more profound, that SIR models even with stochastic (and noisy) inputs will underestimate the fat tail behaviour of pandemics. So your posterior distributions in your paper still underestimate the worst case scenarios.

      “Second, epidemiological models such as the susceptible–infectious–recovered (SIR) differential equations11, sometimes supplemented with simulation experiments12, while useful for scientific discussions for the bulk of the distributions of infections and deaths, or for understanding the dynamics of events after they have happened, should not be used for precautionary risk management, which should focus on maxima and tail exposures instead. It is not rigorous to use naive (but reassuring) statistics, such as the expected average outcome of compartmental models, or one or more point estimates, as a motivation for policies. Owing to the compounding effect of parameter uncertainty, the tail-wags-the-dog effect easily invalidates both point estimates and scenario analyses. However, it is encouraging to note that the impact of parameter uncertainty on the scenarios generated by epidemiological models has recently started to be examined more carefully13.

      Extreme value theory is a natural candidate to handle pandemics. It was developed as a means to cope with maxima14, and it has subsequently evolved to deal with tail risk in a robust way, even with a limited number of observations and their associated uncertainty3. In the Netherlands, for example, EVT has been used to get a handle on the distribution of the maxima—and not the average—of sea levels in order to build dams and dykes high and strong enough for the safety of its citizens2.”
      https://www.nature.com/articles/s41567-020-0921-x

  4. Peter says:

    Sage and the UK government made much of Ferguson’s “reasonable worst -case scenario” and seems to have based policy on that. Who is responsible for that? Do modelled advising on policy have a responsibility to make sure they are not misunderstood? Especially when there maybe conflicts of interest when lockdown can help pharmaceutical business models.

  5. Joseph Candelora says:

    Ioannidis is a hypocrite who seems insistent on getting things wrong. Taleb is arguing against a strawman.

    Worse than Ioannidis’s 40,000 death prediction was this line in the March STAT article that rocketed him to Covid fame:
    “If we assume that case fatality rate among individuals infected by SARS-CoV-2 is 0.3% in the general population — a mid-range guess from my Diamond Princess analysis — and that 1% of the U.S. population gets infected (about 3.3 million people), this would translate to about 10,000 deaths.”
    This suggestion that we might end up with 1% prevalence in an unmitigated epidemic made me think he wasn’t arguing in good faith on this stuff. And that was proven for me in an interview he claimed that Ferguson and the ICL revised their estimated IFR downward significantly after issuing their March 17th paper (didn’t happen) in service of walking back and reducing their prediction of 500,000 UK deaths (also didn’t happen): https://youtu.be/cwPqmLoZA4s?t=2609

    Taleb complains that “many researcher groups and agencies produced single point ‘forecasts’ for the pandemic”. But none of the serious/influential forecasts that I remember were single point forecasts, and he doesn’t deign to actually give us an example of one.

    Not a fan of either paper.

    • Phil says:

      Ionnidis’s “10,000” was itself a point forecast, based on at least three of the things he criticizes: “Wrong assumptions in the modeling,” “High sensitivity of estimates,” “Lack of incorporation of epidemiological features”. And it was probably fairly influential; certainly it got a lot of press coverage. So there’s one example.

      • Joseph Candelora says:

        But it wasn’t a serious forecast.

        • confused says:

          Yeah, I think it was a bad idea to come out with that number, but I don’t think it was meant to be a *prediction* of what would actually occur as much as an illustration of how different assumptions can give very different results, demonstrating how large the uncertainty was (or at least how large Ioannidis thought that it was).

          But the only way I can see 1% infection being even vaguely defensible is if the idea was that there might be *such* a strong dependence on super-spreading events that the disease basically couldn’t spread outside population density levels comparable to European/Chinese cities and cruise ships (IE – not in car-dependent US cities). Which seems pretty extreme.

          • Joshua says:

            confused –

            > but I don’t think it was meant to be a *prediction* of what would actually occur as much as an illustration of how different assumptions can give very different results, demonstrating how large the uncertainty was (or at least how large Ioannidis thought that it was).

            I don’t entirely disagree. I mostly disagree in that I think that Ioannidis has made a series of projections that are all to support his policy advocacy. They’ve all been low (some by a way wide margin and some arguably within a reasonable range if on the low side). But I agree a bit in that it’s better to think of these estimates as conditional projections rather than predictions *if the analysts are clear that their projections are conditional*. I don’t think that Ioannidis has done that well, however.

            Nonetheless, my sense is that don’t extend the same generosity to those that you think made higher projections, that caused in your estimation an “overreaction.”

            • Joshua says:

              For example, when Ioannidis said that he thinks that “there’s no reason to fear,” that wasn’t aimed at stressing uncertainty.

              It was aimed at projecting certainty and minimizing uncertainty.

            • confused says:

              >>I don’t think that Ioannidis has done that well, however.

              >>Nonetheless, my sense is that don’t extend the same generosity to those that you think made higher projections, that caused in your estimation an “overreaction.”

              I probably come across that way, but that’s not my actual position.

              I think the actual Imperial College London model was not terribly unreasonable at the time it was published. The model itself did not really predict 2.2 million deaths in the US. I think the 1.1 million deaths for “limited mitigation” was still too high, but for reasons that probably weren’t clear in mid-March, especially for someone in London (who may not have personal experience of how different contact and travel patterns are in say 75% of the US vs. the UK).

              It’s the way it was presented in the media etc. that I think was utterly irresponsible. The 2.2 million deaths was based on “no action” – on March 16th, a lot of action had already been taken, though much of it was voluntary (businesses going to teleworking) or local (school closures driven by school districts) and thus not captured in the dates of statewide orders.

              It might have been better not to publish the 2.2 million estimate at all, though, since it was based on a scenario that was already known to not be the case.

          • Yes, I agree with you in the main. I did not interpret the 10,000 as a prediction nation-wide. It was based on the Princess Cruise Ship situation, as I remember. Let’s be mindful of the fact that many of these predictions are basically guesses that are being modeled statistized as I like to inject a bit of humor.

            • Carlos Ungil says:

              > I did not interpret the 10,000 as a prediction nation-wide.

              Maybe a it was a non-prediction nation-wide. But when you say that “if we assume that case fatality rate among individuals infected by SARS-CoV-2 is 0.3% in the general population (…) and that 1% of the U.S. population gets infected (about 3.3 million people) this would translate to about 10,000 deaths” definitely you’re definitely calculating a nation-wide something.

              • Hi Carlos,

                I heard John Ioannidis give a larger figure of deaths before I read this 10,000 deaths figure. I did read the article when it came out. But heard a video earlier. I did not record the days. Yes, that seems like a nation-wide something. In the context of John’s entire article, I didn’t interpret the calculation to mean nationwide. I should reconstruct the timeline of his analysis to see why I did not interpret the 10,000 deaths nationwide.

                Wellll…

                Had it not been for the shortages of medical supplies and equipment, we might have had far fewer deaths. I am basing this on the measures that several countries took. Namely South Korea, Taiwan, China, New Zealand, and Germany. Of course, this is now hindsight. Massive testing took place in most of the countries listed. We had shortages I gather 1.3% of the US pop is infected.

                It is noted that we have not had a handle on the numbers of asymptomatic carriers either. A limitation of all efforts to model the IFR it seems.

        • Phil says:

          No True Scotsman…

          • Joseph Candelora says:

            You’re telling me that garbage like Ioannidis’s back-of-the envelope 10,000 point estimate in a STAT article justifies Taleb railing against “many reasearcher groups and agencies”?

            Taleb asserts* that “Both forecasters and their critics are wrong… many reasearcher groups and agencies produced single point ‘forecasts’ for the pandemic…”. And goes on to say “Both forecasters (who missed) and their critics were wrong — and the forecasters would have been wrong even if they got the prediction right.”

            Ioannidis here falls into the “critics” bucket that Taleb is criticizing.

            My question remains — who were the many researcher groups and agencies that produced single point forecasts?

            *Full excerpt from Taleb’s paper:
            “Both forecasters and their critics are wrong: At the onset of the COVID-19 pandemic, many researcher groups and agencies produced single point “forecasts” for the pandemic — most relied on the compartmental SIR model, sometimes supplemented with cellular automata. The prevailing idea is that producing a numerical estimate is how science is done, and how science-informed decision-making ought to be done: bean counters producing precise numbers.

            Well, no. That’s not how “science is done”, at least in this domain, and that’s not how informed decision-making ought to be done. Furthermore, subsequently, many criticized the predictions because these did not play out (no surprise there). This is also wrong. Both forecasters (who missed) and their critics were wrong — and the forecasters would have been wrong even if they got the prediction right.”

            • Phil says:

              Joseph,
              You said “But none of the serious/influential forecasts that I remember were single point forecasts, and [Taleb] doesn’t deign to actually give us an example of one.” So I gave you an example. You don’t like this example, but I think that’s a No True Scotsman issue.

              Ionnidis made an influential point forecast that had at least three of the characteristics he (Ionnidis himself) says are problems with forecasts. I find this kind of amusing — physician, heal thyself!

              • Joseph Candelora says:

                Did you not read Taleb’s paper? And also not read the section of the paper I quoted directly before my comment about serious/influential forecasts?

                Here, I’ll reproduce it again.

                —-
                Taleb complains that “many researcher groups and agencies produced single point ‘forecasts’ for the pandemic”. But none of the serious/influential forecasts that I remember were single point forecasts, and he doesn’t deign to actually give us an example of one.
                —–

                You’re telling me that Ioannidis’s back-of-the-envelope trash inhis Stay article was a serious/influential single point forecast by a researcher groups and/or agency? Come on man.

                That’s not what I was asking for.

        • Zad says:

          Not a serious forecast, eh? Then why write it on a serious medical news publication (STAT News) read by many serious clinicians and medical researchers? Surely someone as smart as Ioannidis would know that writing such statements would result in very serious interpretations by very serious policy makers

          • Joseph Candelora says:

            Of course Ioannidis has been trying to pain the discussion by throwing out garbage numbers like that.

            But an obviously-garbage-on-its-face estimate like that does not rise to the level of a serious/influential forecast by a researcher group or agency.

            And if trash like that _is_ what Taleb was referring to when he coyly asserted “many researcher groups and agencies produced single point ‘forecasts’ for the pandemic”, then his paper is an even worse strawman beating than I thought.

            And it’s even worse malpractice to tar the legitimate researcher groups and agencies doing much more serious work with this blanket condemnation based on crap like that Ioannidis estimate.

  6. Joshua says:

    Not precisely on point to the main thrust of your post, but I would also note from the Ionnaides et al. article:

    > Predictions for hospital and ICU bed requirements were also entirely misinforming. Public leaders trusted models (sometimes even black boxes without disclosed methodology) inferring massively overwhelmed health care capacity (Table 1) [3]. However, eventually very few hospitals were stressed, for a couple of weeks. Most hospitals maintained largely empty wards, waiting for tsunamis that never came. The general population was locked and placed in horror-alert to save the health system from collapsing. Tragically, many health systems faced major adverse consequences, not by COVID-19 cases overload, but for very different reasons. Patients with heart attacks avoided visiting hospitals for care [4], important treatments (e.g. for cancer) were unjustifiably delayed [5], mental health suffered [6]. With damaged operations, many hospitals started losing personnel, reducing capacity to face future crises (e.g. a second wave). With massive new unemployment, more people may lose health insurance. The prospects of starvation and of lack of control for other infectious diseases (like tuberculosis, malaria, and childhood communicable diseases for which vaccination is hindered by the COVID-19 measures) are dire [7,8].

    As seems all to (unfortunately) typical of Ioannidis’ work on the pandemic, this paragraph just flat out ignores important uncertainties. For example, the projections about the pressure on hospitals were made based on existing resources, not based on changes that would take place in the resources available.

    Now it’s certainly reasonable to criticize the forecasters for building into their projections a belief that hospital resources would remain static, but Ioannidis et al.’s criticism isn’t entirely reasonable in the sense that the projections themselves could have been accurate based on the assumption that you made. That consideration doesn’t necessarily make such projections more useful…it’s complicated. But there is a lack of comprehensiveness in Ionannidis et al.’s critique, IMO.

    Similarly:

    > Blindly acting based on extreme value theory alone would be sensible if we lived in the times of the Antonine plague or even in 1890[…] Upon acquiring solid evidence about the epidemiological features of new outbreaks, implausible, exaggerated forecasts [19] should be abandoned. Otherwise, they may cause more harm than the virus itself.

    “Blindly acting” looks awfully strawmanish – not particularly useful, IMO. And “implausible, exaggerated” characterizations can always be made in hindsight, but as a practical matter when confronted with an immediate concern, hindsight isn’t available. If we used the same rhetorical form as they use, we’d say that they are arguing that we could never act to address a crisis because there’s a chance that our projections might be wrong.

    • confused says:

      Maybe it was sensible preparedness vs. over-reaction. But in the western US (and I think everywhere except a few Northeast/Midwest urban areas) the difference between what was predicted about hospital capacity vs. what happened is pretty stark. I definitely think missed cancer screenings and such are a real and significant problem, and were an unnecessary harm in all but about 2-3 states.

      Yes, there are definitely risks from under-reaction as well as over-reaction. But I think it is true that we could set an upper bound on how bad this could plausibly get even back in March, maybe even February once the WHO report from China came out. It was pretty clear this was not going to be the next 1918 Flu even then.

      • Joshua says:

        confused –

        > Maybe it was sensible preparedness vs. over-reaction. But in the western US (and I think everywhere except a few Northeast/Midwest urban areas) the difference between what was predicted about hospital capacity vs. what happened is pretty stark.

        Again, as with Ioannidis et al., this is a bit unfair. The projections were made in a time that was, necessarily, ignorant to some degree of the subsequent changes that would take place – such as shelter in place orders and generally behavioral changes among the public. You go to war with the evidence you have. In hindsight, it may well turn out to be wrong. The ways in which it was wrong should be acknowledged and incorporated into future modeling.

        The alternative to acting with incomplete information is to say, well, we might be wrong about making projections so we should just throw up our hands and do nothing. Other than that, there is no alternative to acting with incomplete information – so unless that’s your argument then you need to not play games with unfair standards.

        And what makes the rhetorical positioning of Iannidis et al. even more troubling is that he frequently, if not necessarily uniformly, treats the uncertainties selectively – such as when he talks about the uncertainties in “died with” verses “died from” categorization but fails to acknowledge the potential undercounting due to deaths at home or at care facilities where no tests are performed.

        Likewise, he frequently talks about economic harm of government mandated shelter in place orders – but ignores the uncertainties of how much economic harm would have been incurred absent such orders as the potential result of far greater deaths, sickness, perhaps panic from deaths and sickness, people not being able to stay home from work and collect unemployment or not get fired, etc.

        > I definitely think missed cancer screenings and such are a real and significant problem, and were an unnecessary harm in all but about 2-3 states.

        Sure. Problems exist. But when you say “unnecessary” you are embedding a counterfactual belief that ignores uncertainties. You actually don’t know what would have been unnecessary had shelter in place orders, and behavioral changes, not manifest.

        > Yes, there are definitely risks from under-reaction as well as over-reaction. But I think it is true that we could set an upper bound on how bad this could plausibly get even back in March, maybe even February once the WHO report from China came out.

        Doing so requires counterfactual thinking: what would have happened had things been different? The problem is that counterfactual thinking, to really be supportable, requires an incredibly high standard of carefully calibrated evidence. We are not able to reach that standard at this point, ever we ever reach that stage.

        So you have to decide in the face of uncertainty. And you have to incorporate your values accordingly. My own values are largely informed by a high prioritization of the welfare of healthcare workers – because they’re on the front line risking their lives for the sake of others. It is their work, along with essential workers, that allow people who are more vulnerable to stay safer. IMO, we should multiply the welfare of each healthcare worker and essential worker into your calculations. You might have other factors that you’d assign more value to – say each cancer diagnosis not made. Those values need to be made explicit – and not dealt with as some kind of putatively unsubjective component of a pure science.

        • confused says:

          >>You actually don’t know what would have been unnecessary had shelter in place orders, and behavioral changes, not manifest.

          Well, not all US states had shelter-in-place orders. There were *some* behavioral changes even in those states, but nothing like what was expected to be necessary in mid-March to prevent unrestrained exponential growth.

          It really does seem fairly clear to me that extrapolating from dense regions like Wuhan and Lombardy produced results that were OK for the Northeast megalopolis (also very dense, high mass-transit, etc.) but not representative at all of the rest of the US.

          >>Likewise, he frequently talks about economic harm of government mandated shelter in place orders – but ignores the uncertainties of how much economic harm would have been incurred absent such orders

          True to a degree, but if the main government message had been “think of this like the 1957/1968 flu pandemics” I find it very hard to believe we would have seen anything like the economic harm we’ve seen. Industries like airlines and tourism, sure, but…

          >>And you have to incorporate your values accordingly. My own values are largely informed by a high prioritization of the welfare of healthcare workers – because they’re on the front line risking their lives for the sake of others.

          See, I would strongly disagree with weighting some lives as “worth more” just because of their job.

          I can see an argument for doing a life-year adjustment, because otherwise you are arguably undervaluing the young (who have maybe 60 years to lose rather than 1, or 5, or 10), IE more actual “life” (though not “lives”) are lost – but even that can get tricky.

          I definitely wouldn’t use any other adjustment. Valuing healthcare workers’ lives more because of their value to society then opens questions of the flip side… what about people who don’t work? Carving out ad-hoc categories is really shaky ground.

          • Joshua says:

            confused –

            > Well, not all US states had shelter-in-place orders. There were *some* behavioral changes even in those states, but nothing like what was expected to be necessary in mid-March to prevent unrestrained exponential growth.

            I don’t know on what basis you’re making that claim. Even in some places where there weren’t SIP orders, schools were closed. People’s mobility decreased markedly. People stopped going to theaters, to bars, etc. We don’t know precisely what changes had what kind of impact, but the kinds of changes that likely had the biggest impact were pretty widespread, at least for a while. The point being, we just don’t have enough info for making highly certain counterfactual arguments as Ioannidis does. Again, his argument is that there is a huge differential and negative impact from SIP orders. In point of fact, we just don’t know that’s the case. Again, if there had been none, many of the same negative effects would likely have occurred, and some would likely have been worse. Imagine the huge impact of employees not feeling safe to go into work, but also not being able to stay home and collect unemployment. What do they do? Stay home and get fired? Same with parents sending their kids to school. Suppose they felt unsafe doing so? Should they stay home from work to take care of their kids and get fired and not be able to collect unemployment? Would that have been worse than the SIP orders? We just don’t know. And it’s sloppy to make an assumption.

            > It really does seem fairly clear to me that extrapolating from dense regions like Wuhan and Lombardy produced results that were OK for the Northeast megalopolis (also very dense, high mass-transit, etc.) but not representative at all of the rest of the US.

            Sure. But that’s a separate issue.

            > True to a degree, but if the main government message had been “think of this like the 1957/1968 flu pandemics” I find it very hard to believe we would have seen anything like the economic harm we’ve seen. Industries like airlines and tourism, sure, but…

            You find it hard to believe. That’s fine. But don’t pretend that you’re making some kind of empirical analysis as Ioannidis seems to me to be doing.

            > See, I would strongly disagree with weighting some lives as “worth more” just because of their job.

            I’m not suggesting that someone’s life is worth more because of their job. Not at all. I’m suggesting that if someone sacrifices in order to enable other people to stay safe, if someone sacrifices in order for other people to be able to buy food, they their welfare (and their ability to make those sacrifices) deserves extra consideration when you’re making policy decisions.

            > I definitely wouldn’t use any other adjustment. Valuing healthcare workers’ lives more because of their value to society then opens questions of the flip side… what about people who don’t work? Carving out ad-hoc categories is really shaky ground.

            See above.

            • confused says:

              >>I don’t know on what basis you’re making that claim. Even in some places where there weren’t SIP orders, schools were closed. People’s mobility decreased markedly. People stopped going to theaters, to bars, etc.

              Sure, but in some states, those changes have largely reversed. Schools wouldn’t be in session in mid-June anyway, in a normal year. Yet cases aren’t resurging in South Dakota and Wisconsin (probably the most “open” states in the US – South Dakota did the least all along, and Wisconsin’s order was totally struck down rather than gradually relaxed as elsewhere).

              Which states are doing badly doesn’t really seem to correlate very strongly with any of the things we’d *expect* to be predictive. It mostly seems to correlate with geography (Northeast and Midwest originally, now Southwest and some of the South), for unclear reasons.

              >>Again, if there had been none, many of the same negative effects would likely have occurred, and some would likely have been worse.

              I can accept “possibly have been worse”, but not “likely have been worse”. I just don’t think history bears that out (in an assumed case where basically no governments took strong actions, probably leading to a rather different media and public response — IE a more restrained “public image of the disease” more like 1957/1968.

              Accurate messaging about risks early on would have helped a lot.

              >>Imagine the huge impact of employees not feeling safe to go into work, but also not being able to stay home and collect unemployment. What do they do? Stay home and get fired? Same with parents sending their kids to school. Suppose they felt unsafe doing so?

              If the public messaging had been clearer on the differential risk with age, and the fact that it’s clearly less dangerous than regular flu for school-age populations, there probably would have been much less fear among parents and the majority of the workforce.

              >>Sure. But that’s a separate issue.

              Not really, I don’t think. The models assumed rapid exponential growth for “the US”, as if the US was one homogeneous population — people were quoting R0 values for “the US” or even for “the disease”, as if it would be comparable in New York City and South Dakota.

              If that underlying assumption is wrong, the models are basically worthless.

              >>I’m not suggesting that someone’s life is worth more because of their job. Not at all. I’m suggesting that if someone sacrifices in order to enable other people to stay safe, if someone sacrifices in order for other people to be able to buy food, they their welfare (and their ability to make those sacrifices) deserves extra consideration when you’re making policy decisions.

              How is that different in practice from valuing their lives more, when it comes to making actual decisions?

      • In my opinion what happened around the US was that stay at home orders worked, and then around the country people pretended that things would have worked out that way even if they hadn’t stayed at home, and all the public health people were full of it.

        So, they started reopening around late May, and of course in the reopening people are still doing social distancing and masks and things, somewhat, so it’s not growing as crazy fast as it was. But basically it’s growing, in many places. Such as:

        https://nbviewer.jupyter.org/github/dlakelan/JuliaDataTutorials/blob/master/COVID-monitoring.ipynb

        AK, AL, AR, AZ, CA, FL, GA, NC, NV, OK, OR, SC, TX, UT, WA

        So having bought ourselves about 3 months of time to prepare for this pandemic by ramping up case contact tracing into the hundreds of thousands of workers, and manufacture N95 masks at a couple million a day for months on end, and do many other things that were called “the dance” by Tomas Pueyo on his second medium.com post (see Mendel’s reference below)… Instead we did none of it, we’re pretending everything is all done and there was no problem, and most people are going to get caught off guard when around July 4th the extent of the insufficiently mitigated proper first wave becomes very obvious in their region

        Sure, it won’t look like Lombardy throughout every county of the US… but that doesn’t matter, because we’ll have some large metropolitan regions where we will have at least some significant hospital overload. The growth rate is now closer to doubling every 10-15 days instead of every 3. But at this rate for example South Carolina will have 2000 cases a day by July 1 in mid July they’ll have death rates in the hundreds per day.

        Of course it’s always possible the virus has mutated, or that summertime makes the virus less virulent due to say vitamin D or temperature effects or whatever. But looking at the obvious exponential growth in SC it’s hard to understand how they’ll be anything but in big trouble by early July. Similarly in NC, and CA, and NV where the casinos are all operating …

        • Chris Wilson says:

          +1 to everything Daniel said.
          Unfortunately, I don’t see the political will being mustered to do anything like the shutdowns again until there is obvious catastrophe playing out. And even there, it will have to be at the local level. And even there, look at Montgomery Alabama as an example of a city failing to pass a mask ordinance that their doctors are pleading for. Of course, the vote broke down on racial lines and 80% of the Covid patients there are African American. Sigh.
          The other thing to note is that CARES act unemployment benefits run out end of July. Moratoriums on evictions and foreclosures are also set to start phasing out. So, lots of economic pain and uncertainty to throw into the mix! Fun times :)

          • Martha (Smith) says:

            Yes. And then there’s what was discussed in the news this morning: That people in the upper income brackets have been cutting their spending, which has reduced employment/ income for the people in the lower income brackets. We need to do something about this — like lowering income taxes for the lower income brackets, while raising income taxes for the upper brackets to more than make up for the reduced taxes from the lower brackets, and allocating the extra revenue to essential services (including health care) for the lower income brackets. (But I’m skeptical that there is the political will to do this more than a token amount.)

            • Martha, your solution might seem logical, but I would argue that it would continue the practice of complex and gameable taxation which has led to enormous problems that are generally unacknowledged. For example, when poor people are able to afford to live on their low wages because they are taxed very little and have subsidized housing and such… and then they work hard to make a little more money… the *marginal* taxes on those extra dollars can be enormous… 50% for example. Combined with loss of qualification for services/subsidies, it can easily become a net negative, even DRAMATICALLY negative outcome to earn more. This traps people in poverty. I’m sure that’s an outcome you wouldn’t want.

              A reasonable alternative which is also incredibly simple: Eliminate income and payroll taxes as they exist today in favor of a flat tax of about 30% on income and a universal basic income of about 10% of GDP/capita. Even those who are not working would take in $6000/yr per person, and every dollar you earn, you take home 70% of it, regardless of how much you earn. Everyone could easily determine whether some new job opportunity was a good idea for them etc. way WAY better.

              • Martha (Smith) says:

                Daniel,
                Your example seems to be supposing that what I was proposing would be some kind of permanent change; I was thinking of something appropriate for the current situation, which would be modified as the situation changes. (Although perhaps the idea of modification as the situation changes might be pie-in-the-sky.)

              • Even for “temporary” situations, the perverse incentives can be very real. There’s a big issue right now with unemployment insurance providing more income than the work people are unemployed from. This means if they go back to work they lose income.

                A UBI and flat tax would ensure that *whatever* your situation, you have some reasonable level of income, enough to keep you from being utterly homeless and starving… while also ensuring that whenever you provide products or services your take home income always increases.

              • Chris Wilson says:

                I’m going to channel my inner Georgist and say we should go after wealth and unimproved land value taxes first :) A flat 2% wealth tax on all American households funds your UBI for instance. We could be even slicker and exempt the first million, which still leaves ~70% of the tax base (IIRC), and go a bit higher, say 3%. Arguably, a higher 5-6% bracket on wealth over say 50 million wouldn’t be hard to justify.
                Unimproved land value taxes remain the perfect swish. Economically efficient. No perverse disincentives anywhere. This is basically socializing a portion of value that is absolutely 100% socially created by anyone’s definition.
                Basically, I think we should soak the rent-seeking class and get real about the implications of non-ergodic, multiplicative accumulation of wealth (https://ergodicityeconomics.com/2017/08/14/wealth-redistribution-and-interest-rates/)
                If we get creative, I think we could fund our entire government at all scales, and a robust social safety net, with nothing but wealth tax, unimproved land value taxes, a modest VAT, and some assorted taxes on a couple key resources (carbon tax) and any resource rents.

              • Martha (Smith) says:

                Chris said,
                “I’m going to channel my inner Georgist and say we should go after wealth and unimproved land value taxes first”

                I’ve got a problem with the. concept of “unimproved land value tax” — namely, that what the real estate system calls “unimproved land” is actually land that is serving a purpose — in particular, providing ecosystem services.

              • Anoneuoid says:

                Unimproved land value taxes remain the perfect swish. Economically efficient. No perverse disincentives anywhere

                Like Martha said (I think), wont this incentivize people building unneccesary structures on that land?

              • Mendel says:

                Wikipedia:

                A land value tax or location value tax (LVT), also called a site valuation tax, split rate tax, or site-value rating, is an ad valorem levy on the unimproved value of land. Unlike property taxes, it disregards the value of buildings, personal property and other improvements to real estate.[1] A land value tax is generally favored by economists as (unlike other taxes) it does not cause economic inefficiency, and it tends to reduce inequality.[2]

                Building a structure on land does not change its valuation for land tax purposes; the tax is levied on the value of the land that it would have if it was not improved, usually by multiplying its size with a tax value for the general location.

                This tax burdens land owners, and has the potential to hit farmers disproportionally hard.

              • Chris Wilson says:

                No, farmers would get the usual agricultural exemptions, etc. A properly implemented land valuation tax would hit landowners who hang onto urban land and extract rent…which in turn is a big part of why small/mid-size cities often have downtowns that struggle. The valuation per sq ft of land increases exponentially with density and location. As noted, no one is penalized for making improvements – in fact, improvements are encouraged under this kind of scheme!

              • Anoneuoid says:

                farmers would get the usual agricultural exemptions, etc.

                So there is incentive to qualify as “farmer” in the eyes of the government. Actually, did you know that depending where they are cows, chicken, and rats are not considered “animals” by the federal government? Instead it’s livestock, poultry, and research organism.

                As noted, no one is penalized for making improvements – in fact, improvements are encouraged under this kind of scheme!

                Can’t you see this is a perverse incentive?

              • Chris Wilson says:

                Anoneuoid et al,

                I think this topic requires more careful analysis and discussion, and I may have contributed to over-simplifying. Yes, I am generally familiar with ag policies – I work in an Agronomy department and have been a farmer myself :) It is pretty easy already to qualify for ag exemptions, but this is regulated at county level – which it probably should be.

                So, what I should have said is, this kind of tax is designed to *disincentive* deadweight. The extent of this disincentive is directly proportional to desirability of location so will be much higher in urban areas than rural areas. You are not *incentivized* to make improvements that *don’t generate value*. And you are only incentivized to make improvements in direct proportion to their value. So, if you have a house, and you make improvements to it, your tax bill would not go up. Was it worth it? That’s entirely up to you. The tax is neutral with respect to this kind of thing. If you have a business, and you make improvements, and business improves you *WIN* under this kind of taxation, rather than incur additional tax liabilities.

                To be super clear, the goal of this kind of taxation is to tax purely speculative land holding and unearned rents – accrued due to the general improvements surrounding land and thus not one’s own industry in any sense. The holders of land are actually incentivized directly or indirectly to either be getting personal value or creating social value from it, or renting it to others who are creating value – precisely so they can pay their taxes! This actually means that incentives of landholders and their business-running tenants come into closer alignment AFAICT.

                I admit, this is a counter-intuitive proposal. We are so ingrained to think of labor and the fruits of labor as the natural target of taxation, but this is quite odd if you pull back for a second.

                All that said, I am not convinced 100% by classical Georgism. I don’t think we should make this kind of taxation do all the lifting, hence wealth + VAT also make sense to me. I am not opposed to income tax overall, but I think we have allowed the goalposts to shift way too much in favor of unearned wealth accumulation and rent-seeking and our society basically cannot get on a better path until we break up this unholy alliance of monopolists, rent-seekers and politicians.

              • Anoneuoid says:

                I’m just saying that all taxes lead to some form of perverse incentive. What is wrong with conserving your land? It is basically saving resources for a rainy day.

        • dhogaza says:

          “In my opinion what happened around the US was that stay at home orders worked, and then around the country people pretended that things would have worked out that way even if they hadn’t stayed at home, and all the public health people were full of it.”

          Well, that’s how I read this statement by Ioannidis:

          “Predictions for hospital and ICU bed requirements were also entirely misinforming. Public leaders trusted models (sometimes even black boxes without disclosed methodology) inferring massively overwhelmed health care capacity (Table 1) [3]. However, eventually very few hospitals were stressed, for a couple of weeks. Most hospitals maintained largely empty wards, waiting for tsunamis that never came. The general population was locked and placed in horror-alert to save the health system from collapsing. “

          He could be saying that hey, the lockdown measures worked, hooray! But … no, that’s not where he’s coming from. He’s essentially arguing that it wasn’t necessary and that things would’ve worked out the same if we’d done nothing, as you put it.

          Then we move to sheer hyperbole:

          “The prospects of starvation and of lack of control for other infectious diseases (like tuberculosis, malaria, and childhood communicable diseases for which vaccination is hindered by the COVID-19 measures) are dire”

          I believe he’s still talking about the US here. I’ve been hearing people complain about gaining wait during lockdown … and no one complaining about starving.

          • dhogaza says:

            Gaining weight … though there’s been a lot of waiting, too :)

          • confused says:

            The starvation / communicable diseases bit is I think more about how the shutdown of the US economy affects the world economy and thus poverty in less developed countries. Not starvation in the US itself. (I think the mention of malaria is key there…)

            • Stephen Cooper says:

              confused – it is kind of you to bother.

              Instruction of the ignorant is a virtue.

              For those people reading this who are not trying to score infighting statistician points, please reflect on this.

              The effects of individual human beings – Cuomo and his kulakization of the low-income elderly in nursing homes being a good example – had an effect on human misery much larger, over the course of the three months, than any measured or measurable effect caused by the unconscious, amoral, and unpredictable nature of our little fellow creature (some say organic, some say not quite organic) whose closest recognizable forebear was born somewhere last year somewhere in China.

              When I was young, people like the people super-invested in the types of arguments foregrounded here were called eggheads – the fragility of their ego, the instability of their passionate search for the truth and the limited goals they set themselves were the reason for that name.

              • Stephen Cooper says:

                By the way, Andrew G., if you are reading this, I was not including you among the eggheads.

                Not that you should care one way or the other. I disagree with you on a lot – not that you should care – but I have no desire to insult you in any way on your own website. You seem like an honest person who does not get super-invested in your own stances.

          • I am not sure whether John is referring to the US exclusively. But I do think that if it were not for that ‘10,000’ figure John’s analysis more broadly is more correct than the other analyses that came out. Specifically, on the basis of his analysis of Italy’s experience, the disease had a disproportionate impact on the elderly with and without co-morbidities. The real problem was that we lagged in massive testing, which is a theme over which there is considerable agreement. We do need better numbers of asymptomatic and pre-symptomatic carriers. That requires massive testing. Oxford noted that 5% to 80% of carriers were asymptomatic or pre-symptomatic. Huh? Okkkkkk.

            • Carlos Ungil says:

              > John’s analysis more broadly is more correct than the other analyses that came out. Specifically, on the basis of his analysis of Italy’s experience, the disease had a disproportionate impact on the elderly with and without co-morbidities.

              Do you think that nobody had noticed that? The following was published more than four months ago, before there was even a single death in Italy (only three cases had been detected in at that point in Rome and there had been only three deaths reported outside of China/Taiwan):

              “The ≥80 age group had the highest case fatality rate of all age groups at 14.8%. Case fatality rate for males was 2.8% and for females was 1.7%. By occupation, patients who reported being retirees had the highest case fatality rate at 5.1%, and patients in Hubei Province had a >7-fold higher case fatality rate at 2.9% compared to patients in other provinces (0.4%). While patients who reported no comorbid conditions had a case fatality rate of 0.9%, patients with comorbid conditions had much higher rates—10.5% for those with cardiovascular disease, 7.3% for diabetes, 6.3% for chronic respiratory disease, 6.0% for hypertension, and 5.6% for cancer.”

            • Joseph Candelora says:

              Ignoring the parts that Ioannidis got wrong, he was roughly on par with the analyses of others, not better. This is not an impressive outcome.

              The age profile of the disease had been studied, published and was in heavy use before Ioannidis published anything.

              Not sure what your Oxford non-sequitur is about.

              • Hi Carlos,

                Thanks for responding. I was referring back to this article co-authored by Johh, which I should have posted also. Apologies.

                During the COVID-19 Pandemic

                https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2764369

                I am not sure that all that many here in the States knew much of anything until February. My own speculation has been that the virus was here in the US perhaps as early as mid-December. Again, just speculation > a hunch. We tend to entertain hunches in these situations. I think there is pressure to provide forecasts and analysis. So I don’t fault experts on that score. We seem to have been ill-prepared on many fronts. Particularly in New York.

                I have followed nearly every expert weighing in on the COVID-19 situation. What many of us non-statisticians/consumers of statistics have noted is that it is exceedingly difficult to discern the statistical assumptions made, notwithstanding the fact that a good deal of analyses is quite technical and obscurely quantitative.

                There are a few highly competent wordsmiths in my observation. And journalistic accounts, even the most accurate and eloquent, do not get the acclaim that they deserve.

                think that ultimately citizens have to be able to evaluate the merit and demerits of the analyses.

                It’s worth reading Frank Von Hippel’s book on the role of the Citizen-Scientist and the extent to which such a role is hindered by scientific advisory boards/commissions who largely provide technical advice pertaining to public policy to the Executive branches of the Federal and State Governments. Thus marginalizing the necessity for citizens to weigh-in. They bear the costs and consequences of opaque data/research methods, and the commercialization of public health guidance.

                Oh that Oxford study? Here ya go.
                https://www.cebm.net/covid-19/covid-19-what-proportion-are-asymptomatic/

              • Joseph Candelora says:

                @Sameera

                You’ve linked to a study published April 7 recurving data through April 1.

                Verity et al posted the pre-print containing their age-stratification estimates on March 9: https://www.medrxiv.org/content/10.1101/2020.03.09.20033357v1.

                Those mortality estimates are what Imperial College London used in their March 16 study.

                All of this cwoek was well completed before Ioannidis published his Stat article on March 17, or his paper in early April.

                What is the relevance of the Oxford article you linked?

            • Joshua says:

              > Ignoring the parts that Ioannidis got wrong, he was roughly on par with the analyses of others, not better.

              Heh. In other words, “Other than that, how was the play Mrs. Lincoln?”

              Or

              The operation was a success but the patient died.

              • Well, everyone makes mistakes, right? It should be urged that before we lambast someone for his mistakes that we list the ones we have made. It would amount to a penance maybe. But that is asking too much.

              • Joshua says:

                Sameera –

                > Well, everyone makes mistakes, right? It should be urged that before we lambast someone for his mistakes that we list the ones we have made. It would amount to a penance maybe. But that is asking too much.

                I completely agree with you! And I applaud you for reminding people of such. It is a hugely important thing to keep in mind. I was writing as much to you in another comment.

                It isn’t the mistakes for which I criticize Ioannidis, but other aspects of what he’s been presenting.

                My point is that that same approach should be extended to everyone attempting to analyze the evidence, and that Ioannidis should do that as well. One of my primary criticisms of Ioannidis is exactly that he hasn’t done that. For example, he called the Imperial College projections “science fiction.”

                (I’m also critical of certain aspects of his methodology, and because I feel that he’s engaging in a political process while portraying himself as merely engaged in science even as he throws shade at others doing science. If you engage in a political policy discussion, just be straight forward that’s what you’re doing).

                And to be clear, I am not defending personal attacks against him. I consider those to be counterproductive.

              • confused says:

                I’m not defending Ioannidis’ approach in general – it definitely seems “one-sided”.

                But was the “science fiction” comment in regard to the ICL model itself, or the 2.2 million deaths number that was being quoted widely in the media? It’s arguably an accurate description of the latter, since that number was contingent on a situation (absolutely no mitigation action, even on an individual or local level) which was already known to be untrue on March 16.

              • Joshua says:

                confused –

                > But was the “science fiction” comment in regard to the ICL model itself, or the 2.2 million deaths number that was being quoted widely in the media? It’s arguably an accurate description of the latter, since that number was contingent on a situation (absolutely no mitigation action, even on an individual or local level) which was already known to be untrue on March 16.

                He and Katz giggled about how the “predictions” of 2 million were “science fiction.” And exactly in one with Sameera’s comment that bugged me because it want a prediction, it was a conditional projection.

                They completely ignored that aspect.

                That is exactly the kind of rhetorical posting that I find disappointing, and unscientific, and counterproductive towards the goal of reasoned exchange of view that I assume both aspire to achieve.

              • Joshua says:

                confused –

                > It’s arguably an accurate description of the latter, since that number was contingent on a situation (absolutely no mitigation action, even on an individual or local level) which was already known to be untrue on March 16

                I think this is problematic also.

                I see this happening with climate science all the time. People take high range projections thst have confidence intervals attached, act as if they were predictions of what would happen with no uncertainty, and then complain that the “predictions” were wrong and therefore the whole scientific endeavor is fatally flawed and biased.

                There is nothing wrong with making a worst case projection to inform the cost/benefit analysis – as long as it is made clear that it is a low probability/worst case scenario. If that is not communicated properly, than sure that communication effort deserves criticism. But so does it deserve criticism when people leverage rhetorical spin and conflate conditional projections with “predictions” of what’s going to happen so they can discredit modelers for producing models that are wrong but useful.

              • confused says:

                My point is that the 2.2 million wasn’t a high range (involving various forms of uncertainty) projection from an existing starting point, it was a projection from a counterfactual starting point. It wasn’t posed as “this is a low-probability outcome”, it was posed as “this is the likely outcome if X is done” but we already were not doing X.

                This is a criticism of PR/reporting, not of the model itself.

              • Mendel says:

                The nonpharmaceutical interventions do not change the 2.2 million number, isn’t that Taleb’s point?
                If we assume 1% lethality and 70% infected at least once until we achieve population immunity, then we’re going to have 2.2 million deaths in the US no matter what (but see below), the nonpharmaceutical interventions just shift them back into the tail instead of having them all in 3 months.

                From the situation we’re in, back to the normal we had before, the only way is across these deaths unless
                a) we deploy a vaccine, or
                b) we develop a cure (or simply get better at treatment), or
                c) we succeed in shifting infections away from the vulnerable people, or
                d) we contain the virus like New Zealand, but globally.

                If you find this number too big and want to do some psychological manipulation, you come up with unrealistic assumptions to field a number that is way too low, and you’re anchoring the perception elsewhere where people don’t take this as seriously as they should.

              • Joshua says:

                Mendel –

                > the nonpharmaceutical interventions just shift them back into the tail instead of having them all in 3 months.

                I think that there’s an argument to be made that the degree of fatality is, to some extent at least, a function of how much pressure there is on the medical system. Likewise, better tracing and isolation can, at least potentially, affect the fatality rate by virtue of something like reducing viral load – even if theoretically, ultimately the same number of people would be infected.

                The same works for the economic costs. There is a complex interaction between the rate of spread and the degree of impact. A faster spread that leads to a higher fatality rate might be less costly because it could lead to a shorter term of impact. Or it could be more costly because the depth of the impact would be more dramatic, and it might not diminish rapidly enough, relatively, to equalize the greater costs of a more intense spread.

              • Joseph Candelora says:

                Counterfactual is a bit strong. The results were made available on March 16, but certainly they had been working on it for weeks. And the steps taken we’re all temporary anyway — e.g. HISD just gave the week before spring break off, with the stated intention of reopening after spring break.

                So the unmitigated epidemic is no more a counterfactual than a scenario where people practice 100% compliance with distancing. They give a range of potential outcomes. These were illustrations where individual behavior is the independent variable; you obviously confound things by feeding in your personal opinions of how people will behave.

                And bear in mind that people like Ioannidis himself were then (and are still) making the argument that Covid would simply be noise against the backdrop of the flu season, so assuming that people would do anything differently than they do for the flu (i.e. nothing) was not justified for the unmitigated scenario.

              • confused says:

                >>Counterfactual is a bit strong. The results were made available on March 16, but certainly they had been working on it for weeks.

                Sure; again, the model itself is not the problem, the PR/media reporting is. That scenario may have been realistic when the model was run, but not when the results were published and the

                >>And the steps taken we’re all temporary anyway — e.g. HISD just gave the week before spring break off, with the stated intention of reopening after spring break.

                School closures in TX were originally very short-term, but not necessarily the businesses going to telework etc. And I think the latter is probably more significant for COVID.

                >>you obviously confound things by feeding in your personal opinions of how people will behave.

                I’d say that’s a prior, not a confounder ;)

                It’s not wrong for the modelers to show a “no action at all” result as one extreme of the possibilities.

                It is wrong for media etc. to report this as a prediction that “2.2 million Americans will die if we don’t lock down”.

                >> assuming that people would do anything differently than they do for the flu (i.e. nothing) was not justified for the unmitigated scenario.

                I disagree, because if the scenario was going to 2.2 million deaths, it would become obvious much earlier than that that what was actually happening was worse than flu, and behavior would change.

        • confused says:

          I think some of those states are not actually growing, just finding cases more effectively.

          And even if they all are, what about South Dakota and Wisconsin, which are more “open” than any of those?

          I really doubt we will have overloaded hospitals in multiple metro areas in the US in a month from now. I guess we’ll see…

        • Joseph Candelora says:

          I’ve looked at new hospitalization counts in a few of those places.

          In Florida, it’s essentially level since late March. I don’t see any reason to be concerned about unconstrained growth there.

          In Arizona, it’s been growing exponentially since early May, but with a doubling time of about 4 weeks.

          I couldn’t find data for all of Texas, but in the Tx Medical Center (major hospital system in Houston), it’s been growing exponentially since mid/late May, but with a doubling time of 3 weeks.

          Contrast that with NY in late March, which had a doubling time of 3 or 4 days.

          So those states (not FL so much) have something they need to watch out for, but it’s nothing like when it blew up in Italy or NYC. In the growth phases there, if you wanted to take an action that wouldn’t have effect for about 2 weeks, you’d be faced with a full order of magnitude more cases before you change took effect. In TX/AZ, you’re looking at less than double the case count under same circumstances.

          My view is that our distancing measures have been sticky enough that they’re still having significant effect. I mean, my company is still work at home countrywide.

          There’s definitely a possibility that even further opening could cause transmission rates to spoke to the point that things could totally get away from us, but given the testing capacity in place and overall monitoring resources, I think we’ll be able to push the limit of resource exhaustion and then choose at that point whether we want to lock down and cool it off or if we’re just going to accept the overload.

          • confused says:

            >>My view is that our distancing measures have been sticky enough that they’re still having significant effect. I mean, my company is still work at home countrywide.

            Definitely this. My workplace is also work at home, in TX, which is relatively open (though not as much as South Dakota or Wisconsin).

            I also think it matters that most of the US is lower population-density and lower mass-transit use than NYC or European cities. (And American demographics skew younger than European ones, as well – and some of those apparently growing states are among the youngest. UT is the youngest state by a large margin, but AK and TX are also very low.)

        • Hi Daniel or anyone else here:

          It is astounding to me that the Oxford study estimates that there are between 5 to 80% asymptomatics. What kind of help is that?

        • confused says:

          >>But at this rate for example South Carolina will have 2000 cases a day by July 1 in mid July they’ll have death rates in the hundreds per day.

          That doesn’t seem consistent.

          If IFR is 1%, hundreds of deaths (say 200-500) in mid-July would imply tens of thousands of infections (say 20,000-50,000) in late June.

          I don’t think testing *now* is so bad that we’d only measure 2000 cases per day if there are 20,000-50,000 infections per day. In April we were probably detecting less than 1/10 of infections, but now?

          Also, IFR for infections happening in late June is probably not 1%, since supportive care for COVID is understood better, and there is some evidence that the age profile of people being infected may be skewing younger.

          2000 cases/day seems plausible, but I really don’t expect a state the size and density of South Carolina to have 200+ deaths/day.

      • Mendel says:

        confused wrote: “I definitely think missed cancer screenings and such are a real and significant problem, and were an unnecessary harm in all but about 2-3 states.”

        People miss cancer screening because they are afraid to visit the doctor. They are afraid to visit the doctor because they think they might get infected there. Interventions that reduce the risk to get infected (that “empty” the hospitals) and efforts to communicate the actual risk (compared to the rrisk of not detecting a detectable tumor) help here. Combating the virus 2aggressively” should lead to more patients getting screened.

        Hospital capacity being reserved for Covid-19 is mostly ICU beds (up to 30% depending on location), and this does not affect cancer screening or radiation therapy, since these don’t require hospital beds.

        Reference 5 in the paper states, “This represents on average 18·1 LYG per patient, which markedly reduces to 17·2 with three months delay and to 15·9 with six months delay.” And that is across all cancer types; if surgeons manage operations by forecast outcome, the number is even lower. The German surgeon Markus Albertsmeier said in an interview: „Die deutschen Kapazitäten werden allgemein so eingeschätzt, dass keine dringende Krebsoperation hinausgezögert werden muss.“ The German capacities are estimated such that no urgent cancer operation needs to be postponed.
        Attacking the virus aggressively, bringing hospitalisation numbers down quickly, should shorten delays for cancer patients and save their lives.
        The reference also shows that not treating a Covid-19 patients leads to more loss of LYG than delaying a cancer operation.

        It seems prudent to not consider health effects one-dimensionally: just because there is a possible negative effect with the current strategy, that doesn’t mean that another strategy fares better. Comparing the current state of affairs to “no epidemic at all” is living in denial.

        • confused says:

          >>Combating the virus 2aggressively” should lead to more patients getting screened.

          Only if people’s fear of the virus is correlated with their actual risk from the virus. I think it’s often not — most people are either overly unafraid or overly afraid.

          What I was talking about is that stronger measures may convey an impression that the crisis is worse than weaker measures do – thus increasing fear despite reducing actual risk.

          >>Hospital capacity being reserved for Covid-19 is mostly ICU beds (up to 30% depending on location), and this does not affect cancer screening or radiation therapy, since these don’t require hospital beds.

          Now, yes. Two months ago, all ‘non-essential’ procedures (including things like this that are critical but less urgent) were not performed in much (most?) of the US. That’s what I was complaining about — not the measures in place at most hospitals now.

          The comparison IMO should be (risk of getting COVID x risk of dying assuming getting COVID) vs (increased risk of death from not getting screened).

          And the people who get the best benefit from cancer screening (if you have only a couple years of life expectancy left anyway, catching cancer early doesn’t help you much) don’t have *that* high of a risk of dying of COVID.

          • Joshua says:

            confused –

            > What I was talking about is that stronger measures may convey an impression that the crisis is worse than weaker measures do – thus increasing fear despite reducing actual risk.

            This is very speculative. It’s ok to speculate. But I think that you’re wrong. I think that there would likely have been a worse effect, and a longer effect, if there had been a higher amount of spread, more people dying, more overloaded medical facilities, less access to PPE, etc.

            Certainly, these effects would have been regionally variable – but we don’t know how people might have traveled if had really gotten totally out of hand in a place like NY.

            The point I’m making isn’t that I know, but that these issues need to be taken into consdieration when someone goes on to make counterfactual arguments about how much economic pain has been caused, differentially, by SIP orders as opposed the potential havoc that might have occurred without them. If you want to make an argument that the SIP orders cause harm differentially, then you need to lay out the argument for why things wouldn’t have been worse, economically, without them. You can’t just say that shutting things down caused harm and then say that therefore, the harm is differentially worse because of them.

            • confused says:

              Yes, it’s definitely uncertain.

              Here’s why I think the economic effects of a less controlled epidemic would have been less severe than they might at-first-glance seem:

              -mortality rate of COVID is much lower than that of 1918 H1N1 pandemic*, so large-scale social breakdown and economic collapses are probably unlikely, since they didn’t occur in the far worse 1918 pandemic.

              -“direct” losses to the economy from sickness will be limited by the fact that the vast majority of deaths are outside the workforce (due to the usual US retirement age), and a lockdown means people aren’t working for a couple of months anyway. A third or half of a company’s employees being off work sick for a month vs. 100% of them being off work for lockdown.

              And a third or half being off work sick for a month is probably a high end estimate.

              -“indirect” losses (due to people’s fear of activities) in my hypothetical will be limited by the fact that the effects would still be very regional. People don’t have a strong “gut feeling” for large numbers, e.g. 120,000 deaths vs. 300,000 deaths. If you don’t know someone personally, it’s not entirely real to you.

              And an uncontained epidemic might end rather quickly (due to herd immunity) and the economy could then return basically to normal.

              *1918 H1N1 killed something like 2.5% of the *entire world population*. ~50 million deaths out of ~1.8 billion. IFR probably something like 8-10% if 1/3 of world population was infected. COVID IFR is probably ~1% at least in places like New York, Spain, etc. Seems to be lower in some places due to what age-groups are infected.)

              I am not at all saying this would be ethically acceptable. But I think history shows (18th and 19th century yellow fever epidemics; 1918, 1957, 1968 flus) that epidemic and pandemic diseases (short of something like New World smallpox or the Black Death) don’t have nearly the economic, social, and political effects that would be expected given the number of deaths. People I know who were around in 1957 and 1968 don’t even remember that pandemics happened then.

              • Andrew says:

                Confused:

                I’ve been thinking about this too, and one thing that puzzles me is why more places didn’t just say, What the hell, we’ll just live our life like normal. It can’t all be blamed on Neil Ferguson or whatever. It does seem that people will put in some effort to avoid spreading the disease.

              • Joshua says:

                confused –

                > -“direct” losses to the economy from sickness will be limited by the fact that the vast majority of deaths are outside the workforce (due to the usual US retirement age),

                But what you’re describing is what took place during a lockdown where most people weren’t going to work. The loses would have been huge if infections were spreading oh necked I workplaces. People would have not wanted to go to work. Does an employer fire them? Then they can’t collect UE.

                > and a lockdown means people aren’t working for a couple of months anyway.

                Maybe the workplace would have been massively disrupted for a longer period with a higher rate of spread.

                Keep in mind, deaths is out the only relevant issue. Sickness is also important. And people getting infected in a highly infectious workplace then spread the virus to family members back home.

                > A third or half of a company’s employees being off work sick for a month vs. 100% of them being off work for lockdown.

                This is a purely hypothetical scenario that seems extremely unlikely to me. In the very least it is the best case scenario at one end of a spectrum of possibilities. It looks to me like you’re reverse engineering from the conclusion you want to draw.

                I think thar cross-country comparisons are fraught, but the economy in Sweden does not show a massive advantage to “locked down” economies in similar countries. And then you need to factor in the “costs” of more deaths. And if a vaccine is developed and widely distributed in a fairly short time frame, I’d say that there would be significantly greater economic cost Sweden, differentially – but the point is its too early to tell. There is a point at which perhaps the economic advantages of a faster spread equalize with those of a slower spread. Much could depend on where you are w/r/t that equilibrium point.

                > And a third or half being off work sick for a month is probably a high end estimate.

                ? If there is widespread sickness and death I’d say thst is a massive underestimate of what rhe work hours lost would look like.

                > And an uncontained epidemic might end rather quickly (due to herd immunity) and the economy could then return basically to normal.

                Purely hypothetical. It might take a long time. Again, awden might be instructive, and there is much I Sweden to make a no “lockdown” approach more viable than here

                > I am not at all saying this would be ethically acceptable.

                I’m not brining ethics up as an issue.

                I think that your are creating a false dividing like between the economic outcomes of the two conditions.

                > But I think history shows (18th and 19th century yellow fever epidemics; 1918, 1957, 1968 flus) that epidemic and pandemic diseases (short of something like New World smallpox or the Black Death) don’t have nearly the economic, social, and political effects that would be expected given the number of deaths. People I know who were around in 1957 and 1968 don’t even remember that pandemics happened then

                I think that trying to use history as a way to work through the variety of uncertainties on the different side so the line only add to the problem. Our society is vastly different than 100 years ago. Our economics are vastly different. Out age stratification is vastly different (we’re quite a bit older). It’s a useful exercise, but I think that a tone of work would have to be done before it really becomes instructive.

                Counterfactual arguments are really, really hard, and they require a very high quality of fats to be valid, IMO.

              • Joshua says:

                confused –

                Imagine trying to run a business with a large % of the the employees sick. Another large percentage wouldn’t want to come to work. Many might be key employees because they’d be older and more senior.

                But the overhead would likely not decrease in proportion to the loss of productivity. So l you’re running your business, and carrying payroll. Even though tons of people are sick or not coming in, and you’re not selling much, but your overhead is high. And you have nothing much to so with your inventory. And you probably can’t get supplies. And morale sucks because your employees don’t want to be there and because you’re not shut they can’t stay home with their kids and collect unemployment. And they’re pissed at you because they have to send their kids to school at the risk of them getting sick or infected (remember that back a bit there was far less verso th about kids getting infected), but giub won’t shit down so that they can stay home.

                Sounds like a nightmare to me. And I imagine that it would be nuts with a lot more people getting infected, and a lot more people getting sick. Differentially worse than shutting down? I don’t know. In Sweden a much larger % of people live in single-person households. They have a functional healthcare system. They have incredible benefits for people on leave. A very high % can work from home. Their population density is relatively concentrated in just one city. And they still took a big economic hit. It’s not clear yet what the economic impact will be relative to other countries.

                Maybe you’re right, but I think you’re taking a lot on faith about the relative economic benefits of not locking down. And that’s the problem I have with Ioannidis’ arguments as well. He doesn’t even make a reasonable attempt at due diligence to obvious potential counter-arguments.

                He just acts like because we know that there are costs associated with “locking down,” the differential economic costs would be greater from “locking down”, as if there there wouldn’t be costs to not “locking down.”. Why do we know that? Becsuee we can’t see them directly like we can see the costs of “locking down?” It’s so obviously, in my opinion, a false choice that’s being presented. And it’s disappointing to me that he’s engaged at such a shallow level in these issues.

                Just as other scientists should consult with statisticians in their research, perhaps so should Ioannidis consult with some economists. His understanding seems to completely lack consideration of basic considerations – akin to externalities or opportunity costs.

                Yes arguing about economic policy without really taking the issue seriously.

              • confused says:

                >>I’ve been thinking about this too, and one thing that puzzles me is why more places didn’t just say, What the hell, we’ll just live our life like normal. It can’t all be blamed on Neil Ferguson or whatever. It does seem that people will put in some effort to avoid spreading the disease.

                Yeah, I’ve been wondering about this too. Specifically, why are Americans reacting to this so differently than to the 1957 and 1968 pandemics? Those caused something like 120k and 100k US deaths, in a population more like 200M – so 1957 might be the equivalent per-capita of maybe 200,000 US deaths today.

                1968 is especially interesting as a comparison, because that was a very politically divided era as well.

                It’s also interesting that many tropical countries with a much higher ‘baseline risk’ from infectious disease took strong measures. Most Americans under 70 or so have never had to worry about infectious disease (except maybe AIDS, which is a very different thing). But in countries where malaria, yellow fever, dengue, etc. are prevalent, and where median age is much lower, COVID might not change the overall risk picture much.

                I think it has a lot to do with “immediacy” of modern media and a more connected world. In the 50s and 60s, places like Italy — or even New York City, if you lived in Texas or California or Colorado — were “a lot farther away”.

                So what is happening there now seems more relevant, and so there’s political pressure to adopt strategies that are (or seem to be) working in other countries.

                And I think back in March the world started to divide into “containment” strategies (East Asia and much of Western Europe) and “limited mitigation/this is no big deal” strategies (the US, Brazil, UK, Sweden). The Imperial College model and political pressure / fear of what was happening in Italy pushed the US and UK into changing their strategies. I think that movement of the US and UK created a much stronger impression of a ‘global consensus’.

              • Martha (Smith) says:

                confused said,
                “Here’s why I think the economic effects of a less controlled epidemic would have been less severe than they might at-first-glance seem:

                -mortality rate of COVID is much lower than that of 1918 H1N1 pandemic*, so large-scale social breakdown and economic collapses are probably unlikely, since they didn’t occur in the far worse 1918 pandemic.”

                This is comparing apples and oranges (or maybe even apples and zucchini?). In particular, the economy and the social factors in 1918 were very different than they are now. In 1918, a much larger portion of the population lived in rural areas or small towns. There was also not as much travel — and especially not the kind of long-distance traveling in a few hours that we have today. Also, health care was not as advanced then — treatment for the 1918 flu was pretty much limited to bed rest and chicken soup (and maybe prayer).

              • Martha (Smith) says:

                confused said,

                “Yeah, I’ve been wondering about this too. Specifically, why are Americans reacting to this so differently than to the 1957 and 1968 pandemics? Those caused something like 120k and 100k US deaths, in a population more like 200M – so 1957 might be the equivalent per-capita of maybe 200,000 US deaths today.
                .
                .
                .
                I think it has a lot to do with “immediacy” of modern media and a more connected world. In the 50s and 60s, places like Italy — or even New York City, if you lived in Texas or California or Colorado — were “a lot farther away”.

                So what is happening there now seems more relevant, and so there’s political pressure to adopt strategies that are (or seem to be) working in other countries.”

                Good point. I was alive in (and old enough to remember events from) 1957 and 1968 — but I don’t remember any talk of pandemics then.

              • Martha (Smith) says:

                However, I do remember talk of the polio epidemic in the late 1940’s, and especially in the early 1950’s — polio was really scary.

              • confused says:

                I feel like I am coming across as saying “we shouldn’t have locked down”. I’m not, I am rather undecided on that.

                But I think the economic/social effects of pandemics seem from history to be surprisingly small short of a Black Death/New World Smallpox type scenario.

                >>> -“direct” losses to the economy from sickness will be limited by the fact that the vast majority of deaths are outside the workforce (due to the usual US retirement age),

                >>But what you’re describing is what took place during a lockdown where most people weren’t going to work.

                No, the vast majority of deaths would still have been outside the workforce, because the age-effect on IFR is *so* strong. There’s a huge difference between being 60 and being 80 for example.

                >>Maybe the workplace would have been massively disrupted for a longer period with a higher rate of spread.

                I don’t see how. How long would it have taken for an uncontained epidemic to infect everyone it was going to infect?

                >>Keep in mind, deaths is out the only relevant issue. Sickness is also important. And people getting infected in a highly infectious workplace then spread the virus to family members back home.

                Sure, but I was speaking strictly of economics, not ethics/suffering.

                >> A third or half of a company’s employees being off work sick for a month vs. 100% of them being off work for lockdown.

                >This is a purely hypothetical scenario that seems extremely unlikely to me. In the very least it is the best case scenario at one end of a spectrum of possibilities.

                I don’t see how, because many infections are asymptomatic, and not all symptomatic infections are subjectively significant enough to act on. People come to work with colds, flu, etc. (which I hate, but it is an observed fact).

                A third to a half of a *working* population (therefore excluding the oldest and least healthy – “healthy worker effect”) being out sick for a month actually seems very pessimistic to me, given the asymptomatic proportion observed in prisons, homeless shelters, the USS Theodore Roosevelt, etc.

                >>I think thar cross-country comparisons are fraught, but the economy in Sweden does not show a massive advantage to “locked down” economies in similar countries.

                Isn’t that largely because Sweden is rather dependent on the rest of the world? I think if *no one* (except maybe China, given that I’m talking about decisions made in March) had locked down, that would be very different.

                In 6 to 12 months we should know more. Maybe Andrew could do a ‘looking back on models and projections’ post then?

              • Carlos Ungil says:

                > why are Americans reacting to this so differently than to the 1957 and 1968 pandemics? Those caused something like 120k and 100k US deaths, in a population more like 200M – so 1957 might be the equivalent per-capita of maybe 200,000 US deaths today.

                I guess the reactions in 1957 and 1968 would also have been different if there had been 15’000 deaths in one month in NYC.

                > But in countries where malaria, yellow fever, dengue, etc. are prevalent, and where median age is much lower, COVID might not change the overall risk picture much.

                Brazil has around one million cases of dengue per year and less than one thousand deaths. In a few weeks it has reported already one million of cases of covid-19 and 50’000 deaths.

              • Zhou Fang says:

                I think comparing to 1957/1968 is an apples and oranges comparison. Firstly, the mortality in those pandemics are calculated based on subsequent analysis, it was not based on a direct count *during* the pandemic. Hence the figures cannot be directly compared, it could be the case that a subsequent analysis years after (especially including second waves!) would put the covid19 death toll as far higher. Secondly the 1957/1968 death toll took place in the absence of lockdown and social distancing, whereas we are seeing similar death tolls to those *despite* some lockdown and social distancing etc.

                Even the argument that Covid19 is less lethal than 1918 Spanish flu is risky – consider that we define cases of Covid19 based on genetic or antibody testing. This is much more sensitive than the diagnosis methods available in 1918, with the result that the usual virulence estimates for 1918 can be overestimates due to many cases of mild infection being missed.

                I think it’s very plausible that if handled the same way as 1918 or 1957-1968, we could see from covid19 levels of death equal or even exceeding that of 1918 spanish flu.

              • confused says:

                >>I think it’s very plausible that if handled the same way as 1918 or 1957-1968, we could see from covid19 levels of death equal or even exceeding that of 1918 spanish flu.

                I don’t think so. Missed mild cases lowering the IFR don’t make that much of a difference here; the percentage of the *entire world population* dying of 1918 flu was significantly larger than any of the reasonable IFR estimates for COVID.

                The CDC estimates 50 million deaths; the world population then was 1.8 billion – so ~2.8% of everyone on Earth.

                So the IFR of 1918 flu, no matter how many mild cases were missed, cannot be less than that, since more than 100% of the world cannot be infected.

                COVID’s IFR is debatable, but clearly not >2%. The higher plausible estimates are a bit over 1%.

              • confused says:

                And yes, comparing numbers directly to 57/68 is somewhat apples and oranges. (Most importantly because this pandemic is not over… reporting is improving drastically as this develops. While there have clearly been missed deaths from looking at CDC excess death data, from that same data I don’t think there is room for the numbers to double or anything close to that. I don’t think it will change the qualitative picture of “much worse than 2009 pandemic or seasonal flu, much less severe than 1918 pandemic” – that’s why I chose 57/68 as comparisons.

                Sure, this is with some degree of social distancing and shelter-in-place orders. But I am not convinced that the way the US did shelter-in-place orders is going to have more than a marginal effect on total deaths, since those orders are no longer in effect most places. If we get a vaccine distributed to the public by November (as in 2009), maybe, but that was working from a known flu vaccine.

              • Zhou Fang says:

                confused:
                > The CDC estimates 50 million deaths; the world population then was 1.8 billion – so ~2.8% of everyone on Earth.

                But how big is the uncertainty around that estimate?

                This most recent study estimates the 1918 death toll to be 15 million (+2 million the next year), implying IFR is only >0.7%.

                https://academic.oup.com/aje/article/187/12/2561/5092383

                This is based off of a multilevel regression model that I haven’t looked into the details of. The uncertainties in that, plus the fact that we still don’t have a clear idea what covid IFR looks like in the third world, leads me to argue we shouldn’t directly compare.

                Instead, more useful is to look at 1918 US statistics, where we estimate 500k-800k dead out of 29.4 million infected. Assuming undercounting of deaths is less likely than undercounting of cases, this bounds IFR in the 1918 US above, so IFR is 0.4-0.8%.

                Put all this together with our current uncertainty on covid19 and I don’t think we can rule out these intervals overlapping.

              • Zhou Fang says:

                (reposting because sigh, html)

                confused:
                > The CDC estimates 50 million deaths; the world population then was 1.8 billion – so ~2.8% of everyone on Earth.

                But how big is the uncertainty around that estimate?

                This most recent study estimates the 1918 death toll to be 15 million (+2 million the next year), implying IFR is only bounded below by 0.7%.

                https://academic.oup.com/aje/article/187/12/2561/5092383

                This is based off of a multilevel regression model that I haven’t looked into the details of. The uncertainties in that, plus the fact that we still don’t have a clear idea what covid IFR looks like in the third world, leads me to argue we shouldn’t directly compare.

                Instead, more useful is to look at 1918 US statistics, where we estimate 500k-800k dead out of 29.4 million infected. Assuming undercounting of deaths is less likely than undercounting of cases, this bounds IFR in the 1918 US above, so IFR is less than 1.7 to 2.7%. Bring in the population total of 105 million and spanish flu IFR is greater than 0.4-0.8%.

                Put all this together with our current uncertainty on covid19 and I don’t think we can rule out these intervals overlapping.

              • confused says:

                Yeah, the 50 million is not set in stone. I’ve seen estimates of ~18 million, but also of 100 million.

                But even 18 million is 1% of the total world population then – *not* a 1% IFR! Probably would be more like 3%, flu pandemics don’t tend to infect the majority of people IIRC. COVID definitely doesn’t have 3% IFR.

                Yeah, COVID IFR could be higher in countries not hard hit yet — but I am very skeptical of this, as nations with poorer medical care tend to have much younger demographics. Though the joker in the deck there is HIV/AIDS prevalence in sub-Saharan Africa.)

                If anything, I think the end-of-pandemic overall IFR will be significantly lower than 1%, since treatments will improve in the wealthier countries at least (already are improving), median ages of those infected seem to be dropping in the US at least*, and demographics are far younger elsewhere.

                *Early hot-spots (Lombardy, Madrid, NYC) seem to have infected the elderly fairly ‘efficiently’.

              • confused says:

                I read that article with the 15-17 million more closely, and I don’t find it terribly convincing. They are working from Europe and India data… but they are more or less making an argument that Europe (with lower death rates) is a better model for the parts of the world we don’t have data on than India (with higher death rates).

                This seems pretty implausible to me, as in 1918 the effects (economic, demographic, etc.) of the Industrial Revolution had not spread very far beyond Europe and the US.

                I think they are right that 100 million is far too high and that 50 million is likely too high, but 15 million?

      • Clyde Schechter says:

        “I definitely think missed cancer screenings and such are a real and significant problem”

        I strongly doubt it. I’ll speak first of breast cancer screening, since it’s the area I work in and know best. Numerous analyses going back decades have suggested that the lead time gained from mammography screening is around 2 years on average, and longer in older women who are the age group that accounts for the bulk of breast cancers. With newer technologies such as digital mammography and digital breast tomosynthesis, the lead time is, if anything, longer still (though probably not by much). The shutdowns have lasted only a few months. Only a small fraction of women who ordinarily have mammograms every 18-30 months will actually transition an occult, but radiologically detectable, breast cancer from treatable with good outcome to lethal during that short interval. Perhaps the greatest danger will be the small group of women who carry very high risk due to BRCA-1 mutations: for them the lead time is much shorter. Evidence that delay of a few months is probably irrelevant can be seen by comparing breast cancer mortality in the US and the UK. In the UK, regular mammography screening is scheduled at 3 year intervals, and their breast cancer outcomes are very similar to ours.

        What may become a problem is diminished capacity for screening as radiology practices reopen. Because the epidemic remains active, and is still in its exponential growth phase in many parts of the country, it is likely that breast imaging centers will have to reduce the number of mammograms that can be performed due to decreased capacity in limited physical space and perhaps equipment downtime needed to disinfect rooms and equipment between patients. A sustained and substantial reduction in capacity could lead to much longer intervals between screenings that do result in non-trivial increases in breast cancer mortality later on. So, the fact that we have not suppressed the virus, rather than the temporary interruptions due to shutdown, may well ultimately lead to an uptick in breast cancer deaths. Time will tell.

        While I am less familiar with the details of colon and prostate cancer screening, if anything these should be less problematic than breast cancer because the lead times provided by screening are much longer, typically a decade or more.

        I am surprised to see people speak of our response to the epidemic as an over-reaction on a site like this where people are pretty well-informed. The US death toll stands today at over 120,000. While the epidemic has crested in the northeast and Midwest, it is now growing exponentially in some parts of the country that previously saw little activity, and it’s anybody’s guess where it will end. Health care people in Tucson and Houston are saying that the hospitals and ICUs are now nearly full and if things keep going, in a few days they will be overwhelmed. What would it take for you to believe that we did not overreact?

        In my view, none of this had to happen. But the only countries that have had good results were ones that acted very early and took the epidemic seriously from the day the first case was identified in their territory. We frittered away precious weeks doing nothing, and then bungled our early efforts at responding. Now, we have simply given up on containing it. Pretty much all policy measures against the epidemic are being dismantled across the country, even in places where it is still on the upswing. People are socializing in bars and restaurants, an mass gatherings are ongoing every day. We will be very lucky to come out the other side of this with only 200,000 deaths.

        • Martha (Smith) says:

          Thank you very much. I hope others with knowledge of specific areas of medicine can give input as well; you effort sets a good example for them.

        • confused says:

          Good points about the screening. I am likely wrong about that.

          >>I am surprised to see people speak of our response to the epidemic as an over-reaction on a site like this where people are pretty well-informed.

          Well, my prior on “overreaction” is past US pandemics. We did very little for the 1957 or 1968 pandemics. Our reaction now is possibly stronger than our reaction to the 1918 pandemic.

          It’s not clear to me why COVID requires an order of magnitude stronger response than 1957/1968. (Those pandemics, per-capita, would be equivalent to a bit under 200,000 deaths.)

          >> What would it take for you to believe that we did not overreact?

          Well, basically, evidence from other countries that our reactions made a dramatic difference to the progress of the disease in the US. (Action taken at the stage we actually did act – back in January would have been very different!)

          It’s impossible for me to believe that every nation on earth is handling this very well. So if the original projections of what an unmitigated pandemic or one with limited mitigation would do were correct, then death rates in the range projected would happen *somewhere*.