Reverse-engineering priors in coronavirus discourse

Last week we discussed the Santa Clara county study, in which 1.5% of the people tested positive for coronavirus.

The authors of the study performed some statistical adjustments and summarized with a range of 2.5% to 4.2% for infection rates in the county as a whole, leading to an estimated infection fatality rate of 0.12% to 0.20%, a strong conclusion because it might be taken to imply that coronavirus is not much more deadly than the flu. As discussed on this blog and elsewhere, there were some problems with the statistical analysis, and these conclusions were not supported by these data alone.

Here’s what I said to reporter Michael Schulson:

It’s not like I’m saying they’re wrong, and someone else is saying they’re right. . . . I think their conclusions were a bit strong . . . they have reasons for believing what they believe beyond what’s in this paper. Like they have their own scientific understanding of the world, and so basically they’re coming into it saying, ‘Hey, we believe this. We think that this disease, this virus has a very low infection fatality rate,’ and then they gather data and the data are consistent with that. If you have reason to believe that story, these data support it. If you don’t have such a good reason to believe that story, you can see that the data are kind of ambiguous.

A commenter asked how I could have said such a thing, as it seemed in contradiction to my earlier statement that the authors of that paper had “screwed up” on the statistics.

But it’s not a contradiction at all.

Let me explain. The data presented in that Santa Clara study were consistent with underlying infection rates in the target population of anywhere between 0% and about 5%. The 0% would correspond to a 1.5% false positive rate of the tests (which is consistent with the data presented in that paper), and the 5% would correspond to a 0% false positive rate plus some random luck plus some adjustments. I can’t be sure of the 5% because I don’t have all the detail on the adjustments; also the study has other data such as reported symptoms that could be informative here, but those data have not been released, even in summary form, so I can’t really do anything with that.

A range of 0% to 5% . . . OK, we know it’s not 0%, as some people in the county had already had the disease. So I stand by my statement that the study did not offer strong support that the rate was between 2.5% and 4.2%; on the other hand, the data in the study were consistent with those claims.

As I wrote in the comment thread:

It seems clear to me that the authors of that study had reasons for believing their claims, even before the data came in. They viewed their study as confirmation of their existing beliefs. They had good reasons, from their perspective. Their reasons are based on their larger understanding of what’s happening with the coronavirus. They have priors, and what I’m saying is that the data from their recent surveys is consistent with their priors. I think that’s why they came on so strong.

But, as I said, if you don’t have such a good reason to believe that story, you can see that the data are kind of ambiguous. I don’t know enough about the epidemic to have strong priors. So, to me, these surveys are consistent with the authors’s priors, but they’re also consistent with other priors.

It’s a Bayesian thing. Part of Bayesian reasoning is to think like a Bayesian; another part is to assess other people’s conclusions as if they are Bayesians and use this to deduce their priors. I’m not saying that other researchers are Bayesian—indeed I’m not always so Bayesian myself—rather, I’m arguing that looking at inferences from this implicit Bayesian perspective can be helpful, in the same way that economists can look at people’s decisions and deduce their implicit utilities. It’s a Neumann thing: again, you won’t learn people’s “true priors” any more than you’ll learn their “true utilities”—or, for that matter, any more than a test will reveal students’ “true abilities”—but it’s a baseline.

One problem here is that people are expecting too much from one study. Ultimately, we will learn by replication.

There was a replication of sorts in Los Angeles county, where a team including some of the same researchers reported an estimate of 2.8% to 5.6% with antibodies to the virus. There was also a study in Miami reporting 6%.

On the other hand, if there are a lot of false positives, then we’d expect these to be overestimates. We can’t really know right now. For the Los Angeles and Miami studies, all we have are press releases. This doesn’t mean the results are wrong; it’s just hard to know.

There was a study reporting 20% with antibodies in New York City. Nobody thinks that is all or even mostly false positives; obviously a lot of people in the city have been exposed to the virus. There are sampling issues so maybe underlying rate was really only 10% or 15% at the time the study was done . . . we can’t really know!

We need better data, followups, open science, coordination, etc. I’m glad that people are gathering data and doing studies, and I’m glad that people are pointing out mistakes in studies, which can allow us all to do better.

The discussion can be frustrating because it becomes politicized and people take sides; see discussion here, for example. I regret saying that the authors of the Santa Clara study “owe us all an apology.” I stand by my reactions to that paper, but I don’t think that politicization is good. In an ideal world, I would not have made that inflammatory statement and the authors would’ve updated their paper to include more information and recognize the ambiguity of their results.

P.S. One reason I’d like to shift the discussion from “bias” or “motivated reasoning” to “priors” is that taking about priors is less inflammatory than talking about bias or motivated reasoning. Bad priors give bias, and motivated reasoning can lead to bad priors, but if researchers can identify their priors and their assumptions more specifically, I think this would help, if for no other reason than that some direct reflection can help clarify internal inconsistencies.

Asking people to specify their priors in a statistical analysis is comparable to the thing that people sometimes say in debate: “What evidence would it take you to change your mind on this issue?”

P.P.S. Catherine Offord summarizes some of the measurement issues in this news article.

131 thoughts on “Reverse-engineering priors in coronavirus discourse

  1. “There was a study reporting 20% with antibodies in New York City.”

    Speaking of that–has anybody seen the actual study? I’ve looked for it online with no success. (But searching is not my strong suit.) Anybody have a link to it?

  2. Rather than all these studies, I would like to see a survey of HCWs to see if the flu, even in a bad season, produces anything like they are seeing now, and if what they see in patients reminds them of the flu (also look at the studies of the huge spike in deaths in the last couple of months, way above the reported numbers in most places). Also in the survey, ask the HCWs what they think of any study that says COVID-19 is just like the flu.

    Sometimes real-life comes at you fast.

    • Such naive arguing can lead to incorrect conclusions.
      You are underestimating how deadly the seasonal flu is, and you are missing what the argument is about.

      Two third of the people in the 65+ age group in the US have the annual flu shot.
      Similar in other risk groups.
      Pretty much every HCW and everyone working with retired people has the annual flu shot.
      This has the risk groups mostly covered, and with a third of the US population covered you are already closer to herd immunity.
      Despite this, the average annual flu season kills as many people in the US as COVID-19 has so far.
      Even the actual normal flu mortality of 0.1% would be enough to kill a quarter million people in the US.

      COVID-19 hits a population without partial immunity from past infections, and without a widely used vaccine.
      The disputed question is whether the difference in deaths is due to higher mortality among infected people, or due to a higher proportion of the population being infected (or both).

      For HCWs seasonal flu infecting everyone in New York within a month with the average mortality of 0.1% would not look different from COVID-19 in April 2020 in New York – same number of deaths within a month.

      • Unfortunately, the efficacy of influenza vaccination on average (varying from year to year) is just over 50% so herd immunity is not possible even with 100% population coverage. Also, a large animal reservoir means it can never be eradicated (like, say, smallpox).

      • “Despite this, the average annual flu season kills as many people in the US as COVID-19 has so far.”

        It’s hard to keep up with these things, as people are dying so rapidly from covid-19. About 57K now.

        As of now, more people have died of covid-19 than died of the flu in any year this past decade other than 2017-2018 (61,000). We’ll pass that by the end of the week.

        For this past decade, the average annual deaths from the flu is about 37,000.

        We passed the “as many people die from the flu” benchmark some time back …

        • “It’s hard to keep up with these things, as people are dying so rapidly from covid-19. About 57K now.”

          So hard to keep up with, that this number is already 2K too low. The last two days were looking optimistic with deaths below 1500. As of right now 17:30 EDT, we’re at 2300 deaths today (based on the data here: https://coronavirus.1point3acres.com/en). The back half of the curve is wicked slow coming down in Western countries (compared to China). We’ll probably average above 1K deaths for the next three weeks. I could see the USA easily bouncing around 2K deaths a day for all of May.

      • Thank you Adrian. A friend explained this to me very early on, and it had a profound influence on moving me from “well, this is going to be big and a lot of people die, but it’s not too different from a bad flu season” to “oh boy, we had better do everything we can to slow the spread or this is going to be a disaster.”

        IFR is the fraction of people who die, conditional on becoming infected. If something had IFR = 0.7 but only 1% of people were susceptible to infection, that wouldn’t be a big deal on a societal level: with that infection rate it could never spread widely, could certainly never create an epidemic. But flip these around, an IFR of 0.01 but 70% of people get it, well, you have a pandemic and you are going to kill an awful lot of people.

        Treating IFR as if it summarizes the severity of the problem is a big mistake.

        • Keep in mind most of the deaths are actually due to being inappropriately put on ventilators because extrapolating from RCTs failed.

        • I don’t think you can say “most”. A bunch of the deaths in NYC yes, but there are still plenty of people who are dying at home, dying in nursing homes, in NYC people were dying in hospitals before they even got them out of the hallway.

          yes, the ventilator thing was bad, but just the sheer speed with which it can attack large numbers is going to cause a disaster if it spreads at typical rates, even if there are no vents in use.

        • I can’t find stats on ICU admissions/deaths, but 25% of the deaths reported in NYC did not even happen in hospitals.

          In France 37% of the deaths reported happened outside hospitals. In Belgium it’s over 55%.

          In Spain the number of ICU admissions is 44% of the number of reported deaths.

          Many people die even when there are no misguided doctors actively killing them.

          (Of course, it’s not logically impossible that if everyone had had just the right amount of the right treatment at the right time no one would have died. Maybe it’s not even one tenth as bad as the flu!)

        • I asked about this like a month ago and I genuinely do not understand, why do you post these numbers without a source?

        • Do you happen to know of any source that backs your claim that most of the deaths are actually due to being inappropriately put on ventilators?

        • Unless they say how many thousand patients died due to being inappropriately put on ventilators and that numbers is over half the total number of people who died, I don’t see how that can be a source for “most of the deaths are actually due to being inappropriately put on ventilators”.

          I’m not even convinced that most people who died were at some point in intensive care. At least is some countries they clearly didn’t, but I have not found info about the US in particular.

        • Why do you keep posting numbers without including the source? Please include the source of your numbers in the future.

        • Right on Phil. It’s a bit surprising to me that this is not in the public consciousness as much as it seems it should. There are many ways this is not like the flu, one of them being its higher rate of contagion, which is EXACTLY why this has turned into a pandemic. In the end we’ll find out this is so deadly not because it kills at a high rate, but instead because it infects so effectively. One thing is for sure: we’re not going to find out it’s because the ventilators killed everyone, or whatever other kook conspiracy people are finding the need to float because they just don’t want to believe whatever truths we are finding out along the way.

        • There are many ways this is not like the flu, one of them being its higher rate of contagion, which is EXACTLY why this has turned into a pandemic.

          It is very contagious for like 3 days. You heard it here first.

        • Right on Phil. It’s a bit surprising to me that this is not more ingrained in our public consciousness by now. One of the many ways this is not “like the flu” is that its infection rate is so much higher. Which is of course EXACTLY why it has become a pandemic. And exactly the reason why we’re going to have to continue to make a concerted effort to protect those that have co-morbidities. A lot of people are understandably having trouble coping with the truths that we are facing, so they find comfort in theories that don’t exactly match reality. Of course while finding and embracing places that have folks sharing opinions along the lines of “ventilators kill people, COVID-19 doesn’t” can be tempting when the truth is too much, I wish our public health experts would shout from the rooftops that this one is less about fatality rate than it is about contagion rate.

  3. Andrew –

    > leading to an estimated infection fatality rate of 0.12% to 0.20%

    That is the part of all of this that I’m still scratching my head about.

    Regardless of the statistical issues with their estimation of an infection rate, it’s their extrapolation from that rate to a broadly applied fatality rate (based on a non-random and non-representative sampling) which seems to me to be the much more significant problem.

    All that much more as information is coming out with respect to uncertainty about trends in all cause mortality, and difficulties in identifying who has and hasn’t died “of” COVID-19.

    And the statistical issues with their estimation of infection rate seems to me more a technical artifact as compared to the political and societal implications of them going on a national publiciry campaign to advocate for policies to address a public health risk in the order of the seasonal flu.

    But as near as I can tell, you don’t even seem to think that extrapolation is a particular problem at all, let alone a problem that’s more concerning in impact than their statististical methodology.

    I remain confused.

    • If you know how many people died, and you know how many people were infected, then you can calculate that ratio.

      The problem with these studies is that they don’t quantify their uncertainty even though there’s a lot of it. Suppose d percent of the country has died, and q percent of the country has had the virus, then d/q is the infection fatality rate (so far, there are still sick people who will die in the next weeks).

      but suppose d is about 100k people out of 330M = .1/330 = .000303 (this assumes undercounting of about 2x) and we’re trying to estimate q, and q can realistically be anything between .003 and .05, then the infection fatality rate could by anything from .000303/.003 = .101 to .000303/.05 = .00606

      anywhere from 10% to 0.6%

      That’s a HUGE range. Anyone who comes out and says “the infection fatality rate is around 0.2%” is not only ignoring a huge range of uncertainty, they’re claiming a number that isn’t even rationally in the high probability density region.

      • > The problem with these studies is that they don’t quantify their uncertainty even though there’s a lot of it.

        Yup. Even if we don’t know the numbers, at least we could have a range of uncertainty reported for the numbers that have been reported – based on epidemiological science that models uncertainty from reports of deaths in past, at least somewhat analogous, situations.

        Look at this headline from today.

        > Coronavirus deaths more than twice hospital toll, data indicate

        https://www.ft.com/content/0ed8ea34-ebc5-4425-b86a-7a29447de57b

        But again, how do you go from a non-random and non-representative sample to even get the most basic requirement to justify a broad extrapolation to a population that’s orders of magnitude bigger and more varied?

        Santa Clara, as an example, has 4x the national median income and isn’t nationally representative for race/ethnicity. SES has a huge influence on health outcomes!

        What am I not getting here?

        • Andrew, Sure, it’s useful to ask the question “what would I have to believe in order to come up with the same result that these people did?” which is another way of saying we should “reverse engineer” their priors.

          things that would get you there:

          1) a spike-like prior for the false positive rate down in the less than 1% range. This is un-justified from prior evidence from this type of test.

          2) a spike-like prior on the IFR, causing you to throw away the possibility that you have low infections and are mostly seeing false positives, or motivated test-seekers (non-random sampling), and instead focus on the theory that it’s pretty good representative sampling and if anything you’re more likely to be under-counting (and hence adjust things upwards as they did).

          I don’t think anyone should accept those as the basis for a starting point on any argument.

          A rational starting point for the country as a whole is that less than 5% of the country has had this disease is probably likely: beta(3,97) perhaps for the infection rate. If we want to make that more diffuse we could do say beta(2,30) which has high probability region all the way up into 20% range… It seems hard to imagine 20% uniformly around the country, so certainly this contains all the reasonable numbers.

          and for the false positive rate, we know it should be a few percent… here we really can’t use a diffuse prior because we know that this kind of test should have a couple of percent false positive, but not exactly how many… so maybe beta(2,90)

          As for the representativeness of the sampling, it certainly seems like the infection risk rate could be anything from 1/2 to 2x the population average (ie. people either sought the test because they thought they’d had it, or maybe people with a lot of money have been staying away from others, haven’t got it, but are curious and have connections that would let them find out about the test more easily than others.

          So, put all together? what do we get? basically not much.

        • Daniel –

          > Sure, it’s useful to ask the question “what would I have to believe in order to come up with the same result that these people did?” which is another way of saying we should “reverse engineer” their priors.

          I just don’t get why people are focusing almost exclusively on the infection rate estimations. Ioannidis and his co-authors said that their estimates of infection rate in Santa Clara justified a national-level extrapolation for fatality rate. Seems to me, that’s the main issue.

          Median income in Santa Clara is *much* higher than the national median.

          What kind of a prior would explain how someone can go from non-representative sampling from Santa Clara, without controlling for income levels, to extrapolate broadly about national fatality rates?

          In the US, there is a well-established association between income-level and health outcomes.

          Perhaps one prior is that income doesn’t actually explain the causality, but there are confounds which do… OK. So then you have to then be explicit about that reasoning and make the case as to why those other factors actually explain the association.

          We might even drop down to – perhaps one prior is that actual income doesn’t actually explain the causality in the association between income and health outcomes but we just don’t know what the actual explanatory variable is. Well, OK, but does that mean that you can just go ahead and ignore the association and extrapolate anyway?

        • You’re absolutely right. What if you had done a 10,000 person survey and got an infection rate of 3.1% +- 0.2% or something… you couldn’t extrapolate that to the country as a whole without some significant assumptions. But, one such assumption would be something like “uncontrolled spread doesn’t distinguish between SES” and that’s not so implausible. This thing seems easy to spread.

          If I were going to extrapolate to the whole country, I’d be looking at a multiplier which itself had uncertainty. so whatever the rate is in Santa Clara, the countrywide rate is probably r*k where k is itself uncertain and maybe something like lognormal with a scale of log(2)/2 or something like that (ie. most likely within about a factor of 2).

          With multiple results from different places, we could zero in.

        • Daniel –

          > But, one such assumption would be something like “uncontrolled spread doesn’t distinguish between SES” and that’s not so implausible.

          Maybe. My priors would be that lower income folks are more likely to be on public transportation, live in higher density communities, be more likely to be essential workers and thus not isolating, etc. But that’s a situation where there isn’t a well-established association that should be addressed.

          With fatality rates and health outcomes generally, that’s not the case. An association is well-established in the literature.

          And I could be wrong, but there seems to be some solid evidence of an association with COVID fatality, in particular, with SES. That association, it seems to me, is far less speculative than there might be for the association between SES an infection rate.

          That’s why I keep going back to the leap they made between infection rate and fatality rate.

          You have to have some pretty strong priors, it seems to me, to justify that leap because it means basically dismissing known, strong associations. OK. So maybe that’s just priors and not bias. But WTF are those priors?

        • Sorry, that should read:

          With SOCIO-ECONOMIC STATUS and health outcomes generally, that’s not the case. An association is well-established in the literature.

          (not “With infection rates and health outcomes generally,…”)

        • Hello Joshuo

          Re: I just don’t get why people are focusing almost exclusively on the infection rate estimations. Ioannidis and his co-authors said that their estimates of infection rate in Santa Clara justified a national-level extrapolation for fatality rate. Seems to me, that’s the main issue.’
          ——

          Where in the article do the co-authors claim an extrapolation for a national-level estimate?

        • Maybe the weighing by zip code and race and gender somewhat makes up for income disparity?
          If you want to do a quick and simple study, it’s good to not have to ask people for personal data you don’t want to store and can’t verify.

          We’re focusing on the IFR because that’s the only measurement that generalizes. Case undercount and absolzte prevalence are specific to Santa Clara, and because of Santa Clara’s history, likely to be higher than elsewhere. Because of the importance of age for the fatality rate, it’s unfortunate that the analysis does not include it as factor.

          Ignoring age is worse than ignoring income.

        • @Sameera Daniels
          “Nevertheless, our prevalence estimates can be used to update existing fatality rates given the large upwards revision of under-ascertainment.”
          I believe they’re implicitly talking about fatality rates in other regions here.

        • Hi Sameera –

          > Where in the article do the co-authors claim an extrapolation for a national-level estimate?

          Good point.

          I have addressed that elsewhere, but I should be careful to do so each time I harp on this.

          AFAIK, it isn’t in the article itself; it’s in the subsequent publicity campaign. Sometimes it was a bit caveated with references to uncertainty, but often not.

          Ioannidis, in particular, said that he Santa Clara study justified a broad extrapolation for the disease. He didn’t specify “nationally,” or “globally,” but it was a broad and inclusive and quite certain statement of a uniform a fatality rate – as one hears when they hear that the “seasonal fatality rate is X%.”

        • Mendel –

          > Maybe the weighing by zip code and race and gender somewhat makes up for income disparity?

          Zip codes in Santa Clara as compared to nationally? (Presumably the LA study wouldn’t be as problematic in that respect). And I only focused on income. There are other important variables. Consider race/ethnicity. Santa Clara is hardly representative (3% African American, if I recall correctly). And how about issues like health care insurance or access to healthcare or availability of hospital beds or amount of PPE.

          Calculate fatality rates in Santa Clara and project broadly on that basis without controlling for the relevant variables? WTF?

          > We’re focusing on the IFR because that’s the only measurement that generalizes. Case undercount and absolzte prevalence are specific to Santa Clara, and because of Santa Clara’s history, likely to be higher than elsewhere. Because of the importance of age for the fatality rate, it’s unfortunate that the analysis does not include it as factor.

          Ignoring age is worse than ignoring income.

          I thought they controlled for age? If not, WTF x 2. That can’t be right that they ignored for age representativeness, can it?

        • Joshua,
          Simply taking a ‘well-established in literature’ influence of SES would be hard to model meaningfully.

          What really counts is how much interaction and mixing between high and low SES citizens has been there in Santa Clara since last November.

          e.g. A high earner in SC county traveling to China every month to monitor Apple sweat shops can be a virus-carrier, but he/she most likely doesn’t shop at Walmart and take public transportation to work, to infect low SES citizens.

          Or how about a low SES member being more likely to be unemployed, thus going to a fewer places and less likely to get infected, etc. etc.

          All the models are simply assuming that 330M Americans are on average going to the same places, exposed to the same extent, touching the same average number of things in their daily life, and so on.

          I mean, you can assume all those things at the population level, but we all know it’s way too simple to be true.

        • Navigator –

          > Simply taking a ‘well-established in literature’ influence of SES would be hard to model meaningfully.

          What really counts is how much interaction and mixing between high and low SES citizens has been there in Santa Clara since last November.

          e.g. A high earner in SC county traveling to China every month to monitor Apple sweat shops can be a virus-carrier, but he/she most likely doesn’t shop at Walmart and take public transportation to work, to infect low SES citizens.

          Or how about a low SES member being more likely to be unemployed, thus going to a fewer places and less likely to get infected, etc. etc.

          +++++++

          Those are excellent points. Yes, I’m guilty of priors that might lead me to be simplistic about the influence of SES, and would likely have overlooked those considerations had you not pointed them out.

          But at the same time – we do know that when you look at averages there is an overall signal of an association between health outcomes and SES. And overall, the association is one of poorer outcomes among with those of lower SES. Does that mean causality? Is it, in fact, the lower income which causes the poorer health outcomes? I don’t know for sure. I can think of mechanisms for causality – less access to heatlthcare being primary among them, but we could offer conjecture for why the income level is a confound and what’s really explanatory are behaviors which aren’t really attributable to income level per se. Obviously, these are big issues.

          But I go back to the simple fact that an association exists. Seems to me that if you’re just going to ignore that association, which these folks are doing when the extrapolate from a population with for times the national mean income to infer mortality rates to a population with, well, the mean national income, it’s incumbent on them to explain why they’re ignoring that association. What is the causal mechanism that they propose to explain why there is, in fact, not a causality to the SES —> health outcomes association Or, they could simply adjust their weighting to account for SES.

          I’ll leave it to the smart folks to conduct the sensitivity analyses to explore and adjust for the explanatory power of different associations, and I understand that we should take even the most carefully adjusted analysis with a grain of salt because we just can’t actually know what the causality boils down to.

          But I think that when there are well-established associations there is some responsibility.

          Let’s play it in another direction. Suppose they went into a really large nursing home and determined an infection rate in that population, and counted deaths in that population, and then said that we should infer a broader fatality rate from that calculation. Would that be acceptable? Not trying to go all ad absurdum here – but my point is that a basic principle in play is that before extrapolating you should do the best you can to make your sampling as representative as possible – or adjust your weighting after the fact as best you can.

          > All the models are simply assuming that 330M Americans are on average going to the same places, exposed to the same extent, touching the same average number of things in their daily life, and so on.

          I don’t think that’s exactly true. I think that some of the models adjust for many of those kinds of parameters, while others fit the curves to the statistics from other communities – but even they, I would imagine adjust for certain identified cross-cultural factors. (I would happy to be corrected if I’m wrong).

          > I mean, you can assume all those things at the population level, but we all know it’s way too simple to be true.

          Sure, it’s all too simple to be “true.” All models are wrong, and some are useful. But let’s not let the knowledge that all models are wrong excuse violating certain fundamental principles.

          Again, I go back to my basic understanding that you just can’t extrapolate from unrepresentative sampling, at least unless you’ve done the best you can to weight your sample accordingly.

          Andrew said he thinks you can extrapolate from unrepresentative sampling and I’m inclined to defer to his expertise in principle – but I still don’t understand why I’m wrong about this…

        • @Joshua
          The early epidemic isn’t the time to extrapolate to demographics anyway, as the distribution of infections will be uneven and determined by where the outbreaks randomly were.
          There hasn’t been enough entropy yet.

          But age in Santa Clara:
          Age – US Census – Stanford Study raw/adjusted
          0-4 … 5.9% … 2.1%/2.6%
          5-18 … 21.9% … 16.5%/14.5%
          >65 … 13.5% … 5.0%/4.5%

          If they had 50 positives, they had around 3 positives in the 65+ age group maybe? and half of these false positives. It’s like “it’s going to be one of these 3 grandmas, let’s deduce mortality data from that”, there’s a whole other margin of error opening up when you get into this that the study didn’t even talk about.

          Considering this, it was a wise choice not to weight by age, but then they ought to not have talked about the mortality in the study at all.

          If you assume that the prevalence in that age group equals the prevalence in the population — and that hasn’t been true in Germany in March! –, then you can maybe make some inferences from the overall prevalence they computed, but since we think the error margins on that weren’t right either, well…

          So forget doing any demographics anyplace where the true prevalence doesn’t at least have double digits. And I don’t trust any serological test that doesn’t verify its positive samples from the lateral flow kit with a more specific test.

        • Mendel –

          Thanks. That’s a very instructive comment.

          > Considering this, it was a wise choice not to weight by age, but then they ought to not have talked about the mortality in the study at all.

          That’s my intuition – but it’s good to see someone lay out the numerical argument. Thanks again.

        • Except that there is no way to conclude that this is “like the flu” in the face of the raw body count. At 50,000+ and still counting, in the space of just over one month covid-19 has already killed more people than we see die from flu in almost any year. We have never had corpses piling up in refrigerated trucks outside hospitals in NYC with a flu epidemic.

          The only way you can conclude that the infection fatality rate is as low is the flu is by proving that the infection attack rate is an order of magnitude or more bigger than that of the flu. So either way, it’s not “like the flu” in any meaningful sense.

          Yes, it does make a difference to how you would best try to manage the epidemic whether it is a highly contagious disease with a low fatality rate or just moderately contagious but with a very high fatality rate. But that’s not what people who say it’s “like the flu” are getting at. They’re trying to argue that it’s not a big deal and that we should not have taken extraordinary steps to try to curtail this epidemic. But the reality is, no matter how you slice it, with no new studies needed, we know that this disease causes substantially higher population mortality than the flu.

          Given that the point of “like the flu” is political, not scientific, it should surprise nobody that bogus evidence and bogus interpretations of real evidence would be brought to bear in its defense, even if it were (counterfactually) true.

        • I don´t think that is fair to Iannodis, my impression is that he is just an honest, sharp researcher trying to find the truth. Not that he stands above that kind of reasoning, but neither does you or I. I think you should stick to critique their methods instead of trying to make people suspicious about their motivations.

        • @Ben,

          I didn’t write the comment specifically about Ioannidis; in fact, I didn’t even have him in mind when I wrote it. But I can see how in the context of the whole comments section it could be taken as such. While I don’t always agree with Ioannidis, I have enormous respect for him and his contributions to medical research, and I did not mean to impugn his personal motivation. I should have been clearer about that–it didn’t occur to me that it could be read as an attack on him.To the extent that my post has been misunderstood by readers, I regret that and I apologize for sowing confusion.

          I was trying to point out that *in the population at large*, the “like the flu” motif is a political, not a scientific, one. Feelings about it will, for most people, represent what Tyler Cowen calls emotional affiliation.

      • There is even more uncertainty. The “death rate” must mean the fraction of people who get sick who later die. So you should divide the number of deaths at time t by the number of cases *when those patients got sick*, not the number of cases at time t. For US data, there seems to be about a week lag between features in the confirmed case counts and in the reported death numbers.

        When the number of new cases is growing rapidly, then that difference of a week makes a large difference in the raw number you would compute. And of course the time lag can change, too.

    • Joshua:

      I do think that extrapolation is a problem. That’s why I said that these conclusions were not supported by these data alone; they also reflect the researchers’ priors.

      I wish these researchers made it more clear what their priors are, and the sensitivity of their conclusions to their priors. Part of the problem, I think, is that in science there’s a norm of acting as if a single study is definitive, and a norm to not talk about priors.

      I connect this to the problems with their statistical methodology, in that in both cases there is an understatement of uncertainty.

      • Andrew –

        Thanks. Not being particularly statistically oriented (I even tend to stumble over pronouncing the word “statistics”), I think is part of my issue here.

        But I can guess that from a statistical orientation the problem might not be more broadly methodological – but from a problem of not properly identifying priors?

        Would that be correct?

        For me, the problem is less mathematical (i.e., extrapolating from non-representative data), but maybe that’s because of my particular angle of approach?

  4. The obituary page is now a section of 18 pages, so Covid-19 kills more people than the flu. I don’t know how you’d otherwise explain that real world data; it’s not like they’re collecting deaths to report them in a big batch every now and then. And funeral homes report greatly increased business. (Black humor opportunity: nice to see someone is doing well in trying times.)

    The composition of the obituary page also reflects how the infection fatality rate is significantly skewed towards the extremely old. Both of these are born out by every reported listing of deaths. In fact, I ran across an analysis that incorporated excess death estimates from Italy that found the same general results as you can find by looking at MA’s daily stats. That intuitively makes sense because I’d expect MA to keep better records. There is little sense that MA is under-reporting Covid-19 fatalities. One can argue the other way: the extreme age of the dead, with many more having died over the age of 100 than under the age of 50, suggests you could list Covid-19 as a contributing factor rather than as ‘the’ cause of death. These reports – can’t really call them ‘studies’ because they’re so incomplete – all suggest the IFR is flu-like for people under 50, and somewhat higher with each decade, until it becomes scary high at 80 and above.

    I’m posting this because I don’t see that changing as more information comes out. That’s my guess.

    In terms of debate, it seems to me that many people don’t want to think about the varying infection fatality rates – which track hospitalization risk, which track risk of being put on a ventilator – because they don’t want to think that the dire warnings about how everyone is at risk may be causing other damage. I don’t only mean economic damage, which people who have jobs and money seem to dismiss rather easily, but the very large drop in the number of diagnostics being done, which means shortened or lost lives (plus pain, etc.) because of delayed diagnosis. As in, you catch a lung cancer at stage 2 and you survive better than if it reaches stage 3, and the cost to the system is much lower. Some cancers don’t matter; you only discover esophageal cancer at stage 4, when you suddenly can’t swallow – because you only need 11 millimeters of open space to function – and that’s a death sentence no matter what. But for example, the daughter of a friend head surgery for ovarian cancer canceled the night before, and was told maybe in June. The cancer will almost certainly kill her but delaying surgery means she could have years less time with her children. (They squeezed her in last week, after a delay of 6 weeks.) And of course hospitals are reporting that people are not seeking treatment for heart and stoke related problems out of fear of getting Covid-19.

    My guess is that we won’t learn much. For example, a main risk factor seems to be weight, because diabetes has a systemic effect on epithileal cells. They’ll likely get more understanding of how that process works with Covid-19, but I don’t see a treatment for that risk other than herd immunity and a vaccine (which might only be relatively effective in people with diabetes). We may, for example, get a better understanding of pre-diabetic effects, meaning the vast number of people who could be diagnosed as type 2, but I doubt we’ll understand how diet impacts this (other than being thinner is better).

    I hope at some point we might better grasp basic differences between, for example, the ability of a virus to be detected and the ability of a virus to infect. And we should at some point better grasp the relationship between susceptibility to the virus and the infection fatality and hospitalization rates. Those are conflated now in scary ways because, for a variety of reasons, people want to believe everyone is equally at risk. To me, this is a terrific example of how groupthink works: stamp out irresponsible behavior that doesn’t agree with the idea that we’re all in this together. I don’t phrase this in forms of ‘liberty’ arguments, but I can see why some do.

    But if the argument is that we need to figure out fatality risk, I think we have a lot of data. That assumes the NYC data is reasonably correct because that says NYC wasn’t so much an outlier about results as it was a larger outbreak than estimated so the curve reached higher.

    • Jonathan –

      > I don’t know how you’d otherwise explain that real world data;

      A couple of explanations that I’ve seen:

      (1) More people are dying because they’re afraid to go to the hospital.

      (2) More people are dying because they’re going to the hospital, and dying from hospital acquired infections.

      Note, that there’s an interesting tension between those two explanations…but that doesn’t seem to bother some people who think this is just as fatal as the seasonal flu.

    • Jonathan –

      > In fact, I ran across an analysis that incorporated excess death estimates from Italy that found the same general results as you can find by looking at MA’s daily stats.

      • Trying again –

        > In fact, I ran across an analysis that incorporated excess death estimates from Italy that found the same general results as you can find by looking at MA’s daily stats.

        Italy has an older population. Fewer ICU beds per capita. Italians tend to live in multi-generational households, and I would assume higher average number of people per household. And, in the area hardest hit, I would imagine there was more to/from traffic from Wuhan.

        IMO, for you to be comparing excess death rates, you should be controlling for a lot of variables before drawing causal conclusions such as ” many people don’t want to think about the varying infection fatality rates.” Of course, there’s absolutely nothing wrong with speculation…

        • First, when I say ‘many people’, I’m not pointing at you or anyone in particular but in general there is a lot of discussion about a generalized infection fatality rate. IMO, people in general avoid talking about stratification becuase they want to create more pressure to adhere to public health requirements, so people who have much less risk comply. My concern is fairly simple: we are teetering on the brink of how we afford to proceed. We have been proceeding as if we have no financial limits. I doubt that can continue much longer. It seems to me that knowing risk levels might offer a way forward: we hope antibody tests work, but we still face the problem that significantly older people (and a few other classifications) will die at a very high rate if they are exposed. That leads to the possibility of a different form of public health response. Example: saw an old woman pushing a stroller, probably caring for a grandchild, giving everyone a wide berth, but I’d think her main infection vector is from her family. If we’re not isolating these people now, we may have to isolate them in the future. You may disagree. You may believe we need to lock down through July or some further defined point.

          I have not compared excess death rates. My comment says nothing about them except that I note an analysis of them in Italy suggests a similar profile of overall fatality infection rate by age cohort when they include excess deaths. I don’t know how their analysis works. I thought I was clear that the evidence points to a roughly .5% death rate. That’s many times the flu. I mean seriously: 1% of 100M is a million people, so half of that is 500,000. That’s right in line with many of the projections all along. There are 325M people in the US. If you somehow think that I’m pooh-poohing the number, then I don’t understand: .5% is a very large number when you look at the percentage of people infected.

          Informationally, all the deaths reported are Covid-19 deaths, not deaths from other causes. There was an excess death calculation for March, 2020 comparing across for that month 1999 to now. It found about 500 excess deaths, but that was an average taken over the entire period, and did not consider population growth or that the population has become older. The 5 year prior average showed excess deaths of about 280. My guess is that this occurred within the same population affected most by Covid-19 because an excess of 280 or 500 or whatever number is less noticeable when it occurs among the very old. It may mean deaths in nursing homes weren’t being reported as Covid-19. It may mean old people dying elsewhere. My guess tends to both. The reason I tend to both is mostly anecdotal interpretation: there are a handful of small clusters of older people who have died proximately. I noticed, for example, 3 people, 2 elderly from the same family, died within 3 days. They ran a convenience store in Fall River, among an immigrant community that I suspect is less tested. This suggests perhaps they were exposed ‘repeatedly’, whatever that is defined as.

          If the idea is that people are dying because they’re afraid to seek treatment, then the messaging is wrong. I’ve noted that before: a cost of the path we’ve taken is that diagnostics, etc. aren’t being done. I was surprised by the size of the drop in hearts attacks, strokes, etc. Given that hospitals have open beds, at least outside NYC, I’d call it a bad public policy result to so scare people that they die or are crippled to a material degree by fear of being treated.

        • Jonathan –

          > First, when I say ‘many people’, I’m not pointing at you

          > I wasn’t taking it personally.

          or anyone in particular but in general there is a lot of discussion about a generalized infection fatality rate. IMO, people in general avoid talking about stratification becuase they want to create more pressure to adhere to public health requirements, so people who have much less risk comply.

          OK, I think it would be foolish to say that there isn’t anyone out there that complies with that speculation.

          But I’d offer some speculation in the other direction. I have seen folks who are working extremely hard to justify their feeling that their freedoms are being infringed upon, and that this is mostly about a bunch of tyrannical libz who are caught up in a hysterical panic. Did you see that video of those Cali doctors who used the most obviously fallacious calculations (obvious enough that they were readily apparent even to me) to argue for why we need to end the government mandated social distancing?

          We should expect for these types of motivated reasoning to take place in both directions, and, IMO, we should avoid extrapolating from the fact that we can find examples in order to build some kind of model where there is some overall imbalance. Take a carefully calibrated approach to examine the uncertainties.

          > My concern is fairly simple: we are teetering on the brink of how we afford to proceed. We have been proceeding as if we have no financial limits.

          I don’t think that’s true in the least. I don’t think that anyone, except maybe the most wealthy people in the country, think that we should proceed as if we have no financial limits.

          > I doubt that can continue much longer. It seems to me that knowing risk levels might offer a way forward: we hope antibody tests work, but we still face the problem that significantly older people (and a few other classifications) will die at a very high rate if they are exposed.

          Sure. That’s one side. The other side is that massive downside fat tail high end risk of just opening everything up and making assumptions about herd immunity, without the consideration that slowing down the spread might allow time for the development of better therapeutics an potentially lower the total number of those infected before some kind of vaccine might be developed.

          And then, of course, there is the factor of resources and supplies to heatlchare workers.
          IMO, since they’re out there putting their lives on the line, they deserve a special level of consideration. But I don’t take any of this lightly in terms of the downside of continued mandates social distancing – and I doubt that there are very many people who do. From where I sit, the argument that there’s some block of power people who simply ignore those possibilities is borne out of an ideological, identity aggressive cognitive bias.

          > That leads to the possibility of a different form of public health response. Example: saw an old woman pushing a stroller, probably caring for a grandchild, giving everyone a wide berth, but I’d think her main infection vector is from her family. If we’re not isolating these people now, we may have to isolate them in the future. You may disagree. You may believe we need to lock down through July or some further defined point.

          No. I think that the idea of isolating the vulnerable is a key factor to consider. But I also think that we need to consider the practicality of that option very carefully. We don’t have a culture that allows a lot of very vulnerable people to realistically isolate – and you’re talking about having those who can do so effectively make the decision that those other people are a justifiable sacrifice. Well, ok, some tough decisions have to be made – and I’m not saying that sacrificing isn’t a valuable consideration – but you have to have the discussion without demonizing anyone.

          My own belief is that what is key is testing and contact tracing (and having the resources for effective isolating). There is, IMO, a basic inverse relationship between the functionality of testing and contact tracing and the need for mandating social distancing. With a robust system of testing and contact tracing, no such mandates would be need at all.

          > I have not compared excess death rates. My comment says nothing about them except that I note an analysis of them in Italy suggests a similar profile of overall fatality infection rate by age cohort when they include excess deaths. I don’t know how their analysis works.

          I don’t understand the point of mentioning that except as a point of reference.

          > Informationally, all the deaths reported are Covid-19 deaths, not deaths from other causes. There was an excess death calculation for March, 2020 comparing across for that month 1999 to now. It found about 500 excess deaths, but that was an average taken over the entire period, and did not consider population growth or that the population has become older. The 5 year prior average showed excess deaths of about 280. My guess is that this occurred within the same population affected most by Covid-19 because an excess of 280 or 500 or whatever number is less noticeable when it occurs among the very old. It may mean deaths in nursing homes weren’t being reported as Covid-19. It may mean old people dying elsewhere. My guess tends to both. The reason I tend to both is mostly anecdotal interpretation: there are a handful of small clusters of older people who have died proximately. I noticed, for example, 3 people, 2 elderly from the same family, died within 3 days. They ran a convenience store in Fall River, among an immigrant community that I suspect is less tested. This suggests perhaps they were exposed ‘repeatedly’, whatever that is defined as.

          I see a lot of uncertainty about death counts. There was just a WSJ article where in England the actual number of deaths is 2x the offical reports. I don’t want to be insulting, but I suspect that you haven’t done nearly enough investigation to come up with a meaningful estimate on your own. I could be wrong about that, but it’s my honest “prior.”

          > If the idea is that people are dying because they’re afraid to seek treatment, then the messaging is wrong.

          Of course! And I don’t dismiss the idea that such situations are occurring. But in the end we don’t know the number of them relative to all the other uncertainties in the death counts.

          > I’ve noted that before: a cost of the path we’ve taken is that diagnostics, etc. aren’t being done.

          There are many costs that we’re incurring here. And I think it’s simplistic to say that’s occurring because of the mandated social distancing. We could just as easily say it’s occurring because of the failures in testing and contact tracing. Or we could say it’s occurring because we’ve been starving out public health infrastructure for the last decade.

          Causality is hard.

          > I was surprised by the size of the drop in hearts attacks, strokes, etc. Given that hospitals have open beds, at least outside NYC, I’d call it a bad public policy result to so scare people that they die or are crippled to a material degree by fear of being treated.

          Sure. But if there were no mandated social distancing, and hospitals had been overwhelmed, and death rates were higher, and resources were more lacking, then I would argue that the situation you’re describing would actually be worse. I don’t think we should just say that this is a “panic” created by bad public policy. Too simplistic for me. And too ideologically tainted. We could say that it’s bad public policy to underfund public health resources also.

        • Joshua, I like your response.
          Just one detail:

          > My own belief is that what is key is testing and contact tracing (and having the resources for effective isolating). There is, IMO, a basic inverse relationship between the functionality of testing and contact tracing and the need for mandating social distancing. With a robust system of testing and contact tracing, no such mandates would be need at all.

          We had contact tracing in Germany by default, but it looks like some people came home from Italy, didn’t realize they were infected, and attended a large carnival party: lots of singing and shouting, close dancing, a room probably not well ventilated: a big outbreak sprung up in Heinsberg.

          If people remember to wash hands – don’t touch face – 6 feet distance — cough into the elbow — wear a mask around people, then you probably still have to mandate large gatherings closed. It’s probably still safer for people to wait outside the store than in the checkout line. Choir practice might still be a bad idea for a long time. And people might go back to the office more, but probably better keep their home offices. And I have no idea how to handle care homes, but opening them to visitors sounds very dangerous still.

          There’s the idea in “The Hammer and The Dance” from Thomas Pueyo that everything you can do contributes to lowering R. Contact tracing does a lot because it identifying and isolating contacts breaks a lot of transmission chains. But by itself, it’s not enough. It may have been almost enough — we had a kindergarten worker develop symptoms on the last day before the lockdown, and they nipped that in the bud –, but you’re still going to want to keep super spreader events like the ones in Heinsberg and South Korea from happening. If you have a spreader in close contact with a lot of strangers, you’re going to see an outbreak.

          So there’s probably no way around doing a nuanced strategy that contains mandatory elements — but you’ll need a lot less of these mandates if your state is good at contact tracing, and your citizens still take this seriously.

        • Mendel –

          Fair enough. I’ll amend to say there’s a (probably not directly proportional) inverse relationship between testing/contract tracing and mandated social distancing within a bounded range?

        • I’m personally interested to see what “Contact Tracing” we develop.

          Based on what we know thus far about the virulence and transmissibility of SARS-CoV-2; it can transmit like the “common cold”. That poses a major problem for Contact Tracing.

          Sure, you can trace people after the develop symptoms. But when does an infected person become infectious? Is it 1d pre-symptoms? 2d? 3d? 7d? How many carriers remain asymptomatic? How many are paucisymptomatic (e.g., presenting symptoms similar to other very common illnesses) that may not get tested because they think its a cold or allergies?

          Answering these questions is key. If someone can transmit the virus even 1d before symptoms, Contact Tracing becomes a very difficult problem for graph/network analysis. (My understanding, which may be wrong, is that Contact Tracing simplifies to a problem of graph/network theory.) A single person in a suburban or urban setting could easily expose tens, even hundreds, of people in 1d.

          To date, I’ve seen some data about when someone become infectious — e.g., the familial studies from Wuhan (https://doi.org/10.1016/S1473-3099(20)30114-6). Those studies are a start, but their overall sample-size is too low to be decisive. Some values from models, too, but those are far from decisive either IMO.

          Answering the above questions, though, is tricky.

        • It’s ~2 days (24-48 hours) pre-symptoms that an infected person starts shedding virus.
          If you contact trace someone, you notifiy and isolate the contacts. If they develop symptoms (pauci or otherwise), they get tested, and *their* contacts get traced and isolated. (When you get notified, they tell you to make a diary of contacts, going back two weeks, and of course keeping it current.)

          The isolation takes care of asymptomatic people (mostly).
          Honestly, I don’t care whether you can analyse it, the main thing is that it works. If you can mandate an upper bound to the edges of a node, it gets easier.

          Exposing hundreds of people requires a large event. The German criteria for a high-risk contact is face to face, >15 minutes, <1.5m, and even then the attack rate is only 5%. So to actually *infect* 10 people, they'd have to be in close contact with 200 for 15 minutes each, if you do it in groups of four maybe it'd work. Or you attend choir practice.

          Answering these questions is easy when you are already contact tracing, and if you've done it early enough to be able to backtrace. That's what the Munich cohort was about in Germany (there's a paper that came out March 31st on the Lancet preprint server).

          Prof. Streek says he's doing that in Heinsberg; Munich was a work setting for all of the infected (and the secondary infections outside work were household members), but Gangelt had widespread community transmission, and maybe they were able to find out which kind of everyday activities are more dangerous than others.

          So that's being done.

          (And "common cold" is actually good; because the virus rarely aerosolizes when it's still in the nose, the distance rule works. If we had aerosol infections being a major route, or fecal-oral, it'd be much worse.)

          Don't just ask questions. You know countries are doing contact tracing, so the answers must be out there. I think there's a multi-page document at the WHO about it.

        • Lots of unknowns going forward with contact tracing.

          My guess is that Aussies are pretty high on the individualism over collectivism ratio, yet:

          –snip–

          > A coronavirus tracing app released by the Australian government on Sunday has already been downloaded more than a million times, despite privacy concerns.

          Registering with the app, called COVIDSafe, is optional but strongly encouraged by health officials who say it will help speed up the contact-tracing process and allow people to get diagnosed and treated early.

          Using bluetooth technology, the app logs each time that a user comes within 1.5 meters (4.9 feet) of someone else. When an app user later on tests positive for coronavirus, their log of recent encounters will be accessed and everyone who has had more than 15 minutes of close contact with them will be notified.

          https://www.washingtonpost.com/world/2020/04/27/coronavirus-latest-news/#link-BGXI32PGOBFVZJFSK7BYB4LET4

          –snip–

          And then we have the consequential unknowns like deciding when when the feds should force companies to do stuff.

          Force them to make masks? Nope. Force them to produce meat? Yup.

          Now which of those decisions impact in which direction the need and effectiveness of contact tracing?

        • Mendel,

          > If you contact trace someone, you notify and isolate the contacts.

          This simplifies a complex process. In ~2d, someone could easily interact with hundreds of objects and people within their 6ft radius while outside their home. How can you expect them to remember all this? Even if you use data from mobile devices, how do you determine what objects in the 6ft radius the person touched? What if someone was in the radius for 1 min versus 10 versus 20? Does that matter?

          What defines ~2d? Can you provide sources for this very key number?

          I’ve read the study from Singapore (https://www.cdc.gov/mmwr/volumes/69/wr/mm6914e1.htm); it uses four clusters of small sample-sizes, so it’s not definitive (but a start and not something to ignore; 1-3d is a working value from this study). This study (https://www.medrxiv.org/content/10.1101/2020.03.03.20029983v1) is more comprehnsive, but they only seem to report average values (on pg 12) for pre-symptomatic infectious period. Is it not critical that we know the bounds of this period?

          > Exposing hundreds of people requires a large event. The German criteria for a high-risk contact is face to face, >15 minutes, Don’t just ask questions. You know countries are doing contact tracing, so the answers must be out there. I think there’s a multi-page document at the WHO about it.

          I am aware. I’ve read the CDC’s page on Contact Tracing. (https://www.cdc.gov/coronavirus/2019-ncov/php/principles-contact-tracing.html) And I still have questions. Therefore, I ask questions because to know what others think and learn. What is wrong with that?

    • I think we don’t disagree, but just to clarify:

      * The data from mortality surveillance shows that many more people die right now than they did in the historical average. These are not people “who would have died anyway” (well, obviously we are all going to die), but would usually have lived on longer. The count of deaths attributed to Covid-19 typically is less than the observed mortality in many states and other hard-hit countries as well. This has been very well documented in Italy.

      * A Covid-19 infection is not necessarily harmless even if you survive. We have anecdotal evidence for long-term lung damage, an influence on blood clotting, and neurological effects as evidenced by the temporary loss of taste and smell in over 80% of cases. Lontg-term effects on health of child development are currently unknown. Surviving a pneumonia is not an easy thing, you will remember this all your life.

      * It’s likely we are seeing health benefits due to the massive reduction in traffic and, therefore, air pollution.

      “I hope at some point we might better grasp basic differences between, for example, the ability of a virus to be detected and the ability of a virus to infect.” — preliminary evidenec suggests there is no difference; it’s genetically the same virus, and the nasopharyngal virus concentration is the same no matter what your symptoms symptoms are (including no symptoms). We already know that people with symptoms infect others before the symptoms reveal themselves. There is no eveídence for a “basic difference” here.

      “Those are conflated now in scary ways because, for a variety of reasons, people want to believe everyone is equally at risk.” — I dislike talking about “people” without support. The messages I see in the media is that we need to curb the spread of the virus to protect the vulnerable populations. Officials communicate the risk to others, “to grandma” (e.g. NY Gov. Cuomo, German Chancellor Merkel, WHO Director Ghebreysus), and ask their population to behave responsibly. We’re all going to be old and vulnerable some day, and most of us wouldn’t want to be sacrificed when the next epidemic comes along. (Think of the economic impact if the workforforce retired 10 years early, “enjoy life before the next epidemic comes”, if you lack empathy.) If you equate “care for the elderly” with “groupthink”, you ought to consider which group you’re in.

      The NYC data is very inaccurate regarding fatality risk, as the percentages in the 65+ age groups have shrunk subtantially this week compared to last week, which should not have happened. This indicates that these data are not a reliable source when it comes to fatality rates. The 65+ age groups are partzicularly important as they have the most deaths; I believe the median age of death in New York was announced to be 70 or 71 years?

      • Well Ferguson from Imperial who is actually an epidemeologist says that 2/3 of those who die “with” covid19 would have died within a year from their serious chronic health problems. Lots of old people these days are kept alive for years with congestive heart failure, COPD, diabetes, cancer. People in the end stages of those diseases are very ill indeed and their health is fragile.

        The correct metric is excess mortality. According to Ferguson’s estimate, excess mortality will be about 1/3 of the numbers of fatalities over a year. It’s misleading to use the raw fatality rates.

        Ferguson’s calculations show that those under 20 have an IFR of about 0.004%. That’s so small as to rate the danger virtually nil. The IFR rates are very strongly age dependent. If we merely protected nursing and care facilities with massive testing and screening, perhaps most other healthy people could get back to normal. Among this group of people IFR’s are almost certainly not higher than those for flu.

    • Jonathan,

      You make some excellent points — many that have been concerning me for weeks now.

      Second, the data from MA is fantastic. Much better than others, like NJ (which is awful) and NYC (which is decent).

      > One can argue the other way: the extreme age of the dead, with many more having died over the age of 100 than under the age of 50, suggests you could list Covid-19 as a contributing factor rather than as ‘the’ cause of death. These reports – can’t really call them ‘studies’ because they’re so incomplete – all suggest the IFR is flu-like for people under 50, and somewhat higher with each decade, until it becomes scary high at 80 and above.

      Exactly! How are we sure that COVID-19 is always the cause of death? Given that most of these patients being elderly or very morbid: it is quite possible that COVID-19 contributed to death, but was not the sole or final cause. How many had co-infection with nosocomial pathogens at death, like strep, c-diff, etc.? How many did not? Understanding this unknown is crucial to really understanding the disease in terms of pathophysiology, and such understanding is crucial to improving clinical treatment.

      What really baffles me is the lack of detailed autopsies thus far. Sure, performing autopsy of a BSL-3 pathogen (SARS-CoV-2’s current rating) is not trivial, but the facilities and are certainly available. But we’ve had two months. I’ve only found one detailed autoposy (e.g., one that includes histopathology of the lungs) to date: https://academic.oup.com/ajcp/advance-article/doi/10.1093/ajcp/aqaa062/5818922.

      > These reports – can’t really call them ‘studies’ because they’re so incomplete – all suggest the IFR is flu-like for people under 50, and somewhat higher with each decade, until it becomes scary high at 80 and above.

      I did a quick analysis for hospital-fatality-rates (HRF) for various locales, which *should* be less uncertain that infection-fatality-rate, since we know the counts for COVID-19 hospitalizations and deaths more accurately that total infections:
      – NYC: 11708 deaths / 40050 hospitalizations = 29%.
      – MA: 3,003 deaths / 5,237 hospitalizations = 57%.
      – NJ: 6044 deaths / 9266 hospitalizations = 65%.
      – CA: 1755 deaths / 3375 hospitalizations = 52%.
      – Spain: 15175 deaths / 79697 hospitalizations = 19%

      The above variance is huge. Clearly, some other factors — risk factors, sub-outbreaks (like in NJ, where 50% of deaths originated from longterm care), different reporting, different diagnostics — are influencing the fatality of COVID-19 besides its inherent virulence and pathogenesis.

      > I don’t only mean economic damage, which people who have jobs and money seem to dismiss rather easily, but the very large drop in the number of diagnostics being done, which means shortened or lost lives (plus pain, etc.) because of delayed diagnosis.

      Great point!

      Some will argue that its a function of the pathogen overwhelming healthcare. But that is a function of healthcare being unable to dynamically adjust its supply to meet changing demand — that problem is exclusive to COVID-19. Any pathogen could cause such stress. Therefore, one cannot conflate the two issues.

      > I hope at some point we might better grasp basic differences between, for example, the ability of a virus to be detected and the ability of a virus to infect.

      Preach. As of now, we really do not know what viral load that PCR detects corresponds to actual infection. Detecting infection via PCR is NOT an accurate practice (as anyone familiar with or well-practiced in PCR could deduce).

      Especially in mild cases, whose symptoms resemble many other common illnesses (which we are NOT testing for to rule them out). Since the predominant number of cases are mild (at least from the data I can find in the US and Italy), how do we know that we are artificially inflating our counts for those infected AND those that are infectious? Why has little-to-no work on this topic occurring given its importance?

      • > since we know the counts for COVID-19 hospitalizations and deaths more accurately that total infections: […] NJ: 6044 deaths / 9266 hospitalizations = 65%.

        Don’t assume accuracy in anything. Deaths are 6044 out of 111188 cases, hospitalizations are 9266 out of 32923 cases with known hospitalization status. We don’t know much about the remaining 80k cases.

        • Carlos,

          Fair point. I should have made that distinction (and my related assumption) more clear; especially since doing so further highlights the gross uncertainty about the true prevalence and fatality of the virus.

    • “all suggest the IFR is flu-like for people under 50, and somewhat higher with each decade, until it becomes scary high at 80 and above”

      Are you using the aggregate IFR for the flu to age-stratified IFR for COVID-19? Shouldn’t we compare age-stratified data to age-stratified data? Trevor Bedford looked at this: https://github.com/clauswilke/COVID19-IFR, and concluded that COVID-19 has a higher IFR for all age-groups.

      • Dalton,

        Even if you age-stratify, the comparison still isn’t equal. The reporting for COVID-19 versus seasonal influenza differ substantially.

        For COVID-19:

        At present, it seems like healthcare is attributing most pneumonia-related deaths to COVID-19 without prudence — e.g., they are not performing detailed autopsy and testing for hospital-acquired infection to discount them as probable causes. I’ve heard rumors online and from friends/colleagues in healthcare — again, these are just rumors — that there is a tremendous top-down pressure — viz, from state DOHs — to record pneumonia-related or unknown-cause deaths to COVID-19. It would not surprise me (as terribly sad as it would be) if there is some pressure from politicians to skew data to fit a narrative.

        Even without top-down pressure, physicians are working extreme hours and likely unable to think clearly (and who really coul, if they are working 12-18hr shifts 6-7 days per week seeing death every day?); so it is possible groupthink is skewing reporting of deaths to COVID-19.

        And as I have stated and as Jonathan addressed, a MAJOR unaddressed question is this: Is COVID-19 the sole cause-of-death or one of many in those who have died during the pandemic? Or, is COVID-19 weakening vulnerable patients, thereby bringing them to the hospital where they quickly contract a hospital-borne infection that “finishes the job” COVID-19 started?

        Further, how many of these patients have risk of dying the next year? The next two? The next five? It would be very interesting to see actuarial data and estimates — like actuarial age — for those who died from COVID-19. (Sadly, obtaining such values is likely impossible because of HIPAA and other red-tape.) For example, someone age 70 has a 2.3% chance of dying during the next year; someone age 80 has a 5.8% chance; someone age 85 has a 9.7% chance. (https://www.ssa.gov/oact/STATS/table4c6.html#fn1) When you add comorbitities like hypertension, diabetes, obesity, etc., these probabilities only increase (granted, the linked table accounts for them somewhat, but not specifically. There are more specific tables to use, I’m sure; granted, many are likely proprietary to insurance companies and their data.)

        For seasonal influenza:

        It has been around for centuries. It is part of life. As a result, many deaths from influenza among at-risk patients go undiagnosed — “just another elderly patient who died of pneumonia.” Furthermore, testing for influenza does nothing to improve treatment once infection has progressed to pneumonia (antivirals only work early in infection), so its very often not performed.

        As a result, we do not have a robust measure for deaths from influenza. Sure, we have some statistics and models, but they have limits. So its not fair to compare the two like the analysis you linked to does.

        • “Lead task force health experts Anthony S. Fauci and Deborah Birx both rejected an increasingly popular theory on Fox News and elsewhere that the number of coronavirus deaths is being inflated because people who are infected are actually dying of other things.

          The theory has been promoted by Fox’s Tucker Carlson and Brit Hume and was a subject of a panel discussion on the network on Wednesday afternoon.

          Birx was clear that if people have an underlying condition, it is exacerbated by the virus that causes the disease covid-19.

          ‘So those individuals will have an underlying condition, but that underlying condition did not cause their acute death when it’s related to a covid infection,’ she said. ‘In fact, it’s the opposite.’

          Fauci then weighed in, warning against such “conspiracy theories.”

          ‘You will always have conspiracy theories when you have a very challenging public health crises. They are nothing but distractions,” he said, adding: “I would just hope we just put those conspiracy stuff — and let somebody write a book about it later on. But not now.’ ”

          https://www.washingtonpost.com/politics/2020/04/08/white-house-coronavirus-briefing-wednesday/

        • > I simply don’t think your questions are honest.
          They contain assumptions, and you don’t seem invested in finding answers.

          Seems like your prior is my being a conspiracy-theorist regardless of what I say from now on…

          > Like, I google “covid-19 autopsy pre-print”, half a dozen papers on the first page of results, you are asking “why are there so little”, you clearly never even looked.

          Of the first 10 links on Google (page 1):
          – Five are the same paper: “Pulmonary and Cardiac Pathology in Covid-19: The First Autopsy Series from New Orleans”; which I’ve read. It analyzed four patients. Nice work that I applaud.
          – Two more links are to unique papers. One I have seen and one I have missed.
          – I checked page 2 of Google and found no new papers. Same for page 3.
          – I gave you the benefit of the doubt and looked for 20min. Found one other paper, that I’ve already read, where they performed minimally invasive autopsies.

          So by “half a dozen” papers, you mean four in addition to the Oklahoma study I already shared?

          Clearly, I missed tons of papers…

          > And it isn’t the job of pathologists to publish a paper on everyone they cut up.

          Sure. But is it not also important for them to share their data in times of crisis for a novel pathogen?

          Again, I’m not criticizing pathologist. Not sure how many times I need to say so. I’m just surprised there isn’t a more public call for this work and its results.

          > But your long reply is again chock full of unfounded assumptions and rhetorical questions that you don’t really seem interested in. If they weren’t also so negative, it wouldn’t matter to me. But they are, and it makes me sad.

          Nice rhetoric: By claiming my long reply is “chock full of unfounded assumptions and rhetorical questions” but not listing them, you don’t afford me the opportunity to defend myself. By not addressing my questions for you to explain yourself further, you avoid culpability of your own points. Are you politician?

          Further, in my whole response, I asked one potentially rhetorical question: “But why, after 2 months, we have so little autopsy-based and general pathological data on COVID-19?” That is it.

          ……so to me it seems like your mission is labeling me a “conspiracy theorist” and nothing else. Have fun, I guess?

        • Anonymous –

          The Hannity comment wasn’t constructive.

          On the other hand, you are promoting a conspiracy theory. Sometimes conspiracy theories are true – but the way to work with that is to provide some actual evidence.

          > It would not surprise me (as terribly sad as it would be) if there is some pressure from politicians to skew data to fit a narrative.

          I would imagine that if that were true, it would necessarily suggest pressure in both directions. But also, if it were true, then it would involve a conspiratorial collaboration between a whole bunch a’ people. Seems to me with a conspiracy that large, someone, somewhere, would be stepping forward to blot the whistle. To say, “Here is documentary proof of a politician pressuring me to skew data.”

          Without such evidence, I question whether there’s something about your own political outlook which inclines you towards such a belief.

        • Joshua,

          I agree my Hannity comment was not constructive. Thank you for calling me on it. I suppose I struggle in coming up with constructive ways to responds to conspiracy theorists. Any counter-evidence or rational argument one offers can always be “refuted” in their minds by adding another layer to the conspiracy onion. E.g. “Well, you can’t believe Fauci or Brix cause their motives are impure because they’ve been bought by the radioactive toilet paper lobby that’s trying to fluoridate our bidet water.”

        • Dalton –

          Yes, impugning motives based merely on generalized speculation as opposed to specific evidence is a sign of conspiracy ideation. Not sufficient to prove it, but it’s a sign.

          It’s a sign because one can use that argument in virtually all situations. IMO,the bar for conspiracies (which do happen, after all), need to be higher than that.

          FWIW, I’ve had some good convos with Anonymous. At least if it’s the same anonymous. Kind of hard to tell but use of “fair” seems like a tell.

        • Joshua,

          Thank you for the kind words! Yes, it is the same Anonymous. (Not a Russian Bot or Republican Operative or Lizard Person or anything…)

          I worded the first paragraph of this thread poorly. And re-reading it, it sounds stupid. Some things you wish you could delete. That’s life, I guess?

          Its frustrating that as I attempt to defend myself, though, when I see the following occur:
          – Me: “Hey, sorry. That isn’t what I meant, allow me to clarity.”
          – Others: “You are Sean Hannity, a Flat Earther, unscientific, eroding society, eroding trust in science” etc.
          – Me: “I’ll try again to explain myself.”
          – Others: “You are a conspiracy theorists”.

          This kind of response makes me regret sharing my thoughts and want to rescind to just reading Andrew’s posts.

          I am not a conspiracy theorist. (Except maybe Bigfoot, because a large bipedal ape capable of making all film grainy would be quite the find.) I’m an engineer by training who performs biomedical research. I want to help people. Conspiracies don’t help people. So you can imagine how such a label makes me feel.

          P.S. I do stand by my comment about groupthink among overwhelmed and overworked physicians; its a real concern to consider. As the Checklist Manifesto highlights, groupthink is just as present in medicine as any other profession — we’re all human. And I’d hope the powers-at-be are watching for groupthink and correcting it, but I’m not seeing that it what they communicate to the public. Maybe I’m asking too much, though. And I am NOT insulting physicians whatsoever; they are doing that hardest work during the pandemic.

        • Anonymous –

          I hope you don’t merely retreat. I have learned from your comments and been pushed to evaluate my own thinking. That’s a product of your willingness to engage in good faith exchange – a rare commodity in Interwebs comment sections.

          I understand the temptation towards in-kind responding.

          On a nit-picky note – writing a comment with signs of conspiracy ideation does not mean you deserve a label conspiracy theorist. And I wasn’t giving you that label. But I won’t back off from asserting that generalized motive impugning without supporting evidence is often a sign of conspiracy ideation. And I’ll point out that being an engineer doesn’t provide immunity from being infected with the the conspiracy virus.

          I certainly agree that group think exists. It’s a real problem. But I will also point out that in my experience, facile application of the group think explanation can often be a form of identify defense.

        • Joshua,

          Same to you! It’s fun to have discourse; that is how learning happens, after all.

          > But I won’t back off from asserting that generalized motive impugning without supporting evidence is often a sign of conspiracy ideation.

          Agreed. I’d delete that first paragraph if I could. Because it does exactly what you say and detracts from my other points.

          > And I’ll point out that being an engineer doesn’t provide immunity from being infected with the the conspiracy virus.

          Also agreed. I wasn’t trying to make it seem otherwise.

          > I certainly agree that group think exists. It’s a real problem. But I will also point out that in my experience, facile application of the group think explanation can often be a form of identify defense.

          Fair point. Groupthink is the wrong word and has too much baggage. I need a better noun to describe what I mean, but my vocabulary is failing me now. Agh.

        • Um, OK. Fox News isn’t my source, if that is what you imply.

          Here is one source: https://www.thelancet.com/journals/lanmic/article/PIIS2666-5247(20)30009-4/fulltext. The two major studies this paper cite raise clear concerns about secondary nosocomial infection and the role it has in the pathophysiology of SARS-CoV-2.

          They are a minority, but there are those in the medical community calling for more thorough autopsies and post-death pathophysiology for two reasons. First, doing so can guide clinical practice; namely, how can we improve treatment? If we know most patients experience nosocomial infections, and if these infections either cause or contribute to their death, they prophylactic antibiotics or other changes may be necessary. Second, doing so can provide a better understanding of how SARS-CoV-2 really infects and affects the lungs and other organs. There is tons of heresay and rumors about hematologic and neurological infection, but no pathophysiological data (just symptoms) to justify said conclusions.

          Here is a second source: straightforward pathophysiology.

          Anyone who receives severe illness is, by definition, much more likely experience nosocomial infection. For SARS-CoV-2, this makes sense: It causes ARDS in most patients. Its well known in medical literature that ARDS has two major facets: (1) Diffuse alveolar damage — e.g., the immune response causes such severe inflammation on the alveolar tissue that they become “diffuse”, thereby allowing interstitial fluid to flood the lungs (pulmonary edema); (2) excessive immune response, which kills numerous stromal, parenchymal, and (of course) immune cells, thereby filling the lungs with said dead cells. (1) and (2) combine to form a “sticky” fluid in the lungs. This fluid (A) impedes respiration of O2 (but not necessarily CO2; hence why pCO2 in many COVID-19 patients is fine while they have very low pO2, e.g., hypoxemia) and (B) is the ideal breeding ground for pathogens. (B) is especially of great concern for patients with invasive ventilation. Infection by antibiotic-resistant gram-negative bacteria is a very real concern for those with invasive ventilation; hence why most standard practice is to keep patients on ventilators for the absolute shortest time possible.

          The above rationale is not conspiracy; it prudent science asking for more information. To not perform thorough pathological analysis post-death of a new pathogen is foolhardy.

          Perhaps instead of calling valid scientific critique “conspiracy theories”, Drs. Fauci and Birx should consider them with credence.

        • Joshua,

          Agreed. I was at fault with my comment regarding politicians. Rumors are rumors, and I should have excluded them from my point. (It is rather fitting for the post, though, I let my priors influence my analysis!)

        • > The above rationale is not conspiracy; it prudent science asking for more information. To not perform thorough pathological analysis post-death of a new pathogen is foolhardy.

          You’ve toned it down to a lower level, but it still is a conspiracy theory. Everything you’ve explained is well known to everyone, there is no coverup, hospital hygiene is a science, these people know what they’re doing even if the general populace thinks that a device to sterilze a breathing tube with UVA while it’s in the patient is a cure for the coronavirus.

          You are assuming pathologists are taking a vacation while this is going on.

          > Even if you age-stratify, the comparison still isn’t equal. The reporting for COVID-19 versus seasonal influenza differ substantially.

          They don’t differ. Do you know how many influenza deaths still have the virus in their bodies when they die? I’ve read that most influenza deaths die from the superinfection they got while laboring under their influenza-induced pneumonia.

          It does not make sense to report these deaths as nocosomial exclusively, because then you cover up the fact that by fighting influenza — more vaccinations, for example — you could have prevented this death. In a multifactorial death, you want to know ALL of the factors. If the virus pushed you down the slide of your preconditions and the spike pit of your immune response or some superinfection killed you at the bottom, you still want to register that SARS-CoV-2 was involved. THAT is scientific.

          Your demand isn’t scientific, because it fails to pose a new question; it fails to call for new observations; it does not contain any new ideas. It is a conspiracy theory because it presumes that the public demand for these things is necessary when it isn’t. It’s uninformed, aggressive, and erodes trust in the systems that support us. And that degree of trust is actually a pretty good indicator of how states/countries do in this epidemic, at least it seems like that to me.

          At one point, you have stopped asking questions. You have switched over to making assumptions about the competence and motivation of people you don’t know and have never talked to. This wouldn’t be a problem if these assumptions weren’t eroding society. Question yourself once in a while.

        • Mendel,

          > hospital hygiene is a science, these people know what they’re doing even if the general populace thinks that a device to sterilze a breathing tube with UVA while it’s in the patient is a cure for the coronavirus.

          Not sure what your point is here.

          I would avoid making claims about the general populace. You seem to have switched over to making assumptions about the competence and motivation of people you don’t know and have never talked to

          > You are assuming pathologists are taking a vacation while this is going on.

          I’m not assuming this. But why, after 2 months, we have so little autopsy-based and general pathological data on COVID-19? Perhaps I may not be privy to limiting factors but this seems odd and frustrating. (If you have papers that say otherwise, please share!) Do you not think this data is crucial and would not be helpful?

          > They don’t differ. Do you know how many influenza deaths still have the virus in their bodies when they die? I’ve read that most influenza deaths die from the superinfection they got while laboring under their influenza-induced pneumonia.

          You do not explain how the do not differ. At present, most people with cold-like or flu-like symptoms are rushing to get tested for COVID-19. When does that happen for seasonal influenza? I’m missing something about your point here.

          Yes, it is well know that most severe infections of influenza usually result from secondary-infection with bacterial or fungal pneumonia. But how is this relevant to my claim that comparing COVID-19 and influenza differs because of different norms for _testing_ and _reporting_?

          > It does not make sense to report these deaths as nocosomial exclusively, because then you cover up the fact that by fighting influenza — more vaccinations, for example — you could have prevented this death.

          More vaccinations for influenza have diminishing returns. As you know, its not 100% effective. In at-risk patients, receiving a vaccine influenza is not a guarantee against severe illness. The vaccine

          Second, I did not say to “report these deaths as nocosomial exclusively”. Maybe you think I implied it, but I’m not.

          Third, it is not responsible or scientific to label cause of death from SARS-CoV-2 alone, when the patient is hypertensive, obese, diabetic, elderly, etc., thereby making them at-risk for severe illness or death from ANY respiratory illness. Still list SARS-CoV-2, but also check for other causes. That data can be useful.

          > In a multifactorial death, you want to know ALL of the factors. If the virus pushed you down the slide of your preconditions and the spike pit of your immune response or some superinfection killed you at the bottom, you still want to register that SARS-CoV-2 was involved. THAT is scientific.

          Yes — this is my point. So I suppose we understand each other now?

          How does SARS-CoV-2 interact with nosocomial infections? What about other common illnesses, like pneumococcus, adenovirus, rhinovirus, etc.?

          Would not understanding these questions be helpful? I imagine their answers could have important consequence on treatment during and after severe infection with SARS-CoV-2.

          > It is a conspiracy theory because it presumes that the public demand for these things is necessary when it isn’t.

          Not sure what you mean here.

          But I’d be careful with your claim. The public can demand whatever they want. Even if its stupid. Sure, it’s annoying; but the inverse — where someone determine when “public demand is necessary” for information — raises some ethical concerns.

          P.S. I’m happy to discuss and learn. But if you are going to continue ranting about my not being scientific, trying to erode trust in science and government, and trying to disrespect people, when I am posing honest questions…then we can move on with out lives.

        • As the comment section here illustrates, we all seem to let our prior opinions and emotions influence our logic!

        • I simply don’t think your questions are honest.
          They contain assumptions, and you don’t seem invested in finding answers. Like, I google “covid-19 autopsy pre-print”, half a dozen papers on the first page of results, you are asking “why are there so little”, you clearly never even looked. (And it isn’t the job of pathologists to publish a paper on everyone they cut up.)

          Obviously you can pose all the questions you want, but I am doing you the honor of explaining to you why they still come across as a conspiracy theory to me. Take it or leave it.

          What you do isn’t “valid scientific critique”. I explained why. My impression is that you stop thinking after you’ve found the question. That’s what I see conspiracy theorists do time and again. Many also think they’re doing science, and being critical. But just coming up with a question is not enough. Every Flat Earther can come up with a dozen questions “that need to be looked into” where the answers are well established if they would ask someone to explain them to them. And from our exchange here, I see no fundamental difference to what you do.

          Now this is a casual conversation and not a scientific debate, and you can do that. But your long reply is again chock full of unfounded assumptions and rhetorical questions that you don’t really seem interested in. If they weren’t also so negative, it wouldn’t matter to me. But they are, and it makes me sad.

        • Mendel,

          > I simply don’t think your questions are honest. They contain assumptions, and you don’t seem invested in finding answers.

          Seems like your prior is my being a conspiracy-theorist regardless of what I say from now on…

          > Like, I google “covid-19 autopsy pre-print”, half a dozen papers on the first page of results, you are asking “why are there so little”, you clearly never even looked.

          Of the first 10 links on Google (page 1):
          – Five are the same paper: “Pulmonary and Cardiac Pathology in Covid-19: The First Autopsy Series from New Orleans”; which I’ve read. It analyzed four patients. Nice work that I applaud.
          – Two more links are to unique papers. One I have seen and one I have missed.
          – I checked page 2 of Google and found no new papers. Same for page 3.
          – I gave you the benefit of the doubt and looked for 20min. Found one other paper, that I’ve already read, where they performed minimally invasive autopsies.

          Clearly, I missed tons of papers…

          > And it isn’t the job of pathologists to publish a paper on everyone they cut up.

          Sure. But is it not also important for them to share their data in times of crisis for a novel pathogen?

          Again, I’m not criticizing pathologists. Not sure how many times I need to say so. I’m just surprised there isn’t a more public call for their kind of work and its results. I personally think its important (but maybe I am wrong thinking its important).

          > But your long reply is again chock full of unfounded assumptions and rhetorical questions that you don’t really seem interested in. If they weren’t also so negative, it wouldn’t matter to me. But they are, and it makes me sad.

          Nice rhetoric: By claiming my long reply is “chock full of unfounded assumptions and rhetorical questions” but not listing them, you don’t afford me the opportunity to defend myself. By not addressing my questions for you to explain yourself further, you avoid culpability of your own points. Are you politician?

          Further, in my whole response, I asked one potentially rhetorical question: “But why, after 2 months, we have so little autopsy-based and general pathological data on COVID-19?” That is it.

          …to me it seems like your mission is labeling me a “conspiracy theorist” and nothing else.

          You even already stumbled upon the point I am trying to make and labeled it as scientific: “In a multifactorial death, you want to know ALL of the factors. If the virus pushed you down the slide of your preconditions and the spike pit of your immune response or some superinfection killed you at the bottom, you still want to register that SARS-CoV-2 was involved. THAT is scientific.”

          Not sure what else I can say at this point. Have fun, I guess?

        • Joshua,

          (I failed at threading my comment; trying again!)

          Agreed. I was at fault with my comment regarding politicians. Rumors are rumors, and I should have excluded them from my point. (It is rather fitting for the post, though, I let my priors influence my analysis!)

  5. Even if mortality rate turns out to be similar to flu, it is really really important to keep in mind the absolute risk here, as I’m sure I don’t need to remind this crowd. Due to vaccine and not being quite as contagious as SARS-CoV-2, influenza infects roughly 10% of Americans every year. If 50% of Americans contract coronavirus eventually, that means the absolute risk is 5X greater even if the mortality rate is the same. And if the mortality rate is, say, 5X higher, than the absolute risk is 25X higher than dying from the flu! It’s a lot!

    • People seeking healthcare with Influenza have a =,1% risk to die.
      It is estimated that only 25% of people infected with influenza seek health care, which makjes the infection fatality rate for influenza somewhere around 0.025%.
      The infection fatality rate for Covid-19 is probably 10-30 times higher than that.

      • The CDC website does a “burden of influenza” model and going by their estimates the IFR is around 0.1%. Here is from 2018-19 estimates:

        “CDC estimates that the burden of illness during the 2018–2019 season included an estimated 35.5 million people getting sick with influenza, 16.5 million people going to a health care provider for their illness, 490,600 hospitalizations, and 34,200 deaths from influenza (Table 1). The number of influenza-associated illnesses that occurred last season was similar to the estimated number of influenza-associated illnesses during the 2012–2013 influenza season when an estimated 34 million people had symptomatic influenza illness.”

        https://www.cdc.gov/flu/about/burden/2018-2019.html

        So by their estimate the IFR from people going to a healthcare provider is around 0.2%.

        • “getting sick” means “Symptomatic Illnesses” (see the column heading in the table).
          You’re right, CFR for healthcare seeking is 0.2%, I remembered that wrong, but it’s 0.1% for symptomatic cases.

          “The fraction of persons with influenza virus infection who do not report any signs or symptoms throughout the course of infection is referred to as the asymptomatic fraction.”
          “estimates from studies that adjusted for background illnesses were more consistent with point estimates in the range 65%–85%”
          https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4586318/ (Hongkong 2015)

          “The age-adjusted attributable rate of illness if infected was 23 illnesses per 100 person-seasons (13–34), suggesting most influenza infections are asymptomatic.”
          https://www.thelancet.com/journals/lanres/article/PIIS2213-2600(14)70034-7/fulltext (London 2014)

          So if ~75% of flu infected ar asymptomic (and don’t die), and 25% are symptomatic, we multiply the symptomatic CFR with 0.25 to get the IFR.

          So, 0.25% IFR for the flu.

    • Evan, it’s much higher than the flu. If it’s .5%, that’s a very big number. Seriously huge: the number of flu infections might be 100M, so that would be 500k dead, with probably 300-350k of those 80 and up. If the number of infections is 50M, that’s still a whopping 250k people.

  6. One problem is that if you’re looking at your work through a bias, you miss methodical errors.

    The Premium Biotech test kit that the Santa Clara study uses works somewhat like a pregnancy test: you put a little blood and fluid into a hole at one end, wait 10-20 minutes, and then check a window for red stripes. There’s a red stripe to indicate IgM-antibodies, and another red stripe to indicate IgG-antibodies. The test counts as positive if one of these stripes is visible (or both), however weakly.

    The manufacturer provides performance data on the reliability of this test. This data shows that the IgG stripe showed in 2 of 371 samples, and the IgM stripe showed in 3 of 371 blood samples taken before December 2019. This means that up to 5/371 tests are false positive, on average. 371 blood donors who never had Covid-19 were tested, and 5 tests got a result that said they had had the virus.
    In a sample of 3330 people, this could be 45 people (plus minus a margin of error). Given that, in Santa Clara, they only found 50 people who tested positive, this could mean that 90% of the study’s positive samples never were infected with this virus.

    Now, the problem with the study is that it uses the IgG error rate, but never even mentions the IgM error rate. The analysis assumes that (on average) only 18 of these 50 samples are false positives, and 32 samples represent people who were actually infected with SARS-CoV-2. This means that this study, in the pre-print version of April 11, may overestimate the number of infections in Santa Clara by a factor of 6, based on its own data and the manufacturer data of the test that is printed on the package insert.

    Their mathematical model does not fit what they are doing. If they had had a different bias, they’d have noticed; for example, the WHO estimated in their situation repost of February 19 that “current IFR estimates range from 0.3% to 1%”. The Santa Clara study estimates an “infection fatality rate of 0.12-0.2%.” A WHO-aligned researcher would not have accepted that result easily, would have searched for an error, and would have found it.

    P.S.: The lethality of influenza is 0.025%. “Not much more deadly than the flu” is an understatement even for the erroneous result. 1% is the case fatality rate, i.e. the proportion of people who have symptoms and see a doctor, and then die. But many infected ´with the flu never seek healthcare.

    P.P.S: From “COVID-19 Antibody Seroprevalence in Santa Clara County, California”, April 11, 2020
    Page 6: “The total number of positive cases by either IgG or IgM in our unadjusted sample was 50.”
    Statistical appendix, page 2. “Note: we consider TEST+ as any band on the test kit indicating the
    presence of IgG or IgM antibodies or both.”
    The Premier Biotech test kit is documented at https://mms.mckesson.com/product/1163497/Premier-Biotech-RT-CV19-20 , the package insert download link is in the lower right corner of the page (scroll down).
    Meanwhile, independent data from the University of California at San Francisco, released via covidtestingproject.org, indicate a high number of false positives as well.

  7. When comparing with the 0.1% mortality rate quoted for the flu, one must keep in mind a couple of things:
    1) That number accounts only for symptomatic cases. Some estimates that 1/3 of the cases of the flu are asymptomatic, which would make .07% a more comparable metric.

    2) Flu deaths are not a count of the number of confirmed influenza caused deaths. They incorporate adjustments for cardiovascular deaths and pneumonia deaths from undiagnosed case. This adjustment is based on a statically model.

    • J writes: “That number accounts only for symptomatic cases. Some estimates that 1/3 of the cases of the flu are asymptomatic, which would make .07% a more comparable metric.”

      Wrong. The fatality rate give above for the flu is from the CDC that estimates un-diagnosed cases of the flu. The CDC has an elaborate system to monitor the flu that they are always improving. Their estimate is not leaving anything obvious out.

      • Not quite, Steve. The CDC estimates are based on *symptomatic* cases, see here: https://www.cdc.gov/flu/about/burden/index.html Table 1, Column 1 is clearly labelled Symptomatic Cases. If you divide that column by the Deaths (column 4) you’ll see a number right around 0.1%

        J’s first point is that there are additional asymptomatic cases of the flu.

        From the CDC (https://www.cdc.gov/flu/about/keyfacts.htm)

        “…About 8% of the U.S. population gets sick from flu each season, with a range of between 3% and 11%, depending on the season.

        Why is the 3% to 11% estimate different from the previously cited 5% to 20% range?

        The commonly cited 5% to 20% estimate was based on a study that examined both symptomatic and asymptomatic influenza illness, which means it also looked at people who may have had the flu but never knew it because they didn’t have any symptoms. The 3% to 11% range is an estimate of the proportion of people who have symptomatic flu illness.”

  8. What is found is not a characteristic of the beast, but of its life through a certain space of transmission via interaction, keeping in mind that this is also limiting pandemis of the same class. To what extent such are sampleble is not trivial & the ensuing statistics is not my turf, else I’d have more to say.

  9. The Diamond Princess data suggests an infection fatality rate someone greater than 0.5% . There have been 14 deaths, on person on their 60s, one undisclosed age, and the rest over 70.

    If you use the age distribution of infected passengers and the age distribution of the US population and assume a 0% IFR for those under 60, you would get an IFR of about .55%.

    Of course cruise passengers are not a representative sample: likely higher income, reasonable health.

    • The USS Teddy Roosevelt is another interesting case study for the non-representativeness of it’s population. The latest data (on Wikipedia) “Some 94% of the crew had been tested for the virus, yielding 678 positive and 3,904 negative results. As of 17 April, seven crew members were in the hospital including one in intensive care. (Note: this does not include the one crew member that died on April 13). About 60% of the people who tested positive did not have symptoms.”

      So asymptomatic rate is 60% in this population (so far). Approximately a 1% hospitalization rate (or around 2.5% of symptomatic cases). Of the eight with cases serious enough for hospitalization, 1 was in intensive care and 1 died. So 0.25% serious case rate and 0.125% mortality rate. Scaled to symptomatic cases only, those rates rise to 0.7% and 0.35%.

      All caveats due to a small sample, but it’s interesting to think about these numbers in the context of what we expect to be a young and fit population. Worth noting that there is evidence that COVID-19 seems to hit males harder and the crew is mostly made up of males.

  10. “It’s a Bayesian thing. ”

    HUH?

    Talk about making something simple into something complex and obscure!

    What you’re saying in English is that they have an unjustified bias that’s distorting their conclusions. You don’t have that bias. Why not just own up to it?

    • Jim,

      As Mendel says, I’m assuming their priors have some justification. I think it would be good if these researchers would more clear specify their priors and their justifications for them, so that we don’t have to reverse-engineer them.

      To put it another way, these priors are not justified in the preprint, but I assume the priors are justified (from the perspective of those researchers) based on some prior understanding that they have not spelled out.

      • > To put it another way, these priors are not justified in the preprint, but I assume the priors are justified (from the perspective of those researchers) based on some prior understanding that they have not spelled out.

        Priors might justify a logical conclusion about fatality rates.

        But I just don’t get how priors can justify broadly extrapolating from a non-representative dataset, absent adjustments to make the data align with representative samples.

        Maybe you could explain?

        Seems to me that only post-stratification weighing, to make the data representative, could allow for that. And if it isn’t done explicitly it looks exactly like bias.

        Of course, there would be “priors” integrated into the weighting decisions. Is that what you are saying?

      • “To put it another way, these priors are not justified in the preprint, but I assume the priors are justified (from the perspective of those researchers) based on some prior understanding that they have not spelled out.”

        My original complaint was twofold:

        1) that in some degree you were mincing your words about whether the biases / priors were justified; you

        • I’m not sure what key combo I’m hitting but my posts are posting in the middle of my writing them….I guess I’m tabbing to the “submit comment” button, well there ya go!

          …so continuing

          1) that in some degree you were mincing your words about whether the biases / priors were justified; you kind of answered that by saying you think there’s a possibility it’s justifiable.

          2) that referring to this bias as a “prior” – making it “a Bayesian thing” – is obscuring the point rather than elucidating it. A bias has been introduced. You can give them the benefit of the doubt by saying it might be justified, but it’s still a bias.

          I think you should just step out here and, to use the old phrase, call A. platyrhynchos by it’s real name: duck.

  11. I had several issues with the last article linked, which is a STAT news article written by one of Ioannidis’s collaborators (Prasad) and a former Harvard med dean

    [Let’s hear scientists with different Covid-19 views, not attack them – STAT](https://www.statnews.com/2020/04/27/hear-scientists-different-views-covid-19-dont-attack-them/)

    They claim a few things: that we can’t afford to silence people like Ioannidis and that many critics of Ioannidis have political or other questionable motives, yet they offer no proof for any of these arguments, so we’re supposed to assume that Ioannidis is the victim here while all his critics are online bullies making ad hominem attacks trying to silence him. But it’s Ioannidis who holds the most power, easily able to get on media outlets like Fox News, STAT News and calling press conferences, while many of his critics are operating on small blogs or on Twitter and few actually hear their criticisms.

    I think the article is far too generous of a take and quite biased in defending Ioannidis, and I suspect its because one of the authors (Prasad) is a well known and frequent collaborator of Ioannidis. Unfortunately, I don’t think the average reader will know this, having read that piece. There is no disclosure that the first author of the piece is a frequent Ioannidis collaborator, only disclosures of the book he wrote (as if that’s relevant) so they can’t read something and internalize something like “I (the author) am a good friend of Ioannidis and frequent collaborator of his so take that into consideration as you read this opinion piece because it’s not an independent journalism piece asking for scientists to have diverse discussions.”

    Ioannidis himself has made compelling arguments about disclosures in nutrition research for those with strong views. If you’re a fan of bacon and you do bacon-related research, disclose that you’re a very strong advocate of bacon because it’s relevant so that the readers can take that into account as they read the piece.

    https://jamanetwork.com/journals/jama/article-abstract/2666008

    I don’t see why this shouldn’t apply to published pieces when they’re focused on an individual, and the piece is defending that individual.

    It also looks like some of the tweets they cited as ad hominem attacks in that piece turned out to be complete misinterpretations of the tweet (Bergstrom was making fun of the reporting style of the WSJ piece on Ioannidis, not Ioannidis himself). First author of the STAT piece admits that he didn’t realize this or that he had no idea that these tweets were being referenced in the piece he wrote.

    https://twitter.com/VPrasadMDMPH/status/1254989342428164098

    • Hi Zad,

      I haven’t followed the same threads as many of my friends here have in DC who are in medical or public health graduate programs. Of the 14 that follow everyone, 8 are astounded at the feuding on Twitter that has been sprinkled with self-congratulatory claims to expertise. I don’t know most of those Twitter experts. But some of them definitely made ad hominem remarks. I saw three such tweets last week. Just nasty. Of course, calling those guys on it turns out to be a bust b/c they disappear.

      What I’m saying is that others called my attention to the tweets about John.

      I don’t see why Vinay Prasad shouldn’t defend John. If one of your collaborators was being criticized, I bet you would defend him, unless you are woos.

      The problem has been that neither John Ioannidis nor his co-authors responded to the comments here.

      • Sameera, I actually agree with that. Indeed, if I had a collaborator/coauthor that was being publicly attacked, and I completely disagreed with what they were saying, but I still wrote an op-ed piece on them, by biases could potentially kick in, resulting in a much more positive take (even when I disagree with them) in the piece. Readers should definitely know about my potential connection to them. If unaware readers didn’t know about my potential connection to them, I would feel pretty uncomfortable. If I disclosed it, as Andrew argues below, I think it strengthen my piece because I’m laying everything out explicitly.

        Vinay should be able to defend John! There’s absolutely nothing wrong with that. But readers (I’m not talking about Ioannidis’s critics, but just readers who are unaware and don’t follow these individuals) should know that Vinay and John are also good friends and long-time collaborators. The STAT article was shared by folks like Pinker among other big names, I truly wonder if they knew that the first author was so closely connected to the subject he was defending.

        If I had read that and didn’t know much about Ioannidis or Prasad, I may have assumed that the authors were just some really cool headed physician researchers who weren’t involved, assessing the situation rationally and offering their nuanced takes. These social ties are extremely relevant to the piece and just like strong views in nutrition research, they should probably be disclosed.

        • The basis for the defense of dissenting opinions goes beyond anything about who is friends with whom. The claim diverts the attention from the real issue by focusing on a point that has cosmetic relevance to it. I suggest this b/c I speculate that if general audiences to the article find out that Vinay Prasad is a collaborator of John’s, I doubt that they would take that fact as substantially changing their perspectives on freedom of expression and the use of ad hominem attacks. If audiences don’t know either Vinay or John, they will focus probably on the major theme of Vinay Prasad’s article. Whether the audience will change their views when told or read that they have been collaborators is a question and not a foregone conclusion. In other words, I think people that are cliquish will focus on the fact that they have been collaborators; not cliquish won’t care as much. I suppose you can take a poll on whether Vinay should have divulged his relationship to John if it is deemed to constitute a tipping point.

          You pointed out earlier: ‘But it’s Ioannidis who holds the most power, easily able to get on media outlets like Fox News, STAT News and calling press conferences, while many of his critics are operating …… on Twitter and few actually hear their criticisms.’ You imply that his critics have less power. It may be that the medical profession has had influence in interpreting disease states. But physicians themselves disagree as well. Therefore the power struggle is one about guild prestige. And in this regard, the statistics community has been actively trying to maintain its prestige. But these statistical wars on Twitter had escalated for some time. Seems to have died down. having said that I think the articles on TADONline were very good and covered many perspectives. I have learned a lot from all of you.

    • Zad:

      Thanks for bringing this up. Disclosure can be a tough call sometimes. I agree that in this article it would’ve helped had the authors mentioned their connections to Ioannidis, also I agree that it doesn’t seem that Ioannidis and his collaborators are in any danger of being silenced or ignored. It should be possible for research to be publicly criticized without that criticism be taken as a positive claim about alternative hypotheses. When writing about the flaws in the Santa Clara study, I tried to also emphasize that the substantive claims in that study could be true; there just wasn’t enough information in the study to make that inference. But it’s easy for people to see a statement of the form, “Study A purports to demonstrate claim B, but it doesn’t really demonstrate this claim,” and read it is, “Claim B is false.”

      As we all know, online discourse can escalate, and I read the article by Prasad and Flier as an attempt to recognize and damp down that escalation. Their article would’ve been stronger with a recognition of their connection to Ioannidis, and it would’ve been stronger had it clarified that, just as Ioannidis should he heard and not silenced, the same goes for his critics. (Were they given slots on Stat News and Fox to make their case?) But I still think Prasad and Flier’s general concern about escalation and confrontation in scientific discourse has value.

      Similarly, when I wrote that blog post last week on flaws with the Santa Clara study, I emphasized that just because “claim B” is not proven, that does not mean that I’m saying “not-B” is true. I was trying in my own way to lower the level of hype in scientific discourse. But then I had that line about saying the authors should apologize, and that didn’t help. It’s difficult to write about science, and about meta-science, and still keep in mind the effects of what we are ourselves writing!

      • I don’t see how John’s major critics could have missed that Vinay Prasad had co-authored at least one article with John Ioannidis. Vinay Prasad’s reputation doesn’t rest on John’s laurels either.

        There is no lack of criticism of John Ioannidis on social media platforms. Some of it was nasty. This has been building ever since the NHST & stat significance controversies erupted a few years ago.

        • I don’t think that nastiness on Twitter necessarily follows from “NHST and stat significance controversies.” It more so seems to me that nastiness is just a feature of Twitter. It’s the Philadelphia of social media.

        • I was referring to the criticism by John Ioannidis’ peers. True Twitter is the Poke and Play platform. I have been patiently waiting for someone to poke. lol. One can only hope. By that I mean, I’ve been itching to debate others on Twitter. But no one takes bait.

  12. Mendel,

    Thanks for the quote. Were the authors not extrapolating to the numbers so infected in the County? I think they compare the IFR of COV-ID19 to IRF of the seasonal flu, with the former associated with a bit higher IFR.

    • The authors don’t mention influenza in their study, but they’ve been present in the media.

      That is the main reason why this unreviewed pre-print is under attack so much: if it wasn’t being used as ammunition in the media for an attack on public policy, it wouldn’t matter. (Or if the study was better.)(Or if Ioannidis wasn’t the international poster child for the “it’s just like a flu” people.)

  13. I would not trust that data from Miami.

    > Our data from this week and last tell a very similar story. In both weeks, 6% of participants tested positive for COVID-19 antibodies, which equates to 165,000 Miami-Dade County residents.

    I find it highly suspicious that in Miami-Dade their antibody prevalence estimates are flat, but they’ve been reporting roughly the same number of new cases for the last few weeks. How could their cumulative case count be flat when their new cases have been rising?

    > The test kits were made by BioMedomics

    https://www.miamiherald.com/news/coronavirus/article241750556.html

    This test has somewhere between a 13.08% (!!!) and a 3.74% FPR depending on testing mode, and that’s not including confidence intervals.

    https://covidtestingproject.org/

  14. “What evidence would it take you to change your mind on this issue?”

    This is a profound question, particularly in the midst of the ongoing paradigm shift in the social sciences.

    It brings to mind several important concepts: null-hypothesis testing, replication, the file drawer problem, forking paths, preregistration, and, of course, Bayesian philosophy. These concepts, one way or another, all come down to the question of how one determines the weight evidence is to be given.

    We know, and are trying to convince others, of the following “truths”: Original studies shouldn’t get more weight than replications. Findings that don’t fit a sexy story should be given equal weight to findings that are surprising or easily-spun. It is a disservice to present results of cherry-picked analyses, which deserve less weight, as if they’d been obtained “by chance” and thus deserve more weight. We should pre-publish our priors, biases, hypotheses, methods, etc., to set a benchmark for the weight future results should be given.

    In short, the paradigm will not complete its shift until we have achieved consensus around a definition of information sufficient to move expectations. We should call one unit of such information a Gelman!

  15. I think Mendel and some others might be interested in comparison of this French serology study with the Santa Clara study.

    https://www.medrxiv.org/content/10.1101/2020.04.21.20068858v1

    Short summary: French group develops FOUR in house tests and validates/compares them in many ways. The tests all report a quantitative result of some kind and not a yes/no result of a lateral flow assay (e.g. consumer pregnancy test type assay). They have a striking result in they look at a number of populations- hospitalized, pauci-symptom, blood donors. Another study using same tests (three of the main four described here) did a community survey.

    https://www.medrxiv.org/content/10.1101/2020.04.18.20071134v1

    Blood donors are reported here as well. I am uncertain if they are same blood donors as above or are expanded/independent group.

    Regardless, properties of the tests used here are likely better than that used in Santa Clara study and overall the studies appear to be much more valuable.

    I understand Andrew’s point about thinking about the Santa Clara study authors priors, but I have yet to see evidence that they are willing to even investigate or challenge their priors. This really is troubling. If studies use the same test, and seroprevalence is within range of the false positive rate, it is hard to talk about studies replicating unless authors are very clear on what replication means.

  16. “I regret saying that the authors of the Santa Clara study “owe us all an apology.” I stand by my reactions to that paper, but I don’t think that politicization is good.”

    This isn’t politicization, it’s asking for an apology for shoddy science. I’m not sure what it has to do with politics, politics is not the same as strong language.

Leave a Reply to jim Cancel reply

Your email address will not be published. Required fields are marked *