Skip to content

“How to be Curious Instead of Contrarian About COVID-19: Eight Data Science Lessons From Coronavirus Perspective”

Rex Douglass writes:

I direct the Machine Learning for Social Science Lab at the Center for Peace and Security Studies, UCSD. I’ve been struggling with how non-epidemiologists should contribute to COVID-19 questions right now, and I wrote a short piece that summarizes my thoughts.

8 data science suggestions

For people who want to use theories or models to make inferences or predictions in social science, Douglass offers the following eight suggestions:

1: Actually Care About the Answer to a Question

2: Pose a Question and Propose a Research Design that Can Answer It

3: Use Failures of Your Predictions to Revise your Model

4: Form Meaningful Prior Beliefs with a Thorough Literature Review

5: Don’t Form Strong Prior Beliefs Based on Cherry Picked Data

6: Be Specific and Concrete About Your Theory

7: Choose Enough Cases to Actually Test Your Theory

8: Convey Uncertainty with Specificity not Doublespeak

2 more suggestions from me

I’d like to augment Douglass’s list with two more items:

9: Recognize that social science models depend on context. Be clear on the assumptions of your models, and consider where and when they will fail.

10: Acknowledge internal anomalies (aspects of your theories that are internally incoherent) and external anomalies (examples when your data makes incorrect real-world predictions).

Both these new points are about recognizing and working with the limitations of your model. Some of this is captured in Douglass’s point 3 above (“Use Failures of Your Predictions to Revise your Model”). I’m going further, in point 9 urging people to consider the limitations of their models right away, without waiting for the failures; and in point 10 urging people to publicly report problems when they are found. Don’t just revise your model; also explore publicly what went wrong.


Douglass frames his general advice as a series of critiques of a couple of op-eds by a loud and ignorant contrarian, a law professor named Richard Epstein.

Law professors get lots of attention in this country, which I attribute to some combination of their good media connections, their ability to write clearly and persuasively and on deadline, and their habit and training of advocacy, of presenting one side of a case very strongly and with minimal qualifications.

Epstein’s op-eds are pretty silly and they hardly seem worth taking seriously, except as indicating flaws in our elite discourse. He publishes at the Hoover Institution, and I’m guessing the people in charge of the Hoover Institution feel that enough crappy left-wing stuff is being published by the news media every day, that they can’t see much harm in countering that with crappy right-wing stuff of their own. Or maybe it’s just no big deal. Stanford University publishing a poorly-sourced opinion piece is, from a scholarly perspective, a much more mild offense than what their Berkeley neighbor is doing with a professor who engages in omitting data or results such that the research is not accurately represented in the research record. If you’re well connected, elite institutions will let you get away with a lot.

When responding to criticism, Epstein seems like a more rude version of the cargo-cult scientists who we deal with all the time on this blog, people who lash out at you when you point out their mistakes. In this case, Epstein’s venue is not email oor twitter or even Perspectives on Psychological Science; it’s an interview in the New Yorker, where he issues the immortal words:

But, you want to come at me hard, I am going to come back harder at you. And then if I can’t jam my fingers down your throat, then I am not worth it. . . . But a little bit of respect.

Dude’s a street fighter. Those profs and journalists who prattle on about methodological terrorists, second-string replication police, Stasi, Carmelo, etc., they got nothing on this Richard Epstein guy.

In this case, though, we can thank Epstein for motivating Douglass’s thoughtful article.

P.S. I’d been saving the above image for the next time I wrote about Cass “Stasi” Sunstein. But a friend told me that people take umbrage at “sustained, constant criticism,” so maybe best not to post more about Sunstein for awhile. My friend was telling me to stop posting about Nate Silver, actually. It’s ok, there are 8 billion other people we can write about for awhile.


  1. Kyle C says:

    Paul Campos has a good post at Lawyers Guns & Money on Epstein as a manifestation of the phenomenon of law professor bs.

    • Steve says:

      Thanks for the recommendation. Here’s the link I’m embarrassed as a lawyer and NYU Law grad, but this kind of stuff has been going on now for several decades, where law professors venture into areas (namely economics) without the requisite expertise. The biggest problem with all of this is that lawyers understand how government works and how the political system works. Which means that lawyers have a crucial role to play in designing any governmental response to any crisis. But, if law professors believe that they can cut out the actual subject matter experts, it creates a real danger.

      • Well, lawyers deal with all kinds of cases that are deemed to compromise health and cause injury/death. I am not adverse to lawyers weighing though given that subject matter experts are not in agreement on the figures being bandied about. See John Ioannidis, Ste Vermond debate for example.

        I gather that each person is couched as an economic statistic. It is explained in that article by Cass Sunstein that I posted here.

        I use to follow Richard Posner’s Law and Economics blog. Quite interesting. Then there is Calabresi’s work too.

        The commodification of every aspect of our lives has devalued human life to an extent that must be addressed with great integrity and humanity.

        • Steve says:

          I’m a lawyer, and I also have no problem with people in various disciplines talking outside their narrow discipline, but it should be collaborative. (Frankly, economists ought to talk to lawyers a little more because legal rules make a big difference in how markets operate.) Epstein goes off on biology and evolution of viruses and just completely makes stuff up. He should feel like he put his job at issue, but I doubt he even feels like his reputation will take a hit.

        • Rahul says:


          Can you explain what you mean by commodification has devalued human life?

          If you are criticizing cold, ruthless descisions based on statistics, however repugnant that may be, as a policy planner isn’t that the only way out?!

          Sure that 0.1% mortality stat. is a lot more personal if it happens to be someone we love. But to a policy planner it must stay a statistic. One cannot get emotionally involved in these hard descisions.

    • Peter Dorman says:

      This will probably interest only the economists on this list, but here is something I posted about the Epstein (Richard) business:

      In the New Yorker interview RE invokes general equilibrium theory, but he has no idea (apparently) of the modern status of that theory and simply jettisons it when his audience shifts to an economist. His arguments are transparently disposable.

  2. Keith O’Rourke says:

    Not social science but public health, but is this co-operating group of researchers pretty much all 10 seem to be being adhered to. Especially number 10. A number of times I was getting annoyed with someone seeming to be a stick in the mud and then an important issue popped out.

    Anyway, slides are now available but the process may not be reflected in them. Other groups will be re-running the analyses on other data providers data and pointing out what they learned.


    Introduction – Daniel Prieto-Alhambra and Patrick Ryan (Slides)
    Literature Review – Jennifer Lane (22:00 • Slides)
    Data Network In Action – Kristin Kostka (26:10• Slides)
    Phenotype Development – Anna Ostropolets (31:38• Slides)
    Clinical Characterization of COVID-19 – Ed Burn (42:10 • Slides)
    The Journey Through Patient-Level Prediction – Peter Rijnbeek (50:12 • Slides)
    Prediction #1: Amongst Patients Presenting with COVID-19, Influenza, or Associated Symptoms, Who Are Most Likely to be Admitted to the Hospital in the Next 30 Days? – Jenna Reps (56:55 • Slides)
    Prediction #2: Amongst Patients at GP Presenting with Virus or Associated Symptoms with/without Pneumonia Who Are Sent Home, Who Are Most Likely to Require Hospitalization in the Next 30 Days? – Ross Williams (1:08:42 • Slides)
    Prediction #3: Amongst Patients Hospitalized with Pneumonia, Who Are Most Likely To Require Intensive Services or Die? – Aniek Markus (1:15:25 • Slides)
    Estimation #1: Hydroxychloroquine – Daniel Prieto-Alhambra (1:23:32 • Slides)
    Estimation #2: Safety of HIV/HepC Protease Inhibitors – Albert Prats (1:31:24 • Slides)
    Estimation #3: Association of Angiotensin Converting Enzyme (ACE) Inhibitors and Angiotensin II Receptor Blockers (ARB) on COVID Incidence and Complications – Daniel Morales (1:36:58 • Slides)
    #OpenData4COVID19 – Seng Chan You (1:45:32 • Slides)
    The Journey Ahead – Patrick Ryan (1:50:28 • Slides)
    Questions & Answers – Daniel Prieto-Alhambra, Peter Rijnbeek and Patrick Ryan (2:08:15)

  3. Anoneuoid says:


    1) Shows cases/deaths without the number of tests
    2) Cites percent of hospitalizations by age group without mentioning the percent of the population that makes up each age group
    3) Considers smoking a possible risk factor for total deaths when smokers are appearing at 1/3 the expected rate (now reported in many studies in China and at least one in the US).

    • Anoneuoid says:

      CDC reports that out of 12,217 COVID-19 patients with data available only 96 (1.3%) current smokers and 165 (2.3% former smokers).

      Apparently since so few outcomes were reported, they cannot draw any conclusions:

      Finally, for some underlying health conditions and risk factors, including neurologic disorders, chronic liver disease, being a current smoker, and pregnancy, few severe outcomes were reported; therefore, conclusions cannot be drawn about the risk for severe COVID-19 among persons in these groups.

      • Anoneuoid says:

        Typo, I grabbed the wrong denominator from the table, it is 7162.

        If they hadn’t covered this up after the first SARS we would probably have no problem at all right now.

  4. Why is it that we hadn’t tackled hospital-related infections which takes an estimate of 100,000 lives annually? For years this statistic has surfaced. And points to the need for strong measures to address them.

    Given the #s of deaths [numerator] and the totals of infected estimated [denomerator], the death rate is still below 1% and not the 3.4 WHO estimate that was bandied originally. Similarly, I saw a prediction for 500,000 cases in the UK. Then it is cast as a worst-case scenario. But now they have revised that to around 20,000 cases. Or was the prediction about deaths.

    • Anoneuoid says:

      It is looking more and more like we are going to see a huge drop in all cause mortality in the coming months as people stay away from dangerous activities like going to the hospital.

      • Anoneuoid sounds very plausible.

        We need more granular post-mortem follow-up data > to glean the actual causes of death in the hospital especially now. I wonder how well pathology reports/autopsies are documented. I recall reading that they are not so well documented allegedly.

        I think social distancing is prudent for now. But I have been observing that people are not so mindful of the surfaces that they touch. In other words, it is near impossible to avoid touching hard surfaces which can constitute potential points of transmission.

        Moreover, I am concerned that there may be some efforts to fast track prescriptions & procedures.

    • Steve says:

      The UK model predicted 500,000 deaths. It was not “revised.” The prediction of 20,000 deaths is just what would happen under assumptions that social distancing was effective. The 500,000 death estimate was what happens assuming Boris Johnson herd immunity theory (“do nothing”) approach was implemented.

      The 100,000 healthcare related number is an estimate that is highly assumption specific, and includes any death where the infection is deemed to contribute to the death. When people are dying, there immune system breaks down, and viruses and bacteria that we normally carry around with no consequence start to affect the very ill. I’m not minimizing the issue. We should be doing a lot more, but it is very different issue than the current virus. It is not a simple issue to solve. If we make our hospitals super clean, they become a breading ground for super bugs. In addition, it isn’t clear that a person dying who picks up say a Staph infection, which is a bacteria you have on your skin, and develops a serious infection that hastens their death would not have had the same outcome outside a hospital. So, the 100,000 just has to be understood in context.

      • Anoneuoid says:

        When people are dying, there immune system breaks down, and viruses and bacteria that we normally carry around with no consequence start to affect the very ill. I’m not minimizing the issue. We should be doing a lot more, but it is very different issue than the current virus.

        This doesn’t sound very different at all. It sounds like exactly what is happening.

        • Steve says:

          No, I think it is very different. There are people dying now, who were not expected to die at all. They may have co-morbidities, like hypertension, but that is a far cry from a 90 year old Alzheimer’s patient who gets pneumonia from a bacteria that she was immune to her entire life. We don’t know the numbers, but we have enough anecdotal information to realize that a lot of previously relatively healthy people are becoming hospitalized, going to ICU and some are dying. To get the 100,000 HAI deaths, those researchers reclassified deaths not originally classified as deaths due to infections at all. So, the 100,000 will include patients who had an infection, might have recovered, but the researchers nonetheless, believed it contributed to their decline. Nothing wrong with that. They are trying to figure out the impact of healthcare acquired infections. But, we know a large number of that 100,000 were thought to have died of another cause, whereas the deaths we are seeing labelled COVID-19 deaths, are labelled that way specifically because an attending physician believed that COVID-19 directly contributed to the death. That can be a big difference.

          • Anoneuoid says:

            There are people dying now, who were not expected to die at all.
            We don’t know the numbers, but we have enough anecdotal information to realize that a lot of previously relatively healthy people are becoming hospitalized, going to ICU and some are dying.

            Here is the age distribution of ICU patients in Manitoba, Canada from 1999-2007:

            There were always younger people ending up in the ICU due to unexpected problems. The media deciding to focus on them is not convincing evidence.

            • I’m going to guess that a lot of younger people in the ICU were involved in car accidents or skiing accidents or rock climbing accidents or doing stupid stunts to impress their friends or whatever. I personally know two people involved in serious car accidents who wound up barely surviving and hospitalized in ICU for weeks to months. One had a severed spinal cord, one was dead on arrival but revived and had hypoxic brain damage. One of them was a high school student, the other mid to late 20s.

              Those are all terrible things to happen, but I don’t think they inform much about respiratory infections.

              • Anoneuoid says:

                I was going to add some anecdotes too but then decided against it. Anyway I have a friend in his who had a relatively minor burn injury that got infected with MRSA and then almost needed to have his arm amputated. I guess someone gave him tea tree oil to put on the wound and that cured it.

              • And sure, anecdotes are just anecdotes, it’s better to have comprehensive data, but basically I know no-one actually hospitalized for an infection. Fortunately!

                It might be useful for someone to go through the mortality data from the CDC and produce age-specific graphs of mortality rates for non-accidental mortality through time for the US, and compare to age specific mortality non-accidents in the last 3 weeks

                does anyone have an age specific COVID mortality data source?

                Also does anyone know a historic hospitalizations data source?

            • Steve says:

              I am missing your point. I was talking about the age of people dying from COVID-19 and you are talking about the age distribution of people going to the ICU from years before COVID-19. Here is the age related CFR for COVID-19.
     The death rate for the cohort between 50-59 is 1.3%. That’s three times the normal mortality rate in that age group.

              • Again, you have to be careful there. Mortality rate is *deaths per person per YEAR* whereas 1.3% CFR is deaths per infected person. They don’t have the same dimensions.

                If you assume the death takes say 2 weeks from infection to death (or 3? whatever) then comparing these two we have

                (2/2) / (1/52) = 52

                so the CFR represents a ~52x increase in mortality rate.

              • Anoneuoid says:

                The point was its mostly the elderly/fragile people dying with a very few young people the media started focusing on once they realized people weren’t scared enough. This is something that is going on all the time:

                > The comparison of winter mortality (December-February) shows in the winter 2019-2020 a low mortality attributable to the reduced impact of seasonal risk factors (low temperatures and flu epidemics). This phenomenon, already observed in previous years (Michelozzi et al. 2016) have the effect of determining an increase in the pool of more fragile subjects (elderly and with chronic diseases) which can increase the impact of the COVID-19 epidemic on the mortality and explain, at least in part, the greater lethality observed in our country


                Once this is over I’m going to cherry pick cities where mortality rate in the elderly is 30% higher than average for the last month and post them here like Carlos Ungil does.

    • Carlos Ungil says:

      > Why is it that we hadn’t tackled hospital-related infections which takes an estimate of 100,000 lives annually?

      I think there have been some improvements in that front lately. But I’m not sure what has it do to do with anything.

      A rough translation of that figure into NYC terms would be around 2000 deaths per year, 6 deaths per day.

      Average all-cause mortality is around 54000 deaths per year, 150 deaths par day.

      There have been 142 reported covid-19 deaths in the last 24 hours. 932 since March 14.

      • Sam says:

        Hi Carlos,

        Wouldn’t we need to conduct a post-mortem followup to conclude that the COVID19 is the actual cause of death? The elderly have more health-related conditions that could have already compromised the immune system. The Italians it is suggested have done a more helpful categorization of the individual with compromised immune systems and who have also been tested for COVID19.

        The 100,000 figure is quite recent. I will double-check. What I’m suggesting is that nearly every health care worker, including physicians, that I have come across dread the high rate of infection. And caution that avoiding the hospital, unless a dire emergency, is wise.

        • jim says:

          “And caution that avoiding the hospital, unless a dire emergency, is wise”

          That’s always good advice.

          But I’d bet it’s far more likely for elderly people to come to the hospital without COVID19 and contract it at the hospital and die then it is for them to arrive with COVID19 and die of something else. =

        • Carlos Ungil says:

          Between 9:30 and 16:30, 164 additional COVID-19 deaths were reported in NYC (182 in 24 hours).

          Yes, unhealthy people are more at risk. Of course, that’s even more true for the hospital-acquired infections you brought up for some reason, as all the victims are unhealthy enough to be find themselves in a hospital in the first place.

          Yes, determining the “actual” cause of deaths can be tricky. If an AIDS patient dies of a cancer or an opportunistic infection, what is the actual cause of death? Does anyone die due to HIV infection?

      • My response to Carlos is in moderation. Eeekkk

    • Zhou Fang says:

      Sameera Daniels:

      A lot has been done to tackle hospital related infections. New antibiotics, enhanced cleaning procedures, so on and so forth. It is unfortunately not clear what exactly more can be done.

      Steve has explained your misunderstanding about the 20,000 case number. I’d add the death rate is not just dividing deaths by infected, because there is a lag between one and the other. Given it takes 2 or more weeks to go from symptom onset to death, taking deaths so far and dividing by cases in a situation where case totals are growing by 20% each day can underestimate the case specific death rate by a factor of 10.

        • Thank you Zhou,

          just to say that nurses and doctors off record do raise the problem of hospital infections. I saw one article a couple of years ago that expressly pointed to the insufficiencies of sanitizing practices. A real concern due to antibiotic-resistant bacterial infections. MRSA, for example, has been hard to extinguish. But the more salient point is that we need the pathology/autopsy reports I would speculate to be sure of causes and effects.

          I am not suggesting nor know what the extent of progress is really.

      • Steve says:

        +1 I would also add that the obsession with the CFR is clouding the fact that this virus is putting health people in the ICU. We have some scary reports out of China of large numbers of lung transplants being done. We don’t know what the sequelae for COVID-19 will be. There may be millions of people spending the rest of their lives with scarred lungs and compromised respiratory systems even if the CFR is lower than we now expect. Stay inside.

        • Speaking of sequelae at one point I posted about lung scarring and bone necrosis. Someone counter posted calling that “twitter level stuff” if I remember. My wife, who studies bone development and healing, thought the bone necrosis issue was interesting. But neither of us could read the chinese papers it was based on (and linked via wikipedia on the article on SARS).

          So she assigned one of her students to do a lit review. The student found that tibial necrosis was a real thing post SARS, but that it looks like it was probably caused by high dose steroids to treat inflammation in the lungs. So, it’s the treatment not the disease in this case.

          In any case, the long term issues for someone say age 30 who gets a major case of COVID and spends weeks on a ventilator and “recovers” are unknown but likely to be serious in at least some of those patients, and just the couple weeks on a ventilator are serious themselves, let’s not even mention the completely ruinous bills coming out of that.

          We need to widen the conversation beyond fatalities to include hospitalizations and the duration and severity of hospitalizations. Until just a few days ago hospitalization wasn’t even a column in the dataset for example.

          • Zhou Fang says:

            Hospitalisation is really difficult to present in a way that is generalisable amongst different countries though.

            Testing is also similar, there’s multiple different types of tests (PCR vs antibody tests for instance) being done, and if you look at contact tracing testing vs ER testing vs drive through testing vs home test kits vs travel screening testing vs testing of people in recovery, all of those will have different mechanics, error rates, and targetting. Someone really should try and collate what different countries’ test regimes actually are, because I find the standard comparisons useless without that.)

            • True enough. One thing this whole event is showing up is how poor the sources of health data are. I’m sure we could learn a LOT about how to keep people healthier and reduce healthcare costs if we had sufficiently detailed public datasets about health care utilization world-wide.

          • Re: We need to widen the conversation beyond fatalities to include hospitalizations and the duration and severity of hospitalizations. Until just a few days ago hospitalization wasn’t even a column in the dataset for example.

            Excellent Daniel

          • Steve says:

            +1 Part of the problem was that terrible WHO report that put hospitalizations in the “Mild” group. So, 80% of the cases in China were “mild.” They should have known that would be reported uncritically in the press. I keep arguing with people who say that COVID-19 is only hitting the vulnerable. Well, look at the list of co-morbidities. In this study,, the risk of reaching the composite endpoint (which include ICU admission and death) went up with co-morbidities, but look at the most important commorbidities — hypertension, cardiovascular disease, diabetes. Luckily, these conditions aren’t a problem in Western countries like they are in China. It would be nice if this fact had been communicated early and often.

            • Martha (Smith) says:

              “Luckily, these conditions aren’t a problem in Western countries like they are in China.”

              Please clarify what you mean by “aren’t a problem in Western countries like they are in China” — the phrase seems subject to various interpretations.

  5. AV says:

    The notion that tests are to qualify models, not id cases, is something worthy of Reuters.

    • Phil says:

      I don’t know what this comment means.

      As Daniel Lakeland pointed out a few posts back, there would be a huge benefit to testing to know the prevalence of the virus in the population even without identifying who is sick. That’s clearly true. Are you saying that idea should get more news coverage?

  6. Bob says:

    While you’re not posting about Sunstein, Stuart Ritchie is, it’s quite good.

    What’s also funny is tht his twitter feed is full of butt hurt clinical psychologists.

    • Andrew says:


      I followed the link, and I agree that Ritchie’s article is excellent. (I’d avoid using the rude term “butt hurt,” though.)

      I did know about Ritchie’s writing about Sunstein; indeed one of my earlier Sunstein-themed posts was motivated by something that Ritche had posted that someone pointed me to.

    • jim says:

      I like this:

      “Sunstein focused on the third one, which included 156 participants, all of whom were undergraduate students reasoning about how much they’d pay to avoid an imaginary electric shock.”

      Yeah, wow, if I wanted to know anything about society in general, a group of undergrads certainly would be the appropriate sample. But at least N > 5, so significant!

      But strangely: “It’s not a criticism of the scientists to say that this experiment is only tenuously relevant to a global pandemic.”

      Well, that’s true, but lets admit that doing research on undergrads has pretty much no relevance to adult society under any circumstances, so unless you’re only going to apply your results to undergrads, don’t waste people’s time. (the upside is it probably doesn’t waste much money since undergrads are cheap research subjects)

      • Martha (Smith) says:

        “but lets admit that doing research on undergrads has pretty much no relevance to adult society under any circumstances”

        “Adult”is often considered to be “Age 18 or greater,” so technically, your claim is not true. However, a sample chosen from undergrads is not representative of adult society in general, so it is accurate to say, “A sample chosen from undergrads cannot give inferences that apply to adult society in general”.

  7. Nick Adams says:

    1. I have a few lawyer friends. They all have the naive belief that they can alter reality by argument and persuasion. I guess it’s a trait acquired by occupation.
    2. Hospital-acquired infections are real but difficult to enumerate. We try hard to prevent them but if you have a bunch of sick people crowded together they’re gonna happen.
    3. Permanent lung damage is very uncommon after pneumonia. I would not expect “scarring of the lungs” to be a sequelae of covid19.

    • Thanks Nick,

      Re: #1. They are having to do that in lawsuits brought against private and public enterprises.

      #2. This has to be teased out even more. They shouldn’t lead to fatalities of that scope.

    • Steve says:

      Hong Kong doctors have already reported that some recovered COVID-19 patients have reduced lung capacity. So, there’s that. Also, patients who recovered from SARs-1 are documented to have had permanent lung damage. So, I think we should anticipate that risk until we can rule it out. (I shouldn’t use the term scarring of the lungs as I am no MD, but the experiences.)

    • Martha (Smith) says:

      Re #1: it could also be that people with the trait are attracted to the profession of lawyer.

    • Jon says:

      How true is #3. This probably depends on the type and severity. Furthermore, we all have a lot of excess capacity in our youth. Some residual permanent damage may not matter now, but could 30 years from now. From personal experience. I have allergies & after a bad cold in grad school for several years after I would get asthma attacks in very cold weather. I live a normal life, do long hikes and 5K runs; but spirometry shows somewhat below normal lung capacity and some chest xrays show “hyperinflated lungs” (I never smoked). Probably some permanent damage from the allergies and infections.

  8. Jon says:

    I have to fault even some people in the profession–Ioannidis, Bendavid, and Bhattacharya for committing some of the same sins they criticize in others. I think a wealth of evidence, especially the load on hospitals, indicates this is something different from the flu. They add their own misleading numbers, such as extrapolating from one small town (Vo) to a Provence or referencing death rates in Germany when almost all of the reported cases are too new to generate deaths. Further, they fail to acknowledge their numbers are also highly flawed.

    While they may really intend to be advocating for more data, a position I agree with; I fear their pieces are more likely to be interpreted as a call to inaction. I say this as a financial analytical professional who frequently has to accept the prospect of guiding decisions with incomplete and flawed data.

    • Brent Hutto says:

      I find it useful to consider Ioannidis, et. al. as having a strong prior (i.e. that historical examples of severe flu pandemics are the most useful analogue to the COVD-19 pandemic) and that the available COVD-19 data isn’t sufficient to provide much of an update.


      I know it isn’t exactly the argument you’re making but your take could be construed as implying that making highly alarmist extrapolations from absent or flawed data is necessary while making very optimistic extrapolations from the same near-non-existent data is unacceptable.

      That seems to be the default response of those in positions of authority or “expertise” in this situation. Since we can’t possibly know how bad it will turn out then it’s best to act as though it will be so bad that every measure we take in response is justified regardless of cost or side effects. A know-nothing stance in pursuit of saving the world is no more justifiable than a know-nothing stance in favor of ignoring a pandemic. And in meaningful terms, we know virtually nothing about the epidemiology of this virus.

      • Andrew says:


        Well put about the prior. I don’t think Ioannidis said it that way, but I think you’re right.

        • Brent Hutto says:

          I don’t have the training or the incentive to do much Bayesian analysis but it’s pretty fundamentally how I process the world when I look around. When we first heard of the coronavirus being “a thing” my wife and I had a conversation that basically went like this.

          “Sounds like this is going to be the mother of all flu epidemics, wonder if we’re taking about a 1918-level event.”

          “I don’t know, people seem to think this one is very different. Could be 1918 or worse, there doesn’t seem to be a vaccine”

          “Well I guess we’ll find out in the next few months.”

          Now it’s a few months later and we’re still pretty much in the “guess we’ll find out…” mode. Looking at all the highly speculative models out there, the range of extrapolations seem to be from about the 1957/1968 type level up to something akin in current population terms to 1918.

          From my perspective, I really don’t see any data to support predictions that fall much outside that range. And I don’t yet see any data to let us narrow it down in between those extremes. It a disaster any way you look at it but a 1918-and-beyond disaster and an 1957/1968 disaster are pretty different in their implications.

          • Brent, it doesn’t take much modeling to see that this pandemic is worse than 1918. Specifically two things, first it grows doubling every 3 days initially, and it’s clear that can continue until say 30% of everyone has it at least. How long would that take? About 90 days. In the last 3 days of that period 15% of everyone would have just gotten the disease.

            Now what’s the consequence of 15% of the U.S. For example, having the disease at once? Clearly about 10% of these people would have a bad form, so we are talking 1.5% of U.S. Very sick. That’s about 5M people. Suppose just 10% of them die, leaving an optimistic estimate of about 1% CFR that is 500k deaths in a ~2wk period. About the same as all of the years of the civil war put together.

            It’s obvious we can’t let that happen. The problem is, with a thing infectious enough to double every 3 days, there’s little else you can do other than extreme social distancing/isolation. So the choice is apocalyptic levels of death, or strong isolation. There’s no real inbetween.

            None of this takes fancy models, it just takes seeing the growth rate, and the order of magnitude of CAR ~ 1%.

            The flu doesn’t double every 3 days… And 1918 had similar 1% order of magnitude CFR, so COVID is obviously worse just for the speed with which it overwhelms everything.

            • Brent Hutto says:

              I believe the 1918 pandemic killed considerably more than 1% of the world’s population. Not 1% of cases.

              To the best of my knowledge COVD-19 has yet to kill even anywhere near 1% of the population in any area larger than a nursing home or small community.

              Nobody knows for sure but it’s safe bet that eventually COVD-19 will be endemic throughout the world. And we can also be almost certain that a million or more people will eventually die after being infected. But that’s a far cry from 30-50 million deaths and even farther from proportional impact of 30-50 million deaths in a world population of 1.5 billion.

              There’s enough to worry about with what’s happening, until we start getting (if we ever start getting) actual data on communicability, case mortality and rates of asymptomatic infection then there’s nothing to be gained by simply assuming away the entire history of human viral disease and declaring this particular virus sui generis.

              • Well obviously the 1918 pandemic went on for about 3 years in an era where they didn’t even have basic antibiotics. It killed ~ 50M in an era where population was about 1.6B so ~ 3% of population. So thats “order of magnitude” of 1%. Evidence from the pandemic so far in places like China is that it’s the same order of magnitude for this virus.

                Today, one hopes that our technology would have made CFR of the 1918 pandemic far lower than it was back then (for example, we could effectively fight secondary bacterial infections now, and most people seem to have died of secondary infections back then).

                So, apples to oranges, but still the basic order of magnitude of fatality rate is the same.


                case fatality rate has a lot to do not just with the virus, but also with the population it affects and the rate at which it attacks and the availability of resources and etc.

                On the other hand, 1918 flu didn’t have anything like the advantages in spreading that COVID-19 does… airplanes, cars, much higher density cities, etc.

                This virus isn’t “sui generis” it’s totally in line with what’s been expected for decades for a pandemic. It clearly spreads during asymptomatic periods, and with asymptomatic populations. This kind of thing was predicted pretty accurately a decade ago or more, it was even in TED talks of all places.


                That it grows at doubling every 3 days or so is pretty clear from data around the world:


                Confirmed deaths don’t have the same problems as tested cases… so growth rate estimates from them should be relatively good.

                That’s taking advantage of all the modern world’s mechanisms for spreading. A pandemic isn’t just a product of a virus, it’s a product of a virus in an environment. We’ve never in the history of the world had an environment where a person could be in Bangkok one day and less than 24 hours later in London and spreading a virus for another 2 days before symptoms.

                Left alone, with everyone acting robotically as if things were totally normal, we’d absolutely expect half the globe to have this by June or so, and CFR in the 5 to 10% range due to the tremendous overwhelming that would cause. In reality panic and riots would be a huge problem as well.

                It’s TOTALLY CLEAR that what we’re doing is appropriate levels of containment. What’s not clear is how well we’re utilizing the time we’re buying. We need drug and vaccine research and NOW. We need models that inform us about control techniques. We need a lot of stuff.

              • Carlos Ungil says:

                It is estimated that it has killed 0.4% of the population of the province of Bergamo last month (not a small community, with a population over 1.1mn). And it’s far from over, unlike the 1918 pandemic (which didn’t finish in one month either).

              • Martha (Smith) says:

                “Well obviously the 1918 pandemic went on for about 3 years in an era where they didn’t even have basic antibiotics.”

                My Great Aunt got the flu then, and survived. The doctor prescribed bed rest and chicken soup. (Her eldest daughter, then aged 12 or 13, knew how to make the soup, but hadn’t killed a chicken before. Fortunately, a neighbor intervened and killed the chicken.)

      • Jon says:

        To clarify: Ioannidis and others are comparing COVID-19 with the seasonal flu, not the flu pandemics. They seem to be finding the most extreme statements made (and sometimes out of context) to attack. There is plenty published that is much more measured and generally supportive of the current set of lockdowns.

      • I had been introduced to Cass Sunstein when he spoke at the American Enterprise Institute [AEI] I had just arrived in DC. Prior to hearing his AEI talk, I had read his books about group dynamics, which were fascinating. I think if you read the Laws of Fear, you will appreciate the major point that he makes: Application of the Precautionary Principle is rife with unhelpful heuristics. He didn’t go on to develop an in-depth cost //benefits approach that he has favored in some environmental and public health crises. It is was a good start.

        I haven’t followed his most recent perspectives much.

        I am with Richard Posner in his call for greater accountability in academia and policy wonks in general, which he covers in his book Public Intellectuals, A Case of Decline. He is more my idea of a incisive thinker. A Renaissance thinker. I don’t necessarily agree with all his views. But I admire the originality poised in his analysis of the law.

    • Carlos Ungil says:

      Ioannidis’ piece in StatNews is not aging well… “A fiasco in the making” indeed.

      “Some worry that the 68 deaths from Covid-19 in the U.S. as of March 16 will increase exponentially to 680, 6,800, 68,000, 680,000 … along with similar catastrophic patterns around the globe. Is that a realistic scenario, or bad science fiction? How can we tell at what point such a curve might stop?”

      One week later (March 24) there were already 681 deaths. 6800 will be reached this week. It’s hard to imagine how the curve might stop short of 68000.

      The following comment from the editor of The Lancet is about the UK but applies equally to most countries:

      “The NHS has been wholly unprepared for this pandemic. It’s impossible to understand why. Based on their modelling of the Wuhan outbreak of COVID-19, Joseph Wu and his colleagues wrote in The Lancet on Jan 31, 2020: “On the present trajectory, 2019-nCoV could be about to become a global epidemic…for health protection within China and internationally… preparedness plans should be readied for deployment at short notice, including securing supply chains of pharmaceuticals, personal protective equipment, hospital supplies, and the necessary human resources to deal with the consequences of a global outbreak of this magnitude.”

      He also published a comment in The Guardian two weeks ago (just one day after Ioannidis article):

      “After weeks of inaction, the government announced a sudden U-turn on Monday, declaring that new modelling by scientists at Imperial College had convinced them to change their initial plans. Many journalists, led by the BBC, reported that “the science had changed” and so the government had responded accordingly. But this interpretation of events is wrong. The science has been the same since January. What changed is that government advisers at last understood what had really taken place in China. […] The UK’s best scientists have known since that first report from China that Covid-19 was a lethal illness. Yet they did too little, too late.”

  9. Justin says:

    4: Form Meaningful Prior Beliefs with a Thorough Literature Review
    5: Don’t Form Strong Prior Beliefs Based on Cherry Picked Data

    10. Don’t conflate “prior beliefs” with “Bayesian priors” (since we can obtain and use background knowledge all sorts of ways other than putting possible subjective brittle probability distributions on parameters that most likely get swamped by the likelihood/data),


    • Andrew says:


      Yes, the whole “subjective/objective” distinction is unhelpful; see here. Prior information (I’d prefer the term “information” or “assumptions,” rather than “beliefs”) comes into analyses in all sorts of ways, a point that I discussed in section 3 of this paper, which in turn came from a discussion on the blog.

Leave a Reply

Best CBD Oil for Pain (2021)

CBD Gummies

CBD Oil for Kids (2021)

Best CBD Oil for Anxiety (2021)