2 reasons why the CDC and WHO were getting things wrong: (1) It takes so much more evidence to correct a mistaken claim than to establish it in the first place; (2) The implicit goal of much of the public health apparatus is to serve the health care delivery system.

Peter Dorman points to an op-ed by Zeynep Tufekci and writes:

This is a high profile piece in the NY Times on why the CDC and WHO have been so resistant to the evidence for aerosol transmission. What makes it relevant is the discussion of two interacting methodological tics, the minimization of Type I error stuff that excludes the accumulation of data that lie below some arbitrary cutoff and the biased application of this standard to work that challenges the received wisdom. It takes so much more evidence to correct a mistaken claim than to establish it in the first place.

I still suspect there is an additional factor: the implicit goal of much of the public health apparatus to serve the health care delivery system. One reason mask use was discouraged at the beginning of the pandemic was to protect the supply of N-95s to health care practitioners. Travel bans were opposed, since international travel and the movement of supplies were regarded as necessary for organizing and administering care, especially in developing countries. Recognition of aerosol transmission would have required the costly air filtration systems used in infectious disease wards to be installed throughout all hospitals and clinics (at least, as I understand it, under current protocols). This also helps explain the prominence given to hospitalization and ICU use as morbidity metrics, and the delayed recognition of long Covid as a health concern in its own right. If the primary constituency, and source of funding and personnel, for the public health apparatus is the medical system, it makes sense that, under conditions of uncertainty, the needs of medical practitioners would take precedence over those of the public at large. You can always justify it by saying we need the hospitals to be well stocked, doctors to travel freely, demand to be well below capacity and operations to not be bogged down by complicated protocols so the public can be better served. This is also relevant to the blog because there has been a sort of slipperiness about what outcomes constitute the costs and benefits that go into decision making under uncertainty.

The pandemic has been a spectacular laboratory for exploring the interconnections between science, the rules of evidence, risk communication, institutional incentives and political pressures. Some probing books about this will probably appear over the coming years.

Interesting points. I hadn’t thought of it that way, but it makes sense. I guess that similar things could be said about education, the criminal justice system, the transportation system, and lots of other parts of society that have dominant stakeholders. Even when these systems have had serious failures, we go through them when trying to implement improvements.

38 thoughts on “2 reasons why the CDC and WHO were getting things wrong: (1) It takes so much more evidence to correct a mistaken claim than to establish it in the first place; (2) The implicit goal of much of the public health apparatus is to serve the health care delivery system.

  1. I wholeheartedly agree with the first point – that it is so much easier (evidence-wise) to establish a false claim than to debunk it. The second, that the CDC and WHO aim to serve the health care sector is more questionable. In one sense, it should be true by definition – that is their mandate, after all. And I am not concerned that they are focused on the needs of the health care delivery system because somebody needs to be. But Peter may be suggesting this is a form of “regulatory capture,” where these agencies deviate from their stated purpose to support the needs of the industry they are designed to regulate (although not really in the sense of most regulatory agencies). I think that is a stretch – at least, I’m not sure the COVID responses are good evidence of some form of regulatory capture.

    I think many people will respond to the actions of the CDC and WHO based on their prevailing beliefs concerning who these agencies serve. If you believe they genuinely serve public health goals, then most of their COVID actions will seem appropriate (although subject to errors – the type that we all make in stressful times with great uncertainties). But if you believe these agencies are mainly protecting the health providers, then most of their actions will seem suspect at best. You can see this played out in the recent discussions about whether the CDC has been unduly alarmist or not. Very little evidence for very strong opinions.

  2. I agree with the basic point about the public health apparatus, but I think it is just part of human nature: when you are trying to figure out what to do, the people you know, or who are like the people you know, loom larger in the mind.

    • From the beginning it seemed that the FDA was strongly oriented to clinical needs and negligent in attending to epidemiological needs. (For example, lack of consideration of pooled testing, a topic of considerable discussion on this blog last year.) Not surprising as a primary mission is approval of drugs for clinical purposes.

  3. A simpler, more parsimonious, explanation is that the CDC is a relatively incompetent and surprisingly political organization. The former leads to lots of mistakes and the later an inability to admit them.

    To illustrate: Their “admission” of aerosols, while correct, isn’t even a net positive as it renders the advice on cloth mask as meaningful PPE incoherent. But backtracking on that is far too political so deferred.

    • I disagree about the uselessness of cloth masks, for a reason I’ve mentioned before but I’ll say again here.

      If you are having a conversation with someone, at normal conversational distance in fairly still air indoors, you are inhaling quite a bit of air that they just exhaled, and vice versa. Hold your hand in front of your mouth and say “Peter Piper”; you can feel the air moving at least a foot away, and it’s not like it comes to an abrupt halt after that.

      Even a cloth mask greatly decreases the velocity of the air in front of your face. If both people in the conversation are wearing masks, you are each breathing much less of the other person’s air. A face shield would do this too. For a short conversation this could make a substantial difference in the total quantity of the other person’s aerosol emissions that you inhale. How big is ‘substantial’? I don’t know and haven’t tried to estimate it, but I’d bet heavily that it is more than a 30% reduction and possibly more than 60% even if both of you are wearing very poor masks.

      Also, even a cloth mask will filter out some particles…not really effective against individual virus, but it is useful to filter out small virus-containing droplets, too: the water will quickly evaporate and reduce these to a size that can remain airborne for a long time.

      I hope someone out there is doing some real measurements of breathing zone concentrations under various circumstances. I’d be very surprised if cloth masks are not somewhat effective.

      • What you conjecture seems plausible. A popular alternative is that cloth masks further aerosolize and propel viral particles into smaller drops (or something like that – I don’t know, think of a hose shot through a chain fence). We can debate priors but at the end of the day without high quality evidence no one knows. And rather than conduct proper, robust, controlled studies the CDC ramps up highly politicized PR campaigns based on anecdotes (two hairdressers in Kansas! mannequins with double masks!)

        The disconnect between low quality of evidence and high population impact is troubling.

    • > To illustrate: Their “admission” of aerosols, while correct, isn’t even a net positive as it renders the advice on cloth mask as meaningful PPE incoherent.

      How is it incoherent? Yeah I agree that we should have moved on to higher quality masks sooner after the shortage stuff was resolved (as has been argued e.g. here https://www.theatlantic.com/health/archive/2021/01/why-arent-we-wearing-better-masks/617656/) but that doesn’t mean cloth masks are ineffective against aerosols. I admit I thought so too at the beginning of the pandemic because of the whole cloth is porous and aerosols are small thing. But mechanistically there is still an obstruction that affects rate and speed at which the aerosols get expelled (e.g. https://pubs.acs.org/doi/10.1021/acsnano.0c03252). It looks like there are a lot of factors that influence the effectiveness of cloth masks but anything above zero is still better than nothing, especially for a stopgap measure.

      Apologies if you were just saying that we should be doing better by now, to which I agree.

      • I totally agree that, at first order, we need to do better by now and aren’t.

        > anything above zero is still better than nothing, especially for a stopgap measure.

        I think this is naive, nothing in life is “free” and if the cost can’t be seen it should be a warning flag decisions for insufficient information. In fact, there are many plausible downsides, both directly — to socialization, fomite transmission via incorrect usage, bacterial growth, etc. — and indirectly — risk compensation, political decay via culture wars, wasting resources and focus, etc. Even just the loss of faith in the CDC and national health institutions due to politicization and (to be maximally charitable) overly certain statements may have serious long term consequences for the next pandemic.

        For as little effort as has been spent benchmarking the potential upsides, even less was spent investigating downsides.

        • I totally disagree. People were talking about fomite transmission via incorrect usage and risk compensation and all those things a ton. And they were the reasons people gave for originally objecting to masks. And yeah sure raising questions like these is often a good thing but doing the cost-benefit analysis in this case was pretty easy I think, and definitely after a couple months into the pandemic. Think, for instance, all the Asian countries that have been wearing masks way before the pandemic.

          Basically what I was presuming in my previous comment was that costs of masking are negligible and people were and are giving them undue weight. And I guess I shouldn’t have understated by saying “anything above zero”. If e.g. we were certain masks would only filter like 0.01 percent of aerosols or whatever then sure that’s also a negligible benefit so who cares. But early on it was clear enough that there was potentially a big upside against negligible costs.

  4. “It takes so much more evidence to correct a mistaken claim than to establish it in the first place.”

    This is primarily because so many people have deeply invested their careers in the prevailing wisdom. There is more to it, but the “serve-the-health-care-delivery-system” factor is barely a footnote.

    The problem in this case is that “Believe Science” has become the fraternal secret chant of the scientific community. As was we saw with the Javert Paradox, the underlying fear that the reputation of the entire community will be damaged by admission of error has become more important to many scientists and science groupies than actually discovering the truth. That has happened because the Scientist as Hero meme has been a major and increasing motivator to join the community for decades. There are numerous scientific organizations deeply committed to policy initiatives. Allowing science to make major mistakes would or could imperil those initiatives.

    • jim –

      > This is primarily because so many people have deeply invested their careers in the prevailing wisdom. There is more to it, but the “serve-the-health-care-delivery-system” factor is barely a footnote.

      I don’t know what method you use to determine what is the “primary” cause,but I do know there’s probavkh another significant Faroe in play.

      Public health officials, at leat in the US, are cognizant

      • Arrgghh.

        …are cognizant of the heavy stakes involved and the penalties to the public health system if they make mistakes of commission versus mistakes of omission. Changing policies from an existing policy requires an affirmative action.

        Arguably, if you change an existing policy and that change turns out to be wrong, public confidence in pubkic health policy in the future will take a bigger hit than if you just roll with an existing policy that turns out to be wrong.

        People are quick to attribute self-serving motivations to other people. My default is to consider the peope making such decision to have similar motivations as what mine would be -and if in a position of making lj mic health decisions my primary motivation would be to advance, well, public health.

        I don’t doubt that the causal mechanism you cite as “primary” is in play to some extent but I don’t think your confident description of public welfare as “barely a [motivational] footnote” is likely more a biased assessment than one well-grounded in evidence.

        • I there is a misunderstanding.

          You said: ‘your confident description of public welfare as “barely a [motivational] footnote” ‘

          I did not make a statement about the motivation to serve ‘public welfare’, as it seems you indicated. I made a statement about the motivation of public healthcare leadership to “serve-the-health-care-delivery-system” *ahead* of public welfare. My intention was to say that, contrary to the suggestion of the referenced editorial, public health officials mostly *weren’t* putting the welfare of the system or of providers ahead of the welfare of the public.

          Rather, their unwillingness to accept aerosol transmission as a primary form of transmission is about:

          1) the old guard simply refusing to accept strong evidence that contravenes it’s long-established view;

          2) the general, and I believe now common, view that science shouldn’t admit to errors or failings lest it lose credibility.

          I get that public officials are in a difficult position with respect to creating confusion etc. That’s the bind that arises when officials make strong statements on weak information. Thus it would behoove them to be more honest about what’s known and what’s not in the first place.

          Ha, too funny, it’s kind of analogous to being a Tesla stock evangelist. You’ve pitched the stock hard but it went the wrong way. What do you do? Double down or sell? Ha, hilarious.

        • This is Upton Sinclair’s principle that “It is difficult to get a man to understand something when his salary depends upon his not understanding it.”

  5. My belief is that we have witnessed (in real time, as measured by calendars)a failure to perform well-motivated hypothesis testing that would almost certainly resulted in policy recommendations designed to save lives. It does not take much evidence to demonstrate radically different outcomes.

    I am non-plussed by arguments defending (or rationalizing) delaying recommendations which would have prevented unnecessary deaths. I see no value in proposing an alternative hypothesis that the “health care system was looking out for its own” (my words, not Peter Dorman’s) when we had immediate claims from the greater health care system (in the US) that the best way to support them was to not get sick and tax the system (ehem “flatten the curve”).

  6. “If the importance of aerosol transmission had been accepted early, we would have been told from the beginning that it was much safer outdoors, where these small particles disperse more easily, as long as you avoid close, prolonged contact with others. We would have tried to make sure indoor spaces were well ventilated, with air filtered as necessary. Instead of blanket rules on gatherings, we would have targeted conditions that can produce superspreading events: people in poorly ventilated indoor spaces, especially if engaged over time in activities that increase aerosol production, like shouting and singing.”

    I’m a bit confused… most of this stuff was implemented early on. Phil on this site that did a post about air filtration in a yoga studio way back in June. Besides the early fumble on masks (which in many places have been mandated for a while (long before the winter surge)), what would have actually been different? Maybe they wouldn’t have closed the parks, but in most places that stupidity let up at the end of April 2020.

    Also, what is the current understanding of the importance of droplet vs. aerosol transmission?

    • Phil is not representative of the entire U.S. population. Yes, a lot of people like Phil and Tufekci recognized the importance of aerosols early on but many did not or chose to ignore it since the CDC guidelines ignored it. How many schools, factories, etc. invested in air filtration? Some did but not all. If the CDC put aerosol spread in their guidelines then places couldn’t say “we’re doing everything we can to protect our workers by following the latest CDC guidelines” and not invest in air quality measures.

      Re: the importance of droplet vs. aerosol transmission, you can take a look at this short Lancet article which Tufekci also co-authored and the references within. https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(21)00869-2/fulltext

      • “If the CDC put aerosol spread in their guidelines then places couldn’t say “we’re doing everything we can to protect our workers by following the latest CDC guidelines” and not invest in air quality measures.”

        I dunno. My office (which is in a 20 story building) was quick to announce that the filtration was up to par with what was needed and that increased circulation of outside air (and less recirculation of inside air) was being implemented. My kids’ schools noted the same thing when they were developing reopening plans last summer (opening windows in school rooms and on buses was highlighted several times). Unless your in someplace that still has an outdoor mask mandate, it seems like the things Tufekci was saying we should have done were actually the things that were done. It is not clear to me that the policies that were implemented or the understanding by the general public would have changed since by May (or at the very latest September) everyone was already wearing masks indoors, everyone knew it was much safer to be outside than inside, etc. Even my mom (who isn’t the most knowledgeable person on health news) knew that going to church was a high risk activity because of the singing. People were aware of the risks; they just chose to take those risks on anyway. I just don’t know what would actually be different.

        By the way, thanks for the link.

        • JFA –

          I had a very similar take.

          I worry about a tendency to hold public health officials to an unrealistic standard, that arises out of overestimating the negative impact of mistakes and rosy counterfactual scenarios that overestimate what would have been better had those mistakes not been made.

          None of that should immunize public health officials from criticism or the responsibility to evaluate what could have done better, of course. But the widespread mixing of impossible standards with the politicization of public health policy institutions and deliberate politically expedient exploitation of the simple reality that results of public health policy will always be sub-optimal, doesn’t bode well for future pandemics and other matters of public health policy.

      • From Michael J’s link:

        ‘SARS-CoV-2 stayed infectious in the air for up to 3 h with a half-life of 1·1 h.12’

        Badabing!! That’s how you make a wildfire pandemic.

    • > what would have actually been different?

      People’s effort and focus is finite. Every unit of attention given to say shaming beachgoers or the indoor cloth mask culture wars (which have at best minor benefit against aersols) reduces resources for interventions that actually help. To say nothing of indirectly discouraging rather than promoting (outdoor) socialization, fresh air, sunlight and exercise which all have clear positive health benefits for most people.

      • anon –

        >…which have at best minor benefit against aerosols..

        His do you know this?

        From what I’ve seen, research evidence is at least mixed (if not dispositive that cloth masks reduce transmission), and since m marginal risk benefit with each individual event compounds across the population level.

        My guess is that people failing to integrate the compounding effect of individual marginal risk benefit into their thinking about the pandemic has increased morbidity and mortality significantly.

    • Thanks for sharing that. It was quite informative (airborne does not equate to aerosol (at least as defined at the 5 micron level)). Maybe I interpreted him the wrong way, but in one of the tweets, he seemed a bit dismissive of the plexiglass dividers but then he provided evidence that proximity can increase susceptibility to infection. In a couple of the graphics provided, it doesn’t take much imagination to see how a plexiglass divider could be an effective mitigation strategy.

      • Plexiglass seems like a pretty reasonable protection from an airborne pathogen when someone is blowing it at you for a short period. But likely it can only be effective for short interactions. If you stand there and talk to someone for 20 min who’s expelling the virus, seems unlikely it would be effective.

    • Wow cool thread! After the aerosol issue came up last year it dawned on me that it must be the primary mode for all respiratory virus transmission. If you try to imagine every step in the path of a virus particle from sneeze to (surface) to (next infection), it starts to look highly improbable, and however improbable, there’s just no way in hell a virus could spread at the rate that COVID has through path.

      • jim,

        Not to mention the idiotic restrictions on “surfaces” that persist even today. There are still thousands of golf course which forbid anyone from touching a flagstick and which have removed rakes from sand bunkers. As if someone with COVID might touch a flag or a rake for a few seconds, the virus will collect on it and remain viable in the sun and open air long enough for another person to catch the disease later in the day.

        It was preposterous in March 2020, no face validity or even common sense behind the “idea” (if you can attribute any rational thought to it at all). It is certainly preposterous now but the pronouncements of CDC and other so-called authorities have created a climate of FUD to the extent nobody can tell the difference between superstitious reflex and reasoned action, much less actual evidence.

        • Everyone’s got an opinion; like everyone’s got a belly-button.

          In March 2020 my opinion was, “every bit of commerce which can conceivably be conducted outside should be done outside”.

          And, “In a year we will know more than we do now; but action must be taken *now*”.

          And, “The estimate of dynamic and evolving differential risks of various activities — from the mountain of accumulating case-histories — should be the effort of prime importance”.

          And, “It was found that — during the London blitz — that morale improved only insofar as the situation was neither sugar-coated, nor exaggerated.”

        • I can vouch for Ron sending me some version of each of those things via email way back in March-June 2020 ;-)

          He was right of course, but the official Federal response at that time was still “it’ll go away” and Trump didn’t start wearing a mask himself until what? September or some such thing.

        • “Move everything outside unless we learn something else about transmission” was so obviously right, I’m sorry your view didn’t prevail.

          Remember,though, that the Italians (and Spanish) were famously locking down indoors (which the U.S. never did), which made that seem reasonable.

        • The “surface” business is analogous to the “drunk looking for the keys under the streetlamp”. The whole business was then so terrifying — to the common-folk and the authorities alike — that some sense of agency was derived; by suggesting mitigations, which may (or may not) have been of marginal utility, but which at least could be acted-upon. And it was not completely far-fetched; or at least not completely at variance with the common-stock of ordinary medical lore. My informal surveys of clinicians over the years, asking, “How do you avoid catching colds and flu when you see patients several times a day”, always elicited the response: “We practice scrupulously to keep our hands down (so not to touch the face)”.

  7. I GET IT!!!

    I get why the CDC/WHO are dragging their feet so heavily over COVID as an aerosol. Ha! It’s because they’re still living in the freakin’ 19th century. OMG. Maybe other people realized this but it just dawned on me.

    It =goes back to the controversy between the germ theory and the miasma theory of infectious diseases circa 1850-1900. The battle was pretty intense I guess and the entire science still has a big hate on for anything that remotely smacks of miasma, which the idea of aerosol transmission definitely does (save the rotting corpse part). So they still have a bug in their bonnet about a hundred year old controversy. Amazing.

    • I thought it was bias from wishful thinking; the authorities were just as frightened of the thing as everyone else was. Even “there is insufficient evidence for X” is psychologically more reassuring than “it is possible that X” when X is something quite dreadful; which no one has a plan to face.

Leave a Reply to Michael J Cancel reply

Your email address will not be published. Required fields are marked *