Under the subject line, “Null misinterpretation of CIs reaches new level of lethality,” Sander Greenland points us to this article with the following in the Results section:
Compared to no masks there was no reduction of influenza-like illness (ILI) cases (Risk Ratio 0.93, 95%CI 0.83 to 1.05) or influenza (Risk Ratio 0.84, 95%CI 0.61-1.17) for masks in the general population, nor in healthcare workers (Risk Ratio 0.37, 95%CI 0.05 to 2.50).
Sander writes, “These results will likely be cited by authorities who have been trying to deny needs for masks. Seems impossible for authors to report that their results were actually too imprecise to establish an effect direction although the estimates were in the expected protective directions.”
I agree with Sander that the above quote is wrong. If you want to say there was a reduction but it was not statistically distinguishable from chance, then say there was a reduction but it was not statistically distinguishable from chance. Don’t say there was no reduction.
On the plus side, the article continues:
That time they got it right.
This illustrates one of the challenges of statistical communication: there are so many opportunities to garble the message. And even if you make no mistakes, people can still misinterpret what you’ve written, given the norm of acting as if every study either makes a discovery or proves that something equals zero.
We wrote about a very similar story last month.
Most of science is concerned with communicating an idea effectively. It’s quite similar to rhetoric in a way, which has developed logic and mathematics to help fashion convincing arguments. We want a scientific result to be uncontroversial.
Let’s take an idea like “wearing masks reduces the Covid infection rate”. Some people already believe that, and they don’t need any additional evidence. Some people are leaning towards believing it, and weak evidence will make them believe it. Some people don’t know, or distrust this idea, and you need stronger evidence to make them believe the idea.
What I would like is some kind of measurement of the effectiveness of some evidence, kind of along the lines “if a person A has a prior belief b(A) about the claim, then this evidence will elevate that belief above a certain confidence threshold, thus making the result uncontroversial”.
NHST used to do this in a very rough way: p>0.05 means it’s controversial, p 64%” or “we’d need to see a second study with strength S2=75 to make this convincing for most people”? It feels like a Bayesian approach would be a natural way to set up a standardized “strength measurement” like that?
Masks probably do have non-zero prevention effects. On the flip side, all the studies (including this one, the Bangladesh one, etc.) are showing very small point estimates, like 10-20% reduction in case numbers. There are probably many NPIs that produce at least that large of a reduction in risk that we seem to be OK with people disagreeing with, or just completely non-complying with. So why did masks become the holiest of NPIs?
Stella –
> Masks probably do have non-zero prevention effects
An effect that would presumably compound at the population level.
> So why did masks become the holiest of NPIs?
Citation needed.
>An effect that would presumably compound at the population level.
The compounding of effects of which presumable would be clearly visible in studies. I mean the Bangladesh study looked explicitly at the population level; so by your argument the point estimates per person should be still lower than the aformented 10-20% for the population?
Matty –
> The compounding of effects of which presumable would be clearly visible in studies.
I’m not familiar with the Bangladesh study. but unless there’s some kind of very sophisticated RCT, I’m not sure how you’d expect to show it empirically.
My thinking is based on a theoretical construct. If you prevent once case of transmission you’re effectively preventing further transmissions down the line. It’s theoretical but I have a hard time thinking of how that wouldn’t play out.
The magnitude of the compounding effect would vary on particulars, of course. If the next person in line who doesn’t get infected is a superspreader who sings in a choir the compounding would be less than if they is [I’m switching over to this sort of verb/subject agreement] a shut-in who rarely interacts with anyone else.
So a quick Google tells me that there was @ 9% benefit over 5 months. So it seems to me it’s only logical that the benefit would compound from there going forward. Even if everyone stopped wearing masks as soon as the intervention was over and everyone got infected eventually, it would happen sooner if no one had worn masks.
Is there something wrong with my thinking? Why wouldn’t ANY marginal benefit compound? If there were a given effect found after 5 months, how would you know it didn’t reflect the compounding of a smaller marginal direct benefit over time?
Er… Compounding effect would be MORE…
(BTW, don’t know if you saw it but I responded on that other thread after the crowd had moved on)…
> My thinking is based on a theoretical construct. If you prevent once case of transmission you’re effectively preventing further transmissions down the line. It’s theoretical but I have a hard time thinking of how that wouldn’t play out.
This is one way of looking at it, the other is “by preventing one case of tranmission you are moving the inevitable infection to a later time in the future”. Clearly neither of the two models is completely accurate, but by your logic the number of prevented infections is probably larger than the world population, so we should be corona-free.
Replying to your other comments:
>So a quick Google tells me that there was @ 9% benefit over 5 months. So it seems to me it’s only logical that the benefit would compound from there going forward.
Or not, I mean the effect over a one day must truly be *tiny* if it compounds to only 9% over 5 months. Lots of room for noise when we’re speaking about such tiny effects. I mean this is really a really naive model of disease spread, the kind that gets you EXPONENTIAL GROWTH everywhere you look. For whatever reason, it seems like these kinds of models aren’t very good at characterizing disease spread in real-life populations.
>but I responded on that other thread after the crowd had moved on
If you mean the Tucker Carlson one, see reply there.
Matty –
> Or not, I mean the effect over a one day must truly be *tiny* if it compounds to only 9% over 5 months.
I would expect only a tiny marginal benefit. A person has to be infectous in the first place (give that the benefit is supposed to be source control), and wearing a mask, and in a situation where a marginal gain from wearing a mask would be realized. Sure, any individual event in a limited time period on a given location with a limited effect on aerosols, would seem like a tiny gain.
Still, I don’t see how any marginal gain would not be compounded at the population level. Do you see how that might happen? Any person not infected then wouldn’t be going home and infecting family members and the like. Who then wouldn’t infect others and so on. I’m certainly open to an explanation how that wouldn’t compound. I’ve bee. Asking about it for over 1.5 years and no one’s given me an explanation yet.
And then we’re treating this as if it were some kind of dichotomous outcome variable. I guess is it may well not be. Maybe wearing a mask has a marginal benefit of reducing the severity of an infection.
>how that wouldn’t compound. I’ve bee. Asking about it for over 1.5 years and no one’s given me an explanation yet.
The obvious explanation is the one outlined above – that there are multiple paths in the social graph that can lead to an infection, and so stopping one doesn’t mean you don’t get infected. Another, more esoteric explanation would be to postulate that masks prevent the kind of transmissions in the transmission graph that die out quickly with/without masks. This would be consistent with masks providing some benefit on average and not providing large net benefit. For example, if the population were composed of two poorly mixing groups of (a) 50% careful individuals that have R << 1, and (b) 50% reckless individuals that wear masks poorly and kiss when greeting, then the masks may well "work" for group (a), but this is irrelevant to the total dynamics.
Another interesting question would be to see how these kind of things compound probabilistically. I mean exponential growth is deterministic; if you set your stochastic model up in a way where masks help on average but make super-spreading slightly more likely, you could presumably have worse growth even though masks help on average. I'm not saying this is true (or even likely) but mathematically it should make sense. And if you let your imagination run wild, you could imagine a world in which masks keep covid in making infections worse, and thus super-spreading more likely.
Ultimately in the long run it’s the average that matters. In the short run a super-spreading event may make things increase quickly, but for super-spreading to be more likely and yet the average decrease slightly, it must be also that more people produce 0 or 1 additional cases where they would have produced say 1 or 2 or 3 before for example.
Matty –
> The obvious explanation is the one outlined above – that there are multiple paths in the social graph that can lead to an infection, and so stopping one doesn’t mean you don’t get infected.
I don’t understand your point. If someone gets infected, there is always a risk of forward transmission unless they’re totally and successfully isolated. If their infection could have been prevented by the person infecting them wearing a mask, you’ve broken an inevitable multi-link chain of risk of forward transmissions. Of course for any particular infection, forward transmissions may not take place. Was that your point? If so, that doesn’t mean that overall, if masks prevent infections at all, they eliminate a necessarily compounding risk of forward transmission.
> Another, more esoteric explanation would be to postulate that masks prevent the kind of transmissions in the transmission graph that die out quickly with/without masks.
I suppose that’s theoretically possible if they only prevent the transmission of infections that would necessarily die out without further transmissions.
I’m not sure I’d use esoteric to describe that possibility, rather than rather strained – given that there seems to be much evidence of significant variability in how people respond to infection the known variability in how much different people interact with others across a large variety of contexts. But OK, I’ll take that as a theoretical way that compounding wouldn’t take place.
> This would be consistent with masks providing some benefit on average and not providing large net benefit.
I don’t think so. Because there’s huge variability in how people respond to infection. A uniformly low “viral load” for all individuals could still produce huge variability in how infectious all infected people are. It’s now like a low dose to everyone would mean that no one would be highly infections.
> Another interesting question would be to see how these kind of things compound probabilistically. I mean exponential growth is deterministic; if you set your stochastic model up in a way where masks help on average but make super-spreading slightly more likely…
Do you have any mechanistic explanation for how that would work? Absent one, I’m not going to consider that as a meaningful pathway that compounding wouldn’t take place. You’re obviously entitled to go with that if you’d like – but it doesn’t work for me.
> And if you let your imagination run wild, you could imagine a world in which masks keep covid in making infections worse, and thus super-spreading more likely.
Sure. We could also go with a multi-verse scenario. But with that same way of thinking we could say there must be a universe where masks work 100% for people who don’t argue that they don’t work. S maybe this is that universe. o what are need is for everyone fo stop saying that and we could stop out the pandemic tomorrow.
Matty –
> > The obvious explanation is the one outlined above – that there are multiple paths in the social graph that can lead to an infection, and so stopping one doesn’t mean you don’t get infected.
OK. I Iooked at that again. It looks like a non-sequitur, basically. Yes, preventing infection to a given person via a given pathway doesn’t mean that person won’t get infected via other pathways. That doesn’t mean that an infection that could have been prevented by mask-wearing couldn’t or wouldn’t compound.
According to the Bangladesh study, the difference between villages with 13% masking and 42% masking was 9% reduction in cases.
If the main benefit of masks was respiratory protection (of the mask-wearer from others), then simple linear extrapolation could be reasonable.
However, the main benefit of masks is considered to be source control (protecting others from asymptomatic mask-wearer).
Therefore, it is reasonable to think that the difference between communities with 13% masking and 71% masking will lead to more than 18% reduction in cases.
It’s not even clear there is an effect, so it’s kind of funny to see that it’s already ok to *nonlinearly* extrapolate from this lack of a statistically significant effect.
That said, I’d need some kind of proof to accept the nonlinearity here, because it isn’t obviously true to me.
“I’d need some kind of proof to accept the nonlinearity here, because it isn’t obviously true to me.”
It is obviously true because viruses spread exponentially. Everything that replicates does until it reaches the carrying capacity of its environment. If you don’t think that is so, then it is hard to see how you could believe in evolution, a theory that depends on the fact that all living things grow at a geometric rate. So, if I put in place mask wearing and stop the spread of a virus, that is going to have a non-linear effect. The problem with the discussion of masks is that mask wearing, which is the intervention we should care about is being conflated with mask mandates and mask recommendations, which may or may not result in mask wearing, which obviously will reduce the spread of the virus. It is like the recommendation to wear your seat belt. We only need the a mechanical studies that show a seat belt can prevent bodily harm to justify the recommendation to wear one. We don’t need studies that show actual reductions in harm because that mixes together compliance with efficacy.
Scenario 1: Do not wear a mask and (socially-distanced) stand in line in your local big box store. Your breath is sprayed primarily downward and outwards. You step forward and the next person steps into your space.
Scenario 2: Do wear a mask, instead of your aerosols being sprayed down and out 10% get captured in the mask and the other 90% clouds around your head. You step forward and the next person steps into your space.
“Your breath is sprayed primarily downward and outwards. ”
Which doesn’t matter much because you’re exhaling air that’s substantially warmer than room temperature and will rise into the (COSTCO, Home Depot, Kroger, Target) skylights far above your head in a few minutes or so.
“You step forward and the next person steps into your space.”
Which isn’t much of a problem since it generally takes a substantial time of exposure to a high concentration of virus to contract the infection.
Granted there must be some risk of exposure under these conditions that common masks offer some degree of protection against, but it’s clearly very small, because places where people hang out for extended times (parties, bars, restaurants) are hotspots for infection, but supermarkets and big-box stores aren’t.
The problem with common masks is that they’re likely very effective where the risk is lowest, and likely not very effective where the risk is high.
Effective mask demo
Not exactly a good scientific study but great fun.
Almost all of the practical issues I can think of would tend to reduce the apparent effectiveness of properly-worn masks, or at least I think they would. For instance, if mask-wearing status is based on self-reporting, I would guess that people will tend to over-report how often they wear masks. Many people who wear a mask will do it badly, e.g. not covering the nose. And of course if what is being imposed or observed is a mandate to wear masks rather than the actual wearing of them, you have the issue of non-compliance. I think all of these issues would tend to lead to an underestimate of “how much is transmission risk decreased if one wears a mask properly.”
That said, I agree with your point that mask-wearing is probably only mildly effective so I think it’s worth asking why it has attained special status, which I agree that it has. I think one part of the answer is that it’s very visible: in a sense you are advertising “I am taking the pandemic seriously.’ In contrast, vaccination obviously has a much much larger effect in all ways, but vaccination status is not visible. If every vaccinated person were wore a purple shirt while unvaccinated people didn’t then I think everyone would focus on vaccination status much more than on masks; at least that would make sense to me.
And of course masks have now become political: some people who don’t wear masks are deliberately making a political statement, and other people do wear them because they don’t want to be mistaken for the other kind of people!
I wear a mask when I’m indoors with other people (outside my friends and family) but not for a single simple reason. Partly I wear one because it’s an easy way to slightly reduce my risk of getting covid or passing it on; partly because I know other people will be anxious if they see me without one; partly because I don’t want people to think I’m a pandemic skeptic; and partly because it’s legally required in my area.
+1.
Phil –
> I would guess that people will tend to over-report how often they wear masks.
That social desirability bias might well run (if perhaps less so) in the other direction for people who run with anti-maskers.
I think another reason reason for the focus on mask-wearing is because, particularly early on, widespread use of masks was seen to be associated with far less spread in Asian countries.
Yesterday I went for a long walk on a beach. I wanted to keep the wind and sun off of my face and I had a handkerchief, so put it over my face as I walked. I assumed that people who saw me figured I was a COVID nut (no one else on the beach was wearing a mask). I felt somewhat comforted that while they thought I was a nut, they probably thought I at least I had an actual reason for wearing a mask rather than just a random nut, as I would have likely been judged had people not started wearing asks during a pandemic.
I like to wear a neck gaiter over my face when hiking at altitude with minimal to no shade – to protect from sun/UV. Along with dark sunglasses, a broad brim hat, and sun gloves, of course. I think the pandemic has made this look slightly more acceptable.
Phil said, “That said, I agree with your point that mask-wearing is probably only mildly effective so I think it’s worth asking why it has attained special status, which I agree that it has.”
One reason mask-wearing has attained a special status, is we have done very little else in the US in terms of real interventions. We don’t impose real quarantines. We don’t provide true notice that someone in your immediate area has the virus. I can go online and find out if someone in my mother-in-law’s building recently tested positive for COVID. Similar measures could have been imposed here (I think without violating HIPPA, but HIPPA could have been revised if needed). We could have had real contact tracing. So, we are left with masks and social distancing, and the public was left to fend for itself.
Masks serve the same purpose as the TSA security checks at airports. A small chance of possibly reducing risk but mostly a large visibility of “doing something” to help people feel safer. But the the feeling safer part only works for people who choose to believe it on no particular evidence, of course.
Even a small chance of reducing a transmission line can have an enormous effect since that line of transmission can lead to thousands of cases over time. The bomb inspector just stops one event at a time. The average effect makes sense in the latter case and no sense at all in the former.
Something lost in the “fog of war” over masks is the distinction of a rate vs an amount.
Even if we agree masks reduce transmission by a few %, if it doesn’t bring the percolation to extinction then at best we push cases to the future (by that small % relative to origin). Pre-vaccine maybe this made sense… but now to what end? Given the strong association between age and outcomes its very plausible that delaying first exposure by 18 months is marginally harmful.
Nearly everyone alive will get covid, those who don’t will die first for other reasons.
d-
On a personal level, I am thinking similarly. I’m almost inclined to want to get COVID now, while my vaccine immunity is still high, rather than delaying and possibly getting a worse case later. But there is something about your tone that makes it sound like you know the answer and are rejecting masks (especially mandates) due to the possibility that this logic is correct. I would also point out that even if it makes sense for me to get exposed to COVID now rather than later, the calculations for my mask wearing on society are something quite different. I could easily infect someone whose immunity is low (either due to being unvaccinated or due to a medical condition) and they might not be better off at all.
d –
> … but now to what end?
A slower rate of spread might allow more time for people to get vaccinated. It might allow for more time to study the effects of vaccination and to revise the formulas for future vaccines. It might allow time for further testing and improvement of therapeutics. It might allow for reduction of stress on healthcare workers. It might allow for the economy to be more normalized.
Granted all of those benefits would be limited . But limited improvement is still improvement. And id say with Healthcare workers even a very small improvement is very valuable.
Andrew, this fallacy and the “difference between significant and non-significant” fallacy are super common IME – agronomy and ecology. I would reckon that a substantial majority of peer-reviewed manuscripts are built at least *partially* around weaving a web of results on this false interpretation, and a smaller but still substantial fraction almost entirely so.
The basic problem seems to be the need to simplify. There are so many comparisons and analyses to be made, and the volume of research through-put demands increasingly compressed communication of stories. I really don’t know what we do outside of a general pumping of the brakes necessitated by more principled work-flows…which will never be tolerated by agency and university administrations.
Your comment is also misleading. You’re not quoting the Results section in the article. You’re quoting the Results paragraph in the abstract of the article.
They are less succint in the actual Results section: “Pooling of all nine trials did not show a statistically significant reduction of ILI cases (Risk Ratio 0.93, 95% CI 0.83 to 1.05) or laboratory-confirmed influenza cases (Risk Ratio 0.84, 95% CI 0.61-1.17) in the group wearing a mask compared to those not wearing a mask […] A separate analysis of the two trials in healthcare workers also failed to show a statistically significant difference between the mask and no mask groups (Risk Ratio 0.37, 95% CI 0.05 to 2.50).”
Carlos:
The #1 thing people will read is the title and the abstract. It’s fine to have qualifiers in the body of the article but I think it’s crucial to not get things wrong in the abstract.
Likewise, most people will read your comment without looking at the paper and will get the impression that the “sloppy” language was from the Results section and not from the space-constrained abstract.
In that context the quote seems more defensible. They are not hiding the reduction: the 0.93, 0.84 and 0.37 risk ratios are there. Those who do not understand what those numbers mean or do not care may not understand or care about the subtleties of statistical significance either.
Carlos:
I think it’s fine if people read my post and conclude that the paper misrepresented the evidence. As far as I’m concerned, misrepresentation in the title and abstract is misrepresentation, and indeed it’s the most important misrepresentation. It’s also fine with me if people click through and read the article in detail.
Yes, the abstract is space constrained. That does not mean they had to write, “there was no reduction.” If people don’t understand or care about the subtleties of statistical significance, that’s fine; just don’t tell them there was no reduction.
Ok, I understand why you don’t like that part of the abstract. It would be better if the “there was no [statistically significant] reduction” was explicit instead of implicit in that paragraph.
On the other hand, it’s debatable if it would have been “less wrong” if they had made the opposite claim: “Compared to no masks there was a reduction of influenza-like illness (ILI) cases (Risk Ratio 0.93, 95%CI 0.83 to 1.05) and influenza (Risk Ratio 0.84, 95%CI 0.61-1.17) for masks in the general population, and also in healthcare workers (Risk Ratio 0.37, 95%CI 0.05 to 2.50).”
Agreed. Better would’ve been to conclude that there’s not enough evidence from these data alone to say much.
I’m not entirely comfortable with this kind of reasoning, maybe because
> These results will likely be cited by authorities who have been trying to deny needs for masks.
presupposes that masks are “needed” instead of treating it like the open question that it is. But more importantly, “our data shows a reduction” will be true about 50% of the time, and is thus fairly meaningless. Saying “we wouldn’t be surprised (p=whatever) to see this kind of reduction even if masks didn’t work at all” is IMO the more important takeaway than the reduction given the large-scale support for masks by scientists all around the world.
“We recommend the use of masks in combination…” is not “getting it right,” it’s hitching fake science to useful science. They have no basis for that recommendation because they do not consider costs and benefits of alternatives.
To be fair, they do not recommend requiring, enforcing, or even encouraging the use of masks; they only recommend using them. They don’t say they are worth the price. But if you’ve got one and don’t mind at all, and no one else minds, go ahead and put it on. wow thanks.
Do we infer that they don’t recommend for or against distancing, in combination with other measures?
I am not sure yet what I think of masking. But I demonstrate my woosieness in this relatively easy way. Now Paul Offit claims that the low flu rate this year may be because people have been masking and social distancing.
Vinay Prasad has been suggesting that there are no credible studies on the benefits/drawbacks of masking. He has certainly spent days and days ribbing stakeholders about the use of masks in children under 5.
Apologies, I meant to add that I do wear a mask in stores and in gym, as required. I think that had we produced & distributed billions of high quality rapid antigen tests back in July 2020, we wouldn’t have seen so many deaths. Everyone was so focused on vaccines and believed that they would lead us out of this pandemic. A few cautioned the FDA and CDC about such an optimistic hypothesis.
I know it’s cumbersome to test so many people. In home rapid antigen testing seems far more promising than does masking, as a mitigating effort.
I struggle with this in the peer review process, especially when there are quasi-parallel statistical tests with very similar point estimates but one has a “small” p value and one comparatively “large” (e.g., .5 < x < .15). Reviewers can get hostile when I try to point out that the tests are yielding similar results and would much prefer I say one is supported and the other is unsupported, as if to say there's nothing similar about them.
Aren’t the standard academically acceptable weasel words here: “qualitatively similar?”
The burden of proof is totally backwards here. Imagine the treatment under consideration were not masks but a shiny new pill from big pharama. Would these results merit FDA approval? Would they merit government _mandates_ on their use? … obviously not. The PR-speak of how failed study gets spun is uninteresting to me.
It’s been 19 months, if masks are to be mandated as “medical devices” let’s subject them to the standard scrutiny first.
Given the low downside risk, public health officials shouldn’t need to “prove” that masks work. Only that there’s a chance they can reduce a huge public health problem. The upside economic and public health gains from a small relative decrease in transmissions are enormous.
I strongly disagree. This very study suggests that plausibly masks increase transmission! (which was the consensus view pre March 2020)
Beyond that, there are many obvious 2nd order costs to enforced masking in a world with finite capacity for action and rapidly diminishing trust in establishment. It’s already game over for apolitical public health going forward, in part because of this very issue.
d –
> This very study suggests that plausibly masks increase transmission! (which was the consensus view pre March 2020).
I haven’t seen studies that say “masks increase transmission.”
I’m open to new information but I’ve been looking at studies all pandemic and what I’ve seen are studies with varying conclusions about the efficacy of masks, some running in either direction. That’s to be expected given the uncertainties involved.
I think it’s unreasonable to expect otherwise.
From what I’ve seen the downside risk is small. And as I said, I think the upside benefit is potentially enormous.
> Beyond that, there are many obvious 2nd order costs to enforced masking in a world with finite capacity for action and rapidly diminishing trust in establishment.
I don’t agree. I see a lot of people who are absolutely determined to use whatever they can to reinforce a political drive to attack any establishment policy that comes from the other side of the aisle. If it isn’t masks, it will be something else.
> fIt’s already game over for apolitical public health going forward, in part because of this very issue.
Again, I don’t agree. This is an issue because it meets fits the desire to politicize public health policy.
d –
> This very study suggests that plausibly masks increase transmission! (which was the consensus view pre March 2020).
OK. I went and followed up on that a bit. I’d say that “suggests that plausibly masks increase transmission” is a bit strong (I think it should be caveated with something like “may?”). But so is my rejection that any studies say “masks increase transmission.” Probably somewhere in the middle would be best.
There is the one study referenced of healthcare workers (not during a pandemic) where cloth masks were associated with higher influenza-like-illness (but not influenza transmission, strangly enough)…
Of note…
-snip-
It is also unknown whether the rates of infection observed in the cloth mask arm are the same or higher than in HCWs who do not wear a mask, as almost all participants in the control arm used a mask.
-snip-
Honestly, I had a hard time making heads or tails of that RCT as applied to the covid pandemic. I guess it does imply a high downside risk, so I should walk that statement back – but in the full range of the studies I’ve seen, it doesn’t move the needle much in my view.
Almost any intervention you can imagine could be considered by someone to have “huge potential upside” if it really lowers transmission risk. But you can’t demand that hundreds of millions of people comply with EVERY measure that someone, somewhere speculates might make a difference. Even if any one of those measures had only a small cost or downside risk, there has to be real evidence on which to implement it as a universal policy.
The reason I do not expect this pandemic to be under control in the forseeable future is not that masks don’t work (whatever “work” means). It’s that nobody is making serious, scientific study of the real-world effects of masks along with all the other things that are promulgated as policy. There are still some agencies out there continuing to press the idiotic “door knob and toilet seat” type advice as those COVID-19 is being transmitted by surface contact.
The so-called science of studying transmission and impact of policy, behavioral and pharmaceutical interventions consists mostly of just aggregating dubiously valid measures from whatever convenience samples come along. That sort of thing is never going to sort out “upside” from “downside”, much less put realistic estimates on the magnitudes of either. It just accumulates so that it can be cherry-picked to justify whatever science-free policy decisions have already been made on a political or tribal basis.
nwbr –
> Almost any intervention you can imagine could be considered by someone to have “huge potential upside” if it really lowers transmission risk. But you can’t demand that hundreds of millions of people comply with EVERY measure that someone, somewhere speculates might make a difference. Even if any one of those measures had only a small cost or downside risk, there has to be real evidence on which to implement it as a universal policy.
Of course.
> Even if any one of those measures had only a small cost or downside risk, there has to be real evidence on which to implement it as a universal policy.
Assessing “real evidence” is equally subjective.
> The reason I do not expect this pandemic to be under control in the forseeable future …
That’s not the standard I think should be used. My standard is trading off risks with a goal of harm reduction.
> is not that masks don’t work (whatever “work” means).
Another subjective evaluation.
> It’s that nobody is making serious, scientific study of the real-world effects of masks along with all the other things that are promulgated as policy.
I don’t think that’s true at all. It’s an incredibly difficult target for empirical evaluation. But there are a lot of people making serious attempts.
> The so-called science of studying transmission and impact of policy, behavioral and pharmaceutical interventions consists mostly of just aggregating dubiously valid measures from whatever convenience samples come along. That sort of thing is never going to sort out “upside” from “downside”, much less put realistic estimates on the magnitudes of either. It just accumulates so that it can be cherry-picked to justify whatever science-free policy decisions have already been made on a political or tribal basis.
I don’t agree with your description there – but regardless I certainly do think it’s a very imperfect science. So for me the point is to evaluate risks as best you can – including the high damage, low probability risks. Holding public policy hostage against an unrealistic standard is a form of decision, just as are vaccine mandates.
My understanding is that in early 2020 public health specialists were divided, with researchers in East Asian being optimistic about masks and researchers in the rich Anglo countries being pessimistic. Also, in February 2020 many people believed false things about airborne transmission of diseases, so their best attempts to set policy were based on flawed premises. And the variants of COVID which were spreading were less transmissible than current variants, and the means of transmission was not well known. Some people thought it might be mainly spreading by touch!
My impression is that the effectiveness of cloth masks against the Delta variant is still debated, and that there is better evidence for other types of mask helping.
D:
I think there are larger issues of trading off risks and benefits. Masks are not medical devices in the usual sense, as anyone can make a simple mask of the sort that we’re wearing every day. I’m here at Columbia University, and it’s complicated. Sure, there are people who find masks annoying—whenever I have to wear them for awhile, I start longing for a deep drag of fresh air. At the same time there are other people who are scared of any exposure at all, and the mask requirement makes them less concerned. I don’t think there are easy answers. Everyone’s always been allowed to wear a mask when they might be sick, so it’s not really a question of FDA approval or anything like that, but only in the past year and half has it been common behavior in this country. As is typically the case, rules instituted by organizations, businesses, and governments are not coming from nowhere; they’re response to people’s concerns and they’re part of the larger picture of wanting to get people back to work and back to school. In that sense, it seems odd to me when people are in favor of getting back to a normal life but are opposed to masks. The purpose of the masks is to make it easier to get closer to normal life. Masks are annoying but I much prefer masked in-person education to unmasked zoom education.
Andrew:
You presuppose that masks are helpful and necessary to get back to normal. An alternative prior is that they do nothing and rather moving past hygiene theater is how we get back to normal. If gathering robust evidence were a priority — and why isn’t it? — we’d have the data necessary to bake in subjective concerns (PTSD/fear, politics, etc.) and find a path forward. Until then, I believe “no intervention” or at least “no mandated intervention” has to clearly be the default stance.
D:
You write that I “presuppose that masks are helpful and necessary to get back to normal.” I said nothing of the sort. I said two things. First, I said that your discussion of FDA approval of masks and the comparison to medical devices didn’t make sense for masks. Second, I said that mask rules are a response to people’s concerns. There’s a general desire to gather robust evidence, but in the meantime we want to get back to work and get back to school, and one way to make this happen is to address people’s concerns, one way or another. You can desire whatever default stance you want, but schools have to address the concerns of students, parents, and employees—and lots of people in all three of these categories are asking for policies to limit covid spread. I don’t really care if you call concerns of covid spread “subjective” or “objective”; in any case, policymakers at the private and public level have to address them. I’m pretty sure that your default stance of no intervention (I guess you mean no masks?) would make things very difficult at Columbia and other places. Again, these are institutions that have put in a lot of time, effort, and money into allaying people’s concerns.
The FDA thing is certainly rhetorical and not my central point. Though I will note boxes of masks now say things like “does not prevent covid19” to avoid the implication of actually being approved medical devices! (And had similar generic warnings pre-covid to reflect the pre-2020 consensus on their ineffectiveness.)
I understand what you’re saying that mask wearing is an olive branch and may be necessary socially (or politically) to move forward. This may even be good policy on net! But let’s be honest then and say, as you are: there is no robust evidence they work, but let’s wear masks anyways for a bit while everyone gets comfortable.
Instead what nearly every institution is poisoning the creditability of public health going forward by starting from the premise “they must work!”, scurrying to find evidence to justify and then overstating the certainty. Kudos to the original authors for not doing so.
D:
I guess I’ll have to look at Columbia’s covid materials to see what they say. I don’t think Columbia is saying that masks must work; I think they’re just requiring them in some settings. There are lots of requirements at the university (and elsewhere) about which there’s no robust evidence. Such requirements often annoy me; I also see that they’re often there for a reason.
Mask rules are surely “a response to people’s concerns,” but I don’t think that’s at issue; at issue is whether they work and whether mandating them is good public health policy. D is arguing there isn’t clear evidence they work, that they could even be harmful, which would make them a poor default policy. Imagine if a university tried to allay people’s concerns by telling them that wearing red prevented covid.
Unlike D, I think masks are probably more effective than wearing red, but let’s discuss evidence and mechanisms, not merely public health authorities’ desire to be seen as doing something.
This was meant to reply to Andrew’s response to D above, beginning “You write that I ‘presuppose…'”