Going through old emails, aiming for Inbox Zero, I came across a note from Sander Greenland from Nov 2020 pointing to this online petition from Noah Haber, Mary Kate Grabowski, et al., requesting that PNAS retract a paper, “Identifying airborne transmission as the dominant route for the spread of COVID-19.” The petition is from June 2020, and some googling revealed that I discussed that article around the very same time, in a post entitled, “The point here is not the face masks; it’s the impossibility of assumption-free causal inference when the different treatments are entangled in this way.” I’d been alerted to the article by Adam Pearce.
Anyway, now it’s March 2020 and this post is scheduled to appear in August, more than a year after that petition. I see on google that the controversial article has nearly 400 citations (which I assume will be over 400 by the time this post appears). It does not seem to have been retracted; there’s a correction note (just minor things, for example, “78,000” has been changed to “75,000”) and a couple of letters.
I hope we can all agree on the impossibility of assumption-free causal inference when the different treatments are entangled. This is related to a point made by Rubin in his classic 1978 paper, that randomization gives robustness.
P.S. In comments, Joseph Delaney writes:
What I [Delaney] find most fascinating about this issue is that now, in August 2021, the CDC guidelines clearly recognize airborne transmission now:
“Current evidence strongly suggests transmission from contaminated surfaces does not contribute substantially to new infections.” which suggests a large role of droplet and/or airborne (given they only identify three modes).
This brings up the interesting question of what to do about bad papers that happen to be confirmed despite having methodological issues. Are they prophets or should we be citing the first paper that was rigorous/robust (and how do you decide on a sliding scale of grey). These are hard problems.
I agree.
What I find most fascinating about this issue is that now, in August 2021, the CDC guidelines clearly recognize airborne transmission now:
http://observationalepidemiology.blogspot.com/
“Current evidence strongly suggests transmission from contaminated surfaces does not contribute substantially to new infections.” which suggests a large role of droplet and/or airborne (given they only identify three modes).
This brings up the interesting question of what to do about bad papers that happen to be confirmed despite having methodological issues. Are they prophets or should we be citing the first paper that was rigorous/robust (and how do you decide on a sliding scale of grey). These are hard problems.
Well what got me interested in evidence based medicine was in listening to physicians provide or not provide good enough evidence for a particular prescription. They hardly ever added the possibility of side effects. During informal conversations with medical academicians/researchers, I get a better grasp of their reasoning. And it isn’t satisfactory often. John Ioannidis actually alludes to this situation often. Allopathic medicine is limited theoretically. This has been becoming more and more recognized. Here and there there are superbly talented academics. But they too end up marginalized, imo.
What I have concluded there are far too many experts in every field. John Ioannidis mentioned recently that there were over 495,000 authors on the subject of COV-2. Who is to say why only a subset is privileged to be viewed as authoritative?
And what I find surprising is the lack of data/evidence for a particular policy recommendation.
“Allopathic medicine is limited theoretically.”
[1] What is “Allopathic medicine” ?
[2] What are the other varieties?
[3] What are the “theoretical limits” of those other varieties?
First – I believe there is a typo in Andrew’s post, that probably should be March 2021.
Second, Joseph you beat me to it. I don’t know enough to comment on the methodologies in the paper, but what interests me is that usually (though not always) when people create a petition to have a paper retracted it is because they not only find the methods flawed but disagree with the conclusions. I would be curious how the people who started and signed the petition now feel about the effectiveness of different treatments (direct and indirect) on COVID. (And I sure hope we can avoid a discussion here of how effective different treatments are, that is not my point.) And thereby, I wonder if the people who started the petition now feel the same way, given that there have been a number of studies that support the conclusions of this paper, if not the methods.
Andrew –
> I hope we can all agree on the impossibility of assumption-free causal inference when the different treatments are entangled.
You’re quite the optimist!
Consider your recent previous post where there was much claiming being made of assumption-free casual inference, where different treatments were IMPOSSIBLY entangled.
I’m looking at what’s going on in Florida now, in reference to which large %’s of the American public are deluding themselves that they can make assumption-free causal inferences where treatments are entangled. A few weeks ago Republicans were pointing to Florida as proof that Republican politicians save lives and Democrats were trying to avoid discussing the COVID outcomes in Florida. Now the tables have turned.
I wish that it were true that COVID would make it more obvious to people that assessing a causal signal in a complex and uncertain mix of treatments and influential variables should demand circumspection. But I think the opposite is actually true. So many people are now fully convinced that they know all they need to know to conduct a proper epidemiological analysis of causality with so many COVID-related phenomena.
We’ve seen this previously with respect to climate change, where in fact there’s polling data that shows that people who are most confident in their ability to assess casualty are among those who are least able to accurately describe the science. COVID, I fear, shows that the dynamics in play there have only gotten much worse.
What on earth is “assumption free” inference? Does it mean that a large collection of facts is presented in a table, or in a box of random newspaper clippings; set upon the floor; whence everyone then confronts it and on the force of the “data themselves” a gradual, but incontrovertible “aha!” slowly emanates from the crowd of onlookers; who are thereby persuaded of something that the did not know before; but which understanding is free from any assumption at all; persuasion being brought-about (I hesitate to say “caused”) merely by the imminence, the spectral presence of the data, simpliciter?
Andrew, At one point I emailed you about the whole science of Aerosols and you wrote a post on it, maybe tomorrow would be a good day to follow up on this topic with that post? We know Aerosols are a dominant transmission mode because mechanism says they basically HAVE to be, and that the “dogma” about “aerosols are only sub 5 microns” was always wrong. Mechanism is much stronger than causal inference from observational data ever will be.
Daniel:
OK, I think I found the post you’re talking about. It was scheduled for Oct but I bumped it up and it will appear tomorrow.
“Mechanism is much stronger than causal inference from observational data ever will be.”
Daniel, what do you mean by “observational data”? It seems like there’s another assumption embedded in your statement: that “observational data” is something that comes in rows and columns of measurements that can (only?) be analyzed mathematically.
The early CDC reports – which I categorize as “observational data” even though they didn’t provide rows and columns of measurements – clearly showed rapid transmission at large gatherings in which people who were never within six feet of the infected person became infected. It’s unfortunate that the “observational data” (?)sensu Lakeland/Gelman(?) is useless. But the fact that we don’t have rows and columns of measurements doesn’t mean we don’t have useful or even unequivocal “observational data” (e.g., sensu Jim).
Recently on CNN Micheal Osterholm summed it up: if an infected person is smoking and you can smell that person’s cigarette, you are probably exposed to COVID. There’s simply no question about that.
No “observational data” to me means “data taken from outside a controlled experimental context”. and it’s NOT “useless” as you’ve put words into my mouth. It’s just not as powerful for coming up with causality as a known mechanistic experiment would be. For example, people can put a mist of fine droplets into a room containing some detectable chemical, and then go swab all around or collect air samples with a machine, and determine the quantity of that chemical you’d be exposed to at a certain distance from the source at a certain time interval etc. If doing this kind of experiment you can show that certain kinds of chemicals transport through the air a certain amount… then later you can be sure that when that chemical is replaced with droplets of the same size containing virus, that you will have exposure. nothing about having the virus in the droplets vs having a tracer chemical will make a difference.
You’ve already basically confirmed my point by the Osterholm quote, which is that people do that “experiment” all the time, aerosols from cigarettes go all around a room pretty quickly. So we know from that experiment that you can spread aerosol particles widely. The more controlled experiment would try to reproduce the particular spectrum of particle sizes and quantities from human respiratory activity rather than the burning of a cigarette.
An ‘observational dataset’ of “so and so many people happened to get covid at this particular site at this particular time” provides much less causal inference capacity than these controlled experimental mechanistic inferences, since you don’t know if that’s because they were kissing, singing, eating the same food, whether it was a particular person who happened to infect everyone else, or there was a small group of people who were infectious etc… you have to figure out a lot of what happened without having a controlled context that eliminates other confounding variables.