Biostatistician Jeff Morris writes:
I [Morris] have downloaded and evaluated the recent Israeli data in detail to explore what it tells us about efficacy vs. severe disease with the Delta variant.
In spite of the fact that ~60% of those with severe infections are vaccinated (as emphasized by anti-vaxxers as well as people pushing third dose boosters), the data clearly show the efficacy vs. severe disease is 85-90% in both younger and older age groups.
I have written an article on my covid-datascience.com blog clearly explaining this paradoxical result step by step.
My explanation illustrates the erroneous arguments made by people who just compare raw counts in discussing vaccine efficacy, and also highlights how Simpson’s paradox rears its ugly head here given older people are both more vaccinated and have inherently higher risk of hospitalization and thus any overall efficacy results produces misleading results if not stratified by age.
Morris’s one-sentence summary:
Many are confused by results that >1/2 of hospitalized in Israel are vaccinated, thinking this means vaccines don’t work. I [Morris] downloaded actual Israeli data and show why these data provide strong evidence vaccines strongly protect vs. serious disease.
I agree with Morris that this is a policy-relevant example of a general statistics principle.
But also, isn’t that the wrong metric to be comparing? We shouldn’t be looking at P(vaccinated | hospitalized) but instead, P(hospitalized | vaccinated) – right?
The topline ~60% of severe infections being vaccinated individuals is oft-quoted by the media, but the linked article talks about the more insidious figure of “67.5% vaccine efficacy against severe disease”, which can be written as 1 – (P(severe | vax) / P(severe | no vax)).
_This_ vaccine efficacy figure is where the author argues that Simpson’s paradox comes into play. The vaccine efficacy figure is easily confounded by factors that affect both the likelihood of severe infection _and_ vaccination rate. In this case, age significantly affects both of these factors — older people are both more likely to have severe infections and be vaccinated, the opposite is true of young people. So to get a valid causal effect of the vaccine on severe disease, the author argues you have to control for age.
The point being even if you look at the “right” quantity P(hospitalized | vaccinated) as you suggest (which is what vaccine efficacy captures), you can get the wrong conclusion due to confounding effects.
Succinctly and accurately explained — thanks!
Maybe the conclusion is not wrong the data is correct and we are still using fear to control the people to do what the government wants them to do. If you look up what is the way we can tell we have herd immunity then it says 70-90% we are at 89% or close to it. Does it get better than that no one is actually using science to come to any real conclusion just politics. Time to get the politics out of science and wo/man up. Keeping a lock down going is not science, no one ever used to lock healthy people up to stop a pandemic.
Gidgamoe:
I think you’re responding to something other than this post or this discussion. This post is not about lockdown, it’s about the probability of severe infection given age and vaccination status.
Apart from finite periods in a couple of cities, the U.S. has never been in “lockdown,” yet we’ve had complaints about “lockdowns” in comments here since March 2020. I was rude to one commenter about it, and Andrew told me to pipe down so I did. But I was right.
I tried to post this over there since the author seems to be closely monitoring the comments, but didn’t feel like signing up. Maybe he’ll see it here.
The elephant in the room is whether vaccination state affects the decision to get tested or put in the hospital. Watch out, hospitalization could be a very misleading proxy for serious cases.
We should also like to see all-cause hospitalizations, not just those labeled as covid-19. Pfizer appears not to influence all-cause mortality, at least in the hyper-healthy population chosen for the RCT:
https://www.medrxiv.org/content/10.1101/2021.07.28.21261159v1.full-text
Finally, there may also be a culling effect of the vaccine in older populations. The Pfizer vaccine is reported to cause lymphocytopenia after the first dose, which lowers the resistance to infection. It is possible that vaccinated are also more likely to have been previously infected (so the survivors have robust immunity, and those that did not survive are not going to be currently hospitalized). At the same time, the elderly who refused vaccination may have done so due to frailty, making a serious case more likely.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7583697/
Anoneuoid –
> It is possible that vaccinated are also more likely to have been previously infected (so the survivors have robust immunity, and those that did not survive are not going to be currently hospitalized).
Isn’t it possible that the unvaccinated are more likely to have been previously infected. (1) people thinking they don’t need vaccination because they already have immunity, (2) people thinking that their reaction to the vaccine would be more severe, or even that it would be more dangerous, because they’ve already been infected (there are misinformers out their pushing those theories) and, (3) people who are more likely to think the vaccines are dangerous or a plot by Bill Gates to implant chips in their arms, etc., are more likely to have previously engaged in behaviors that led to infection.
> people thinking that their reaction to the vaccine would be more severe, or even that it would be more dangerous, because they’ve already been infected (there are misinformers out their pushing those theories)
Maybe this is not what you meant by “theories pushed by misinformers” but it seems that the vaccine produces more frequent side effects on previously-infected patients (for the first dose, not for the second one) in the same way that for non-previously-infected patients the second dose produces more side effects than the first one.
https://ejhp.bmj.com/content/early/2021/07/27/ejhpharm-2021-002933
Carlos –
> Maybe this is not what you meant by “theories pushed by misinformers”…
Thanks for the good faith reading. Yes, what I wrote was ambiguously worded. I meant that people who have pushed misinformation have pushed the theory (or evidence) that reaction (or danger) among the already infected in response to the vaccine would be worse. Not that the theory in itself is misinformation
Anoneioud –
> It is possible that vaccinated are also more likely to have been previously infected (so the survivors have robust immunity, and those that did not survive are not going to be currently hospitalized).
Isn’t it possible that unvaccinated are also more likely to have been previously infected because, maybe:
(1) They avoided vaccination anticipating a more severe reaction by virtue of having been infected.
(2) They felt less of a need to get vaccinated, thinking they already had immunity.
(3) Not getting vaccinated is a behavior associated with more risky behaviors – because of less concern about the dangers of COVID, or because they think the whole thing is a “hoax” set up by Bill Gates to make millions from injecting microchips in your arm, vaccination is just an attempt by libz to control you, etc.
I agree with Joshua. Hypothetical nitpicking can go both ways. Anonyroid clearly has some pro-antivax bias, despite an avalanche of data demonstrating vaccine effectiveness. Anonerude has yet to present any remotely convincing argument in this space.
Unanon –
> Anonerude has yet to present any remotely convincing argument in this space.
We all have our biases. It’s hard to judge bias in others. I disagree with you in that Anoneuoid has provided some good information re vaxes. I have taken issue with some of his logic (such as motive-impugning) regarding vaxes, but I’d rather try to discuss that free from assumptions about biases.
Unanon, this is a Bayesian stats blog. If you ever want to bet on something let me know.
I tried to bet twice so far about utterly obvious stuff being denied.
1) That antibodies would confer immunity when the WHO tweeted there was “no evidence” last year.
2) That cases in Israel would go back over 5k by the end of the year when they dropped near zero 4 months ago. As expected, Israel is nearly at all time highs today: https://www.worldometers.info/coronavirus/country/israel/
No one would take those bets.
You should take issue with claims from those with a bad track record that mislead you. Not me. If you ever want to put your money where your mouth is let me know.
So far, it seems to me that there isn’t really virological data indicating that Delta is significantly more infectious than the dominant strains when you made that bet. Epidemiological data, how we, is consistent with the view that it is.
When you offered that bet, many experts seemed to think that immunity from infection would fade.
Soz although I don’t think it’s dispositive, evidence suggests that rhe vaccines were quite protective against infection with the earlier strains.
As I recall, your bet was offered on a foundation of arguing that the vaccines wouldn’t protect against infection
Damn…
… weren’t protecting against infection from the earlier dominant strains, because you saw no mechanistic explanation, and instead that the drops in infection concurrent to the rollouts of infections were attributable to other causes.
If I got anything wrong there, I apologize and welcome correction.
If I’m right, your claiming victory with your counterfactual bets should receive some more scrutiny.
…rollouts of vaccines, obviously.
how we = however.
I’ve been saying there would be resistant strains since last spring. You would have to deny 150 years of basic biology to think otherwise. And especially for a vaccine vs only the S1 segment of the spike, which is the fastest mutating region of coronaviruses. And even more especially due to no mucosal immunity.
Everything going on is textbook and shouldn’t be surprising anyone.
Anoneuoid –
> I’ve been saying there would be resistant strains since last spring.
Are you saying Delta is specifically a vaccine-resistant strain as opposed to a new strain for which the vaccines are less efficacious?
I mean in a sense it is vsccine-resistant, but that’s a pretty complicated claim, since it seems to have developed in barely-or non-vaccinated India in December.
Also, other strains developed before Delta that appeared to be more infectious – so clearly tying new strains to vaccines is complicated.
I don’t think anyone doubted the evolution of new strains which could be more infectious.
Tying the development of strains to vaccinations seems theoretically possible but as to the plausibility, it seems to me the jury is still out and there seems to be much misinformation being put out there in that regard.
Also, perhaps there were other arguments you made about vaccines that don’t stand as well against the test of time?
Andrew –
I’ll stop now.
Agree with Joshua.
Also, this is a Bayesian stats blog, not a betting blog.
Also, if it were a betting blog, there are many reasons why one would not decide to take a bet. I think Dr. Gelman described some of those reasons during (and regarding) the 2020 election cycle.
I was referring to the older population much more likely to end up in the hospital, in particular those in nursing homes.
As for the younger population, I have to think a lot of these are reinfections after a mild case last year, but we haven’t been hearing much about that aspect.
Natural immunity is still going to wane over a year, just like it does for all other viruses that primarily replicate in the mucosa (flu, rhinoviruses, coronaviruses) vs the blood (measles, smallpox, polio). Anyone who thinks you can’t get reinfected after either infection or especially the vaccine has been mislead.
And this is a good thing, if not your mucus and blood would be so thick with antibodies towards all the different pathogens out there you couldn’t breathe.
“Isn’t it possible that the unvaccinated are more likely to have been previously infected.”
Well, from the article:
“One caveat with any efficacy analyses with the Israeli dashboard data is that the previously infected are not separated out. Note that:
Israel did not allow previously infected to be vaccinated until 3 months into the vaccination campaign (in March)
Then made only optional (given they awarded immunity passports to previously infected even if unvaccinated) and only limited them to one shot.
Given the high vaccination rate, it is plausible that a substantial proportion of unvaccinated were previously infected.”
Yes, there are SO many caveats to sort out, and clearly my simple illustrative analysis does not even come close to addressing them all (I wish I had the full data that would be able to do a more detailed analysis!!)
I agree that of the many sources of bias in these observational data, and certainly factors differentially affecting testing or going to hospital are one. I am not sure if this is as big a deal in Israel depending on how they defined it on this dashboard, but the issue of “hospitalized with positive covid test” or “hospitalized because of covid related symptoms and positive covid test” are two very different things. I assume most hospitals still test everyone coming in.
Also you hit on some other issues that are clearly potentially relevant — the systematic differences in the vaccinated/not subpopulations. The frailty you mention is one key one in the older groups, but in Israel an even more major one is the “previously infected” as I mention in the post — it is likely these are mostly in the unvaccinated and single dose groups given Israel’s distribution strategy, and few in the twice vaccinated, which could attenuate the efficacy results for full vaccination and inflate those for partial vaccination. Just another of the issues that given more data we could potentially adjust and understand better what is really going on.
Your article is very helpful, thanks.
This truly opens up a can of worms but should be fertile ground for statisticians. It was wrong from the start to claim that vaccine effectivness is a magic constant – the same number applies for all subgroups of the population, all types of vaccines and across time. This analysis draws attention to validity problems with prior real-world studies e.g. the matching study in Israel results in a heavy skew to younger people because the high vaccination rates of older people make it hard to match by age, and yet the finding has been broadly applied as if the age bias did not exist.
Andrew –
Thanks for posting this. It’s helpful for those among us who are less statistically literate.
Given the evidence of vaccine efficacy waning, I wonder how the issue of time from vaccination interacts with this analysis. Seems to me that this kind of analysis would only be a snapshot in time, and thus the statistical analysis will necessarily be changing over time.
Yes, it is just a snapshot of “active infections” as of the date of the data 8/15/21. On the positive side, this means they are likely all delta, but on the negative side they may not represent the entire trend over time, and also there may be some other biases in it (e.g. that people hospitalized longer are more likely to be captured in the snapshot so the distribution of severe cases might not quite be representative of the entirety).
The effect of potential waning is one of the most important ones and I didn’t try to delve into that at all here.
But it is possible that the inference of waning immunity might be incorrect — this is very difficult given the systematic differences in those vaccinated early and late in terms of age, risk, exposure probability, attitudes and behaviors, health care access, etc. And on top of that the delta variant took over in the past few months and introduced new dynamics that could interact with the immune system and vaccine mechanism, specifically much higher viral loads.
I’ve seen a few nice analyses from Israel and other places that try to adjust for these factors, but I have not been fully convinced by any of them.
I suspect that the waning is not as people are worrying about, and that it may predominantly affect certain subgroups (especially immunocompromised), and also may affect asymptomatic and mild symptomatic infection rates more than serious disease or death, but I am waiting for more data and time to think before I publicly state an opinion and maybe put out a blog post on it.
Jeffrey –
Don’t know if you’ll see this – thanks for that response (I missed it earlier).
I’ve seen some recent talk from anti-vaxer types kind of another possibility – that what appears to be the impact of Delta might actually be an impact of waning.
Although the source is people working overtime to downgrade the efficacy of vaccines, it does seem at somewhar plausible conjecture to me – but I lack the expertise to have much confidence about that.
I have heard some experts question the practice of trying to evaluate the attributes of new strains, such as virulence or infectiousness, by looking at epidemiological data as opposed to microbiology-based research. That also seems reaonable to me, given the many potential confounding variables uncertainty in the epidemiological data.
I haven’t looked at this data but did some back of the envelope calcs from reporting in the news media. I was looking at a different question, which is what is the vaccine effectiveness against *infection*. What I came up with was that it’s not particularly high, like 40-60% or so.
And I think there needs to be some communication about this, such as “the vaccine is very effective at keeping you from getting serious sick, but was really never intended to keep you from being infected in the first place” and “the pandemic will end once most everyone has been infected, so you should get the vaccine so that your infection likely doesn’t lead to serious consequences”.
This is as I understand it, basically expected. As someone pointed out here a week ago or so, IM injections don’t induce IgA antibodies which are in the mucosa, they induce IgG antibodies that circulate in the blood. So the immune system doesn’t really start attacking the virus until after it’s initially infected the airways and starts to spread more widely. At that point, you expect the vaccine protection to kick in and the viral load to clamp down hard compared to unvaxed. And that’s exactly what’s seen in viral load data.
Daniel –
> IM injections don’t induce IgA antibodies which are in the mucosa, they induce IgG antibodies that circulate in the blood. So the immune system doesn’t really start attacking the virus until after it’s initially infected the airways and starts to spread more widely. At that point, you expect the vaccine protection to kick in and the viral load to clamp down hard compared to unvaxed. And that’s exactly what’s seen in viral load data.
Thanks for that clear mechanistic description. Yay mechanistic descriptions.
> was really never intended to keep you from being infected in the first place
Wait what? Yes it was in multiple ways (protect individuals, and by your own estimates you give it 40-60% chance of doing that, stop the pandemic so less people get infected, etc).
It lets you avoid being infected in any given exposure, but as time goes on you’ll get more and more exposures, eventually either you’ll be infected, and have a mild case, or you’ll be one of the lucky few percent who avoid being infected entirely. Now that Delta has base R0 ~ 5-6 or so, we expect all but 1/5 or 1/6 of people to get infected in the world before we stop having “booms”. so basically 80% of everyone will eventually have been infected. Even if 70% of everyone got the vaccine before they were infected.
R among vaccinated is probably somewhat less than R among not vaccinated, but it’s not a lot less, it’s not dropping a 5-6 to a 2, it’s maybe dropping 5.5 to (1-.4)*5.5 = 3.3 so yes it does reduce the chance of eventual infection from 81% to say 1-1/3.3 = 70% but that’s not the primary reason to be vaccinated. The primary reason to be vaccinated is so that inevitably when you do get sick you have instead of a say 10% chance of hospitalization maybe (1-.95)*10 = 0.5% chance of hospitalization and instead of a 1% chance of death you have (1-.99)*1 = .01% chance of death.
Those are BIG effects, the reduced risk of infection from ~80% to about 70% is not.
Daniel –
> among vaccinated is probably somewhat less than R among not vaccinated, but it’s not a lot less.
That may be something of a moving target, no? On the one side we have what may have turned out to be waning immunity, and an increase in vulnerability to infection among the vaccinated over time.
On the other hand we have boosters, which may decease succeptability to infection again over time.
An on the third hand we have the wild card of new variants.
Of course on the fourth hand there seems to be a good chance that the virus may run out of new and significant variant adaptations.
BTW – have you seen any info on the virus in terms of comparative infectiousness and/or virulence if its coming from the lungs as opposed to mucosal virus?
I don’t know if I agree with this message. In the sense that you mean, every person on the planet is perpetually being infected by *something* many times a week. Our relationship with microflora in general is much more complicated than most people know, and the colloquial understanding of an “infection” certainly does not capture it.
I literally mean that over the next year or so we will get to the point that say well over 50% of everyone in the US has had some variant of COVID invade their upper respiratory system, begin replicating, and get to the point where they have at least a few thousand viruses per ml of saliva for example. So let’s call this 60-80% of everyone.
Only at that point in time will it be the case that we don’t see booms in “cases” anymore.
Vaccination adoption of ~60% say is still not going to save us from having fewer than 50% and probably closer to 60-70% of everyone with an infection, as determined by viral titer of at least a few thousand per ml saliva for example.
Sure, with high vaccination rate we could hope that of the people remaining to be infected, only a few percent of them will require more than a week of resting, but it’s not the case that more than 50% of us will “get out of this without ever being infected” for example.
I believe that’s true, but I don’t think that’s what people understand when you say infection. Every human body has a trillion viruses replicating in them at any given time, but I wouldn’t say “everyone is always infected by something.” Infection to me and, I suspect, everyone else, means sickness and risk of death.
Then what does “asymptomatic infection” mean? Note I’m not talking about little tiny infections that never go anywhere, like you get 30 virions and they replicate to a few thousand and are then clamped to nothing before going further. I’m talking about a few thousand per ml of saliva which is a relatively light amount but would be definitely a “case” and detected on testing.
The point of treating infection as “presence of a significant quantity of virus” is that’s what’s necessary to spread the thing. It’s also what’s necessary to induce the mucosal immunity to prevent reinfection and continued boom and bust caseloads.
@Daniel
That is not going to happen:
https://journals.plos.org/plospathogens/article?id=10.1371/journal.ppat.1009509
Also as I tell my friends and family. Dose matters. So even if the end result is that you should expect to get this thing, and you’re vaxxed so you’ll most likely be ok, you still shouldn’t **seek out** the infection just to “get it over with” because you’re much better off getting a small initial dose through indirect exposure with a mask on, and get a relatively light initial viral load, and get over the whole thing relatively quickly than you are to seek out someone to cough in your face, get a big ass viral bolus and most likely get a kind of bad case that knocks you on your ass for 2 weeks even if you are infected. So, yeah, everyone will get infiltrated by this horrible virus and have some kind of level of infection, but avoid getting a high dose even if you’re vaxxed.
Unless you are wearing an FFP3 mask, they have no effect on inbound transmission. The data is very clear on that.
Which is hardly surprising given they are operating against suction at that point. Any more than outbound don’t work too well because they are operating against pressure at that point and they leak around the side. Since SARS-2 is likely aerosol transmitted there is limited effect systemically in rooms with lots of people in them, which is why it is difficult to detect any impact in real world data. (The difference with the vaccine is night and day). The viral concentration goes way up very rapidly. This is why the predicted ‘exit waves’ don’t happen, and likely why you get peaks of infection after people have been vaccinated (the vaccination centres are indoors!).
Moreover viral infection is threshold based. Once you are over the threshold and have an infected cell shedding you’ll get a rapid spread of the infection. And everybody’s threshold differs – which along with random chance is why people in the same room often don’t go down with the disease.
Get vaccinated, but belief in the mask is just the same in believing in the pocketful of posies in plague villages. They just don’t work outside tightly controlled environments.
I don’t think that’s entirely right. First off if you’re using an N95, or KN95 (or FFP2 is I guess an equivalent European standard to those) you will have substantial protection from inbound transmission. Data on transmission in hospital settings in the US where they use N95s is pretty clear that they offer substantial protection. I’d say you could expect about a 10x reduction in inhaled viral load. FFP3 is even more substantial than that, but I think FFP2/N95/KN95 is sufficient to see protection.
If you’re using some kind of homemade cloth mask you will have *some* protection. I’d suggest perhaps 2x reduction. And while “threshold” isn’t quite the right thing, yes there’s a kind of sigmoidal shape to the severity vs inhaled viral count, so if you’re getting a moderate dose, and you cut it even in half, it could be enough to make your infection from the “severe” type to “mild” type. (initial dose affects how long your body has to respond to the virus before the viral load is substantial enough to make you very sick, a lighter initial dose should let you clamp down on the virus more effectively before you get super sick)
But your basic idea that the general public has a kind of “superhuman” belief in masks as protection is probably right. I definitely see people acting like “wear a mask, any mask” is enough to make them basically safe in any kind of common environment. It’s not. It’s also why my kids aren’t in in-person school (and they’re too young to vax). I’m not sending them indoors with a bunch of other people in the same room even with open windows, or HEPA room air filters. I’m sure all those mitigation measures will help, but they’re not a silver bullet. I expect schools around here to have substantial transmission. I guess they’re now requiring weekly testing for under 12 year olds, which should help substantially.
Daniel –
> But your basic idea that the general public has a kind of “superhuman” belief in masks as protection is probably right. I definitely see people acting like “wear a mask, any mask” is enough to make them basically safe in any kind of common environment.
What does the look llike? I don’t know anyone who thinks wearing a mask makes them impervious from infection and I kinda doubt many people do, even if I don’t doubt that at the same time some people might over-estimate their efficacy.
Lucan –
> Since SARS-2 is likely aerosol transmitted there is limited effect systemically in rooms with lots of people in them, which is why it is difficult to detect any impact in real world data. …They just don’t work outside tightly controlled environments.
>>…but belief in the mask is just the same in believing in the pocketful of posies in plague villages.
I’m not sure what you’re looking at to give you such confidence in those views.
From what I’ve seen.
It’s at least reasonable that even non-N95 masks can have some benefit both inbound and outbound with respect to “droplet” transmission, and given that the boundary between infectious droplet transmission and infectious aerosol transmission is pretty arbitrary, it seems to me it’s not terribly scientific to argue that even thinking masks have some benefit against aerosol transmission is tantamount to believing in the efficacy of a pocket full of posies.
I’ve read that wearing a mask may block some degree of transmission even among smaller aerosolized particles, may increase humidity behind the masks and alter airflow in such a way that they might limit the distance of travel for the larger droplets.
Even a relatively marginal decrease of risk at an individual transmission level compounds across the population level. What is this evidence that makes you so certain that cloth masks have zero marginal benefit at the individual event level?
> Since SARS-2 is likely aerosol transmitted there is limited effect systemically in rooms with lots of people in them, which is why it is difficult to detect any impact in real world data.
Certainly, while what you refer to there is important, there are many reasons why it’s difficult to obtain dispositive real world data on the efficacy of masks.
well maybe not totally impervious but you see or hear people saying things like “If only they’d worn a mask” about people they knew who were infected, or “I always wear a mask to go shopping and I feel pretty safe” etc. I think a lot of people overestimate the efficacy of masks. Like they might cut the viral load you get in half, but not 100x. And with Delta putting out 1000x the virus, a mask today is way worse than no mask last summer.
The common masks (cloth, surgical) have negligible effect on the aerosols that fill the room. What they do is redirect them to concentrate around the head instead of getting sprayed outward.
A good recent paper here:
https://aip.scitation.org/doi/10.1063/5.0057100
Daniel –
> I think a lot of people overestimate the efficacy of masks.
I don’t disagree. I wouldn’t know how to measure that problem, and I haven’t exactly heard people making comments like those in your example, but I certainly have heard people speaking in ways that suggest they miss the putative main benefit of wearing a mask – to reduce risk outbound as compared to reducing risk inbound
One aspect of transmission that so many miss is that even if most masks poorly block aerosol transmission, they do block “physical” transmission — i.e., it is commonplace to pick up the virus on one’s hands and then rub or otherwise touch your hands to your mouth or nose transferring the virus thusly — a mask largely blocks this behavior which occurs many, MANY times per day (of course rubbing eyes with hands is also commonplace and few are wearing adequate eye coverage).
For some reason I never figured out (not sayin it is wrong), it has been decided that fomites are not very important for covid. Anyway, how does putting on/off a mask multiple times a day block the behavior of touching your face, in fact you would be expected to do it more often? Not to mention the mask *is* a fomite…
Anoneuoid-
The paper you sourced actually opposes what you’re saying:
“The near-field velocity measurements indicate that the forward momentum of breath exhaled through the nose is reduced significantly and redirected when the subject is equipped with a mask. Furthermore, this attenuation of the forward momentum increases with the filtration efficiency of the mask material when a proper fit is ensured. Thus, the present results endorse the use of high-efficiency, unvalved masks with a proper fit when the recommended social distancing guidelines cannot be maintained between individuals.”
Sam R.,
Can you quote what I said that you think disagrees with that quote from the paper?
> there needs to be some communication about this, such as “the vaccine is very effective at keeping you from getting serious sick, but was really never intended to keep you from being infected in the first place” and “the pandemic will end once most everyone has been infected, so you should get the vaccine so that your infection likely doesn’t lead to serious consequences”
In the spirit of the recent post about admitting mistakes, maybe it’s not a good idea to say now that it was never about vaccines protecting people from getting the disease and passing on the pathogen, breaking any chains of transmission, achieving herd immunity, etc.
“Will we ever get to herd immunity? Yes—and hopefully sooner rather than later, as vaccine manufacturing and distribution are rapidly being scaled up. In the United States, current projections are that we can get more than half of all American adults fully vaccinated by the end of Summer 2021—which would take us a long way toward herd immunity, in only a few months. By the time winter comes around, hopefully enough of the population will be vaccinated to prevent another large surge like what we have seen this year. […] Prolonged effort will be required to prevent major outbreaks until vaccination is widespread. Even then, it is very unlikely that SARS-CoV-2 will be eradicated; it will still likely infect children and others who have not been vaccinated, and we will likely need to update the vaccine and provide booster doses on some regular basis. But it is also likely that the continuing waves of explosive spread that we are seeing right now will eventually die down—because in the future, enough of the population will be immune to provide herd protection.”
https://www.jhsph.edu/covid-19/articles/achieving-herd-immunity-with-covid19.html
Yes, I agree that many people, even in biology and public health, have not been aware of the issue that injections typically don’t provide mucosal immunity. But I’m pretty sure that people who work on vaccines actually knew this all along. I believe that Pfizer and Moderna both chose to have their trial outcomes be “symptomatic COVID” on purpose. They knew that if they tested everyone weekly it would both be expensive, and show that their vaccines didn’t prevent initial infection particularly well.
They (some people, at some point, from those companies and from the FDA based on their filings) also claimed likely efficacy against asymptomatic infections.
For example https://www.pfizer.com/news/press-release/press-release-detail/real-world-evidence-confirms-high-effectiveness-pfizer
Rule number 1, never let the scientists anywhere near the press releases.
Rule number 2, see rule number 1
+1
Daniel –
You might find this interesting:
> Conclusions We conclude that rare vaccine breakthrough infections occur, but infectious virus shedding is reduced in these cases.
https://www.medrxiv.org/content/10.1101/2021.08.20.21262158v1
I agree the overall protection vs infection (and whether waning) is an interesting problem but not addressed here.
To your immunological interpretation, I might add that the other issue is circulating IgG vs. memory B-cells. The former decline over time after vaccination, while the latter seemingly do not. So to build on your hypothesis — one reason for potential waning (especially against asymptomatic/mild symptomatic disease) is that the IgG when circulating get there even sooner, but after several months there is less in circulation and the B-cells have to produce more (which they may do very effectively, thanks to their existence due to the vaccine) — and this may take 2-3 more days to build up antibodies.
Thus the vaccines may be doing their job in producing a strong and lasting immune response when exposed to SARS-CoV-2 and effective neutralization, but it may be a little slower later on because the reduction of circulating IgG requires production of new ones.
Jeff Morris’s explanation hinges on what has often been called “the ecological fallacy.” The line connecting centroids goes one way while within each group, the line goes the other way. Simpson’s paradox is the ecological fallacy with only two centroids. But, I have always been mystified as what this has to do with the now common term, ecological. I did find this on the web
“In 1950, Robinson coined the term ecological fallacy to refer to the error of interpreting variations in environmental settings as variations among individuals.”
Somewhat incidentally, I am very much annoyed at the fact that statistics like “60% of currently hospitalized are vaccinated” are swept under the rug. The vaccine skeptics I talk to are not too stupid to understand Simpson’s paradox, but when I look for strong statistics supporting the vaccine in conversations like these, all I see in the media statistical tricks (maybe “statistical crimes” is more accurate) to make it seem like P(hospitalized|vaccinated) is lower than it actually is (*). This means that anybody with a bit of statistics knowledge following the media thinks the media is full of shit, and since many vaccine skeptics are not stupid, they are left with a feeling of everyone trying to trick them with bad statistics. It really is frustrating.
(*) For example, this article from the NYT https://www.nytimes.com/interactive/2021/08/10/us/covid-breakthrough-infections-vaccines.html , which simply looks at the proportion of the vaccinated in hospital “since start of vaccination campaign” which is obvious nonsense because for a long time only very few people were vaccinated)
Sir, the question that matters to you and to me is not the probability that a given “case” is vaccinated; but the probability that a vaccinated person is a “case” ! The two are related, but through terms such as the prevalence overall of vaccination and of cases (which are themselves are not so easily estimated across the board and dynamically).
As you well know, if everyone were vaccinated, then *all* the cases (hospitalized or otherwise) would indeed be vaccinated.
Not sure why I’m a “Sir”, but ok :)
I agree, though I’ll nitpick and add that I care less about whether they are a “case” and more about whether they get sick/have lasting problems.
Sorry, mum!
I completely agree. This frustrates me too more than anything else in the pandemic.
When the media representing the scientific communities (and even scientific leaders sometimes!) play tricks to try to fool people into taking the vaccines (a noble goal!), they lose credibility for themselves, and for the scientific community.
As a result, the people who are hesitant/on the fence feel like “both sides” are misrepresenting the data to fit their own narratives, and they just choose to side with their own political side, which they trust. And this for many people involves believing the vaccine alarmists and covid deniers.
That is, the lack of transparency pushes them into the arms of people spreading negative misinformation.
I have seen this happen again, and again, and again ..
We need better scientific communication coming from the top, and coming through the media (and of course we need statisticians have a “seat at the table” and more visibility to make sure these quantitative nuances are taken into account and clearly communicated)
Here’s one: a month or so ago I found that the LA County public health department (and probably others) were broadcasting the emphatic figure that some 99.xx% of all cases were among the unvaccinated. The footnote to that figure was that the denominator consisted of all cases between December 7 and July 7 ! Bloody prattling horses-arses are we? They might as well have made it even rosier by including in their pretty denominator all last year’s cases too. These are the MD/PhD/MPh crowd who bloody know better. But what they do *not* know (and what they cannot be blamed for not knowing) is how to lead. It’s not their job after all. But it is someone’s god-damn job isn’t ? At any rate, I write to them, Dear Dr. So-and-so, haven’t you considered including in your daily columns of figures the comparable figures for the more *recent* interval; and so on and so forth. Of course insofar as I haven’t got the MD/PHd/MPh cachet they haven’t got the time to bother even responding with a yes-sir we ought to consider it, or no-sir you are sorely mistaken. But they do have time to stick in-between their voluminous daily notes on the break-down of cases into categories of ethnicity and age and neighborhood and so on, the inevitable simpering, nauseating paragraph that Dr. So-and-so extends her “deepest sympathies to [those of us who have had loved ones] ‘pass away…” But they cannot be bothered to provide updated numbers relating in some way shape or form to P(C|V) which is what the horse’s arses like myself are really interested in knowing something about — or ought to be. P(V|C), P(V) and P(C) on a weekly or biweekly or even monthly schedule would be a start. But the daily lachrymose paragraph “extending their deepest …” instead of actionable, useful data takes the cake. Nauseating!
Matty –
> they are left with a feeling of everyone trying to trick them with bad statistics.
I would imagine that not stupid people with a bit of statistics knowledge would know that there’s likely a pretty low limit to the level of statistical sophistication embedded in most media reports on the pandemic.
Thus “…statistical tricks” and “trying to trick them,” might be the conclusion that some vaccine skeptics arrive at, but they might also realize that it’s a bad idea to attribute to malice that which can be explained by less that sophisticated grasp of statistics.
Why would a vaccine skeptic arrive at an attribution of malice when an attribution of a lack of a sophisticated grasp of statistics might suffice? I’d suggest to you that there might be more in play than meets the eye, and that people carrying a hammer ‘oft times do indeed find a nail. So blaming media for their conclusions might only be partially helpful.
@Joshua: I almost missed your reply since it wasn’t under my comment.
> that there’s likely a pretty low limit to the level of statistical sophistication embedded in most media reports on the pandemic.
That’s a nice way of putting it, I personally would phrase it as “most media reports on the pandemic are full of shit”. That said, personally I had higher expectations from the NYT.
> but they might also realize that it’s a bad idea to attribute to malice that which can be explained by less that sophisticated grasp of statistics.
Well maybe, but it’s the NYT; I find it really, really hard to believe they can’t hire a statistician to statistics-check themselves. I just don’t think they really care about the truth in this case. I can’t blame a vaccine skeptic for concluding “elites don’t care about the truth, so why should I trust them on the vaccine?”. FWIW, I was looking for quite a long time for good statistics as to real-world vaccine effectiveness, I thought I had found them when I read the NYT article but didn’t look at their data closely enough. At some point even someone like me starts to wonder whether the vaccine is as effective as claimed given that the most high-profile data is useless.
Matty –
Your arguments reads to me as a bit too conspiratorial. There’s a gap between about whether reporting overplays vaccine efficacy and thinking “they’re trying to trick us.” Or, “They don’t care about the truth.”
There are many potential reasons why the reporting might overplay vaccine efficacy short of those conclusions, particularly given that COVID is a fast-moving and enormously complex topic.
In my experiences of observing debates over climate change, which I consider to be a VERY close parallel to debates over pretty much anything COVID related, people who are pretty much predetermined to think that the NYTimes is out to trick people and that the NYTimes doesn’t care about the truth will reach that conclusion pretty much independently of what the NYTimes actually prints.
IMO, the number of people who aren’t already inclined to go in that direction, and who see the problems in reporting you’re talking about, and *after that point* go to “maybe they’re trying to trick us because they don’t care about the truth* is probably very small in number, and waaaaay dwarfed by the overall community of anti-COVID vaxers.
As a parallel, there are a ton of climate “skeptics” who said that “Climategate” caused them to believe that they were being sold a bill of goods on climate change. My guess is that only a tiny fraction of them weren’t similarly oriented on teh topic prior to “Climategate.” Evidence shows that all of them share a rather specific ideological orientation. But still, when asked, they will say that it was “Climategate” whot dunnit.
Yikes.
That should read… “almost all of them share….” That was a bad (and un-intended) over-generalization.
Someone needs to mention base rate fallacy or base rate neglect (better IMHO) so that searches can turn up the right page. :)
So some latent thoughts having read some great comments:
1) The idea that vaccination was never about infection but about hospitalization is a bit fallacious. Funny though, the latter is an endpoint in many studies (which of course is most important). Still, so long as a virus is allowed to reproduce, it will have progeny with various types of mutations (because viral RNA/DNA replication isn’t perfect for a variety of reasons). The majority are usually ‘duds’ but it only takes that *one* new mutation that has a pathogenic effect to then multiply and cause trouble. So incidence of infection has to be an equally important endpoint, if for now, a secondary one.
2) It’s not always about ‘dose’ (but it does matter in some folks). Yes we are getting new data suggesting the attentuation of immunity after an initial vaccination in Pfizer (some preprints suggest Moderna may be a bit better, but more work needs to be done here). But there’s another variable to muddy the intervention waters a bit (which I think was previously alluded to in another post). The vaccination assists what we generally refer to as the ‘humoral’ or blood-based immunity. It doesn’t address immunity with respect to the nasal mucosa (and I would argue as well the mucosa of the throat, bronchi, and lungs), which is in general medicine believed to be a first line of defense against infection.
– You might know this from having been given the option of a nasal vaccine versus an injection. Similarly, nasal route vaccines are also being investigated, perhaps as a monotherapy, perhaps as an adjunct. We don’t know what fruit that will bear yet, if any.
– But ‘route’ is nevertheless a question we must ask in addition to ‘dose’ in addition to ‘booster’, in addition to addressing societal norms surrounding how various groups will determine how they protect themselves (or not) in the presence of a pandemic event.
3) I mentioned the term ‘adjunct’. Interestingly studies involving current vaccines in a ‘mix-and-match’ scenario are underway (i.e. the trials surrounding a vaccine series which involves one mRNA vax and one DNA vax. The most discussed studies in the UK right now involves a vaccination series consisting of Pfizer and then some time later issuing the AZ vaccine or maybe the other way around – who knows?). What the actual results might be in mix-and-match scenarios such as this are yet not completely clear. More work on this at least is warranted given what had been suggested thus far. And still the issue of future coverage for emerging variants remains as well.
There aren’t magic bullets here, at least not that anyone is aware. And one study over time tends to correct another as more data are collected. This is where the dangerous business of course correction comes in – both in the data collection process as well as experimental design and inference. Keeping this entire ball of yarn into a single frame as it rapidly changes is difficult at best. Setting public expectations on how the yarnball of knowledge is evolving even more difficult.
Funny how I still happen to give this virus the term of ‘novel’. I still feel we are in the adolescent stages of developing knowledge about this virus, its pathogenic forms, and how the world community can be protected from it and its sequelae. That last tidbit is, as we’ve found, as much a social and political experiment as it is a scientific one.
August 23, 2021 at 3:14 pm
“Setting public expectations on how the yarnball of knowledge is evolving even more difficult.”
Like the captain of a ship has to do — to communicate something to keep the passengers from creating a chaos on-board, when the ship’s in the face of terrible seas; first of all, the passengers need to know the ship’s being piloted; they need to know that the course will change as conditions change; and what they must be reminded outright — and it is the brute truth and it is helpful to remember it: if anyone can get a ship through a terrible storm like this one, it’ll bloody well be a ship’s captain who’ll manage to do it!
According to current UK government data about 80% of those dying with covid-19 mentioned on their death certificates are double-vaccinated. On the surface this looks bad for the vaccines but as we know Simpson’s paradox is invoked to ‘show’ that the vaccines are still reasonably effective. However, I argue that Simpson’s paradox is actually creating an illusion that the vaccines are much more effective than in reality and that this apparent effectiveness is due to the fact that younger people tend to be fitter and healthier than older people and so less vulnerable to the virus.
Consider what would have happened had we vaccinated only the over 70s and the vulnerable then the effectiveness of the vaccines would not look so good (probably around 50% according to the UK data) and age stratification would not be able to ‘correct’ for this since nearly everyone vaccinated would be in the over 70s age category. But now consider adding to this data set younger vaccinated people who are hypothetically 100% resitant to the virus with or without a vaccine. Then we could invoke Simpson’s paradox to show that the vaccines are more effective than they really are. Of course, younger people aren’t 100% resitant to the virus but resitance does increase with decreasing age. I believe that the risk/reward calculation for younger people has been greatly misjudged and that a policy of vaccinating the over 70s and vulnerable groups would have been by far the best approach. I also believe that the reason why many highly vaccinated countries are still struggling with the pandemic is, as I have argued, that the vaccines are not nearly as effective as claimed.
Derek:
Just to clarify: I don’t think it’s quite right to say that Simpson’s paradox is invoked to show that the vaccines are effective. Rather, there is direct evidence the vaccine is effective. Simpson’s paradox is invoked to explain how it is that the vaccine can appear to be ineffective in aggregate data, even though it is effective for individuals.
You’re making a different point, which is that the risk-benefit calculation for vaccination is different at different age groups. The key here is to look at both sides of the equation. If risks are essentially zero, then vaccination makes sense because there are benefits: people under 50 can and do get severe covid cases, also they can transmit covid to older people, also mass vaccination has an economic benefit in moving society back to normal. For example, most teachers and school employees are under 70 years old, and it’s important to get kids back into school. You might personally prefer a world in which everybody under 70 just gets covid and doesn’t worry about passing it on to other people under 70–the way we generally act with the common cold—but I don’t think that’s gonna happen. Covid has already killed lots of people, which has motivated most people to get vaccinated once the vaccine has been available. Yet another possibility would be to not give vaccines to under-70s in the U.S. and instead give these vaccines to elderly people in poor countries. That could be a good idea in terms of total welfare, at least in the short term, but: (a) I don’t think this would be politically possible, and (b) arguably, a strong U.S. economy is good for the world more generally.
Derek –
> I also believe that the reason why many highly vaccinated countries are still struggling with the pandemic is, as I have argued, that the vaccines are not nearly as effective as claimed.
As I cruise around Worldometers, I see a LOT of countries where the cases to deaths (and presumably serious illness) ratio was much, much lower (fewer cases per death) early on in the pandemic. On the other hand, we have a country like Russia, where it seems relatively few are vaccinated, and where that shift in the ratio doesn’t seem to have materialized.
Of course, these data are very, very noisy. Texting rates vary widely. Counts are fairly unreliable. Treatments outside of vaccination have improved. Factors like NPIs and mask-wearing may well play moderating/mediating roles in that ratio, etc.
But in the end, I think that your statement that the epidemic data indicated that vaccines aren’t as effective as “claimed” needs some fleshing out. “Claimed” by whom? What statements, exactly? Are these “claims” about protection against serious illness and death, or against infections?
The data on this issue are very complex. Getting good samples to compare (i.g. controlled for age, testing rates, or other correlates) is extremely difficult Thus, I think we should be very careful about reaching conclusions such as the one you stated.
> According to current UK government data about 80% of those dying with covid-19 mentioned on their death certificates are double-vaccinated.
It would have been super nice if you had given the data source.
https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/bulletins/deathsinvolvingcovid19byvaccinationstatusengland/deathsoccurringbetween2januaryand24september2021