A study comparing young and middle-aged adults who got covid to similarly-aged people who were vaccinated

Joe Stover writes:

I came across this article that finds vaccinated individuals are 8 to 21x more susceptible to the Delta variant relative to those with previous covid infection but unvaccinated (i.e. with some natural immunity). It makes sense that natural immunity might be superior, but the result seems so extreme as to be questionable. I’m wondering if there is some identifiable error in their methodology or what potential places where bias could have affected the result. Of course, of its solid, that’s fine too, I just want to know.

I figured this might make for a nice blog post of you find anything interesting to discuss.

The article in question is called, “Comparing SARS-CoV-2 natural immunity to vaccine-induced immunity: reinfections versus breakthrough infections,” and it is by Sivan Gazit et al.

They have data from an HMO in Israel and they do three comparisons of vaccinated and previously infected people, in each case matching on age, sex, and socioeconomic status. In one study they also match to the time of first event (vaccination or infection), in one study they don’t do that matching, and in one study they matched previously infected individuals to previously-infected-and-once- vaccinated individuals. They had 32000 people in their first study, 46000 in their second study, and 28000 in their third study. I guess there was overlap among the people in the different studies. There are certain rules about when people could be infected to be in the study; I don’t have a great sense of the effects of these rules on their results. The paper says the HMO has 2.5 million members so I guess there must be more members who got covid than were included in the study.

The outcomes they looked at were documented RT-PCR confirmed SARS-CoV-2 infection, COVID-19, COVID-19-related hospitalization, and death.

They did some logistic regressions but since they went to the trouble to match on age, sex, and SES, I thought it would make sense to focus on the raw results. Here’s the result for study 1:

During the follow-up period, 257 cases of SARS-CoV-2 infection were recorded, of which 238 occurred in the vaccinated group (breakthrough infections) and 19 in the previously infected group (reinfections). . . . As for symptomatic SARS-COV-2 infections during the follow-up period, 199 cases were recorded, 191 of which were in the vaccinated group and 8 in the previously infected group. . . . Nine cases of COVID-19-related hospitalizations were recorded, 8 of which were in the vaccinated group and 1 in the previously infected group. No COVID-19-related deaths were recorded in our cohorts. . . .

Study 2 gave similar results, and study 3 found slightly better results among the people who had both past infection and vaccination, compared to those with just past infection. Nobody died in any of the groups, which I guess doesn’t tell us that much, given that (a) everyone in the study was vaccinated or had previously been infected, and (b) only about 5% of the people in the study were over 60 years old. I don’t quite get that last bit. Didn’t lots of over-60s get covid? Maybe they’re not in this particular health plan?

In answer to Stover’s question above: no, I don’t see any statistical reason to question the claims, other than possible concerns about selection of who was included in the studies. The estimate of 8 to 21 times the risk (or, as the authors quaintly put it, “8.08 to 21.11”) just applies to this particular study; we’d expect to see different results in different places or times, but it is what it is. If you really get 238 infections in one group and only 19 in the other, then, yeah, that’s a big difference! It’s all descriptive, so this difference could also be explainable by differences in behavior, so the key is to appreciate the descriptive result for what it is. It’s great to have causal estimates when available, but describing what’s happening is important too.

The other thing that makes it hard for me to evaluate this article is that there are so many papers coming out, and I don’t have a clear picture of what’s going on in different places in the world. For that, I like to listen to science reporters such as Ed Yong, whose job is to keep track of everything. Here’s his article from a couple weeks ago, “How the pandemic now ends.”

63 thoughts on “A study comparing young and middle-aged adults who got covid to similarly-aged people who were vaccinated

  1. How was A/B split? The article tells us the distribution of ppl that get reinfected with covid, but not the starting populations.

    Were there more vax vs naturally infected in the study?

    • Maji:

      You can follow the link to see all the details (or, at least, all that I know). They had equal numbers of vaccinated and unvaccinated people in each study. The equality was by design because they did one-to-one matching. More efficient designs would be possible that would use all of the data, but the differences between the groups were so large that I guess efficiency wasn’t really a concern.

  2. > I like to listen to science reporters such as Ed Yong

    You should not trust Ed Yong to interpret scientific literature correctly. Evidence here and here. I contacted Yong about his errors. He never replied nor corrected his Atlantic article.

  3. When does a study cross from descriptive to causal? Only when a randomized trial is involved? It seems pretty clear they did the steps to want to draw causal conclusions from it. If you wanted to make causal conclusions from this data, what would you do different?

    • Jackson:

      Randomization is not necessary for causal inference. What I’d say is that all studies are descriptive, and then you can figure out what casual conclusions can be drawn. It’s not a matter of “crossing from descriptive to causal,” and I don’t mean “descriptive” as a put-down. We take the description that we get, and then we draw causal inferences to the extent that we can. In this case, the comparisons between the two groups in each study seem pretty clean. One concern is selection (who’s included in the study); I don’t see any obvious large selection problems, but there can be things I didn’t think of. The other concern is that, as discussed in the above post, the two groups could be behaving differently. I could imagine that people who actually got covid might behave more carefully than people who were vaccinated and didn’t infected. The infected and vaccinated people could be living in different places. Or maybe this isn’t a big idea. As I wrote above, I think the description has value.

      If you wanted to make stronger causal conclusions from these data, one approach would be to gather more data on the people in the study; another approach would be to do some modeling and calculations to see how much the causal conclusions would change based on some range of reasonable assumptions about differences in behaviors among the two groups.

  4. A big thing I wonder about is *detection* of infection. If you had it before are you more likely to get a mild case, and to believe that you couldn’t get reinfected, so not get tested, compared to say vaccinated individuals who maybe get somewhat less mild cases, and are therefore more likely to get tested?

    If you had a population that was being tested regularly (like say, all athletes in a sports league, or all students at a univ. campus), you’d be much better off for this kind of study.

    Still, it’s a striking number.

    • Daniel:

      Yeah, I found out I got covid because I had to take a covid test for an airline flight. After I tested positive, I was like, ok, I guess that one-day fever wasn’t just a cold. But some people I’d been with had temporary loss of taste and smell, so I guess maybe I would’ve figured it out. If we hadn’t had to get tested and none of us had had specific covid symptoms, we would’ve had no idea. So complicated selection issues here.

    • I’m inclined to think that without a measure of testing bias the result is uninteresting. At the very least, you would need some kind of measure of access to testing. For example, people who live in more urban areas may have easier access to the vaccine and easier access to testing.

  5. Thanks for taking the time to look at and comment on the paper. From your comments, I figure the best way to look at it is that this is just a descriptive study on a particular regional dataset and doesn’t really justify making a general claim about unvaccinated natural immunity vs vaccination alone. Clearly the population included isn’t representative at all of, say, the general US population (and probably not globally either).

    • Joe:

      For reasons discussed in the above comment thread, I’d rather not say “just” a descriptive study. Description is what we’ve got. Also I’m not sure what justifies making a general claim. This study seems like it should be part of whatever story we all come up with. I’m not quite sure the direct policy relevance—it still seems like a good idea for just about everyone to get the vaccine—but it does seem relevant to know who’s getting covid.

      • Andrew:

        The “just” is just because I don’t know all the types of studies there could be.

        I don’t mean to sound dismissive of this study as it is indeed a real part of the story. I’m just interested in the general question of natural immunity vs vaccination, and on how this study should feed my intuition on that matter. My takeaway is that this study does very little to approach such a general intuition.

        In my non-professional-statistician-mind, what would justify a general claim, might be if we had some high quality large (100 ish?) body of studies that largely (95/100 ish?) found that natural immunity was highly superior. It still wouldn’t be conclusive, but it would make a good case.

        • It would take 100 studies to change your mind?? You must have one strong prior!! But also, shouldn’t the amount of data, not the number of studies be the relevant influencer?

          I don’t think you need 100 studies. I think you just need studies to round out the questions this one leaves open: are other nationalities similar? Is risk taking among groups a factor? What about age groups? Stuff like that.

        • As long as the overall body of work is high enough in quality, fewer studies would suffice. My thinking is that the result is so extreme/unexpected (natural immunity is an order of magnitude better than a vaccine) that I would want to be cautious. Then again, maybe this is the expected result for someone who understands the microbiology and medical science etc.

          I’m not a statistician, but usually for complicated health phenomena, I find that when there is a large number of studies, they often find results that don’t agree, so what I look for is an overall pattern from a metastudy perspective, and if that exists, it usually convinces me of the overall direction of the general phenomenon at the full population level. Like dairy consumption and cancer, there do seem to be some real population-level patterns where it increases rates of some cancers and decreases rates of others (via metastudy analysis), but individual studies go both ways all around.

          Then again, I could be thinking about this completely incorrectly!

        • I think actually this is the expected result as long as you’re discussing *infection*. For example intramuscular injections don’t induce much IgA antibodies, and those are the first-line defense in the mucosa against initial infection. So, the IM injections are primarily about keeping you from having much in the way of symptoms, and definitely keeping you out of the hospital or morgue, but not keeping you from getting that initial day or two or 5 of infection.

        • But I think the study showed that the vaccinated individuals had higher rates of symptomatic cases too, at similar relative rates: like an order of magnitude higher.

          It’s the expected result in that natural immunity is better, but should it really be expected to be *that* much better? Maybe after a much longer timeframe it should (1 year or longer?). They didn’t give much detail on the “time of first event” variable, but, from their Fig 1, I might deduce that when matching time of first event, most individuals will have gotten infected or vaccinated in Dec 20 to Feb 21, so we are looking at 6-8 months later. It is still surprising that the vaccine is *that* much less effective (for an infectious variant) 6 months later relative to natural immunity. Again, if I had proper medical/microbiological knowledge, I might not be surprised at all, but I simply don’t know.

        • A priori you would expect mucosal + blood immunity after exposure to multiple variations on all 29 viral proteins to be far better than only blood immunity vs half (the S1 region) of one protein that is not varying at all. And that sequence hasn’t even been in the wild for half the pandemic because the virus mutated to get around the immunity monoculture.

          These vaccines never sounded like they would induce robust immunity to begin with.

        • I’m less pessimistic than Anoneuoid. The vaccine clearly works very well compared to the alternative which is not “previous infection” but rather “nothing”. Also one suspects that prior infection + vaccination is going to be better than prior infection alone, so you’re still taking the vax, you’re just not falsely believing that it’s going to save you from getting infected. You’ll probably get infected even after vaxxed, so even more reason to get the vax.

          Since testing is related to symptoms, it’s not at all surprising that symptomatic cases are more among the vaxxed group. This is consistent with a model where you mainly detect symptomatic cases. If there were regular testing of everyone in the study that would be far more dramatic, but if you’re relying on people to seek a test, then yeah, whichever group gets more symptoms will seem to be dramatically more infected than the other.

        • @ Daniel

          Think about it this way. A vaccine is basically a warning to your body that a threat is out there.

          I wouldn’t assume that an inaccurate warning is helpful in the end. And the more times you do this the worse it could get. Like police trained to look for a thief based on seeing the same single picture multiple times over the course of months. Then they can’t notice him because he grew a mustache.

          Indeed, this is a well known phenomenon (for 100 years) that manifests as OAS and ADE. People need to stop ignoring hundreds of years of science just because of hopeful beliefs.

        • You’re ignoring the fact that currently hospitals are full of non-vax patients and have relatively few vaxxed patients. So the last 4 months very clearly points to vaxxes prevent hospitalization and death. There is zero evidence for ADE even though say healthcare workers were vaxxed back last December, and many have been working in COVID wards continuously since then, but aren’t having sudden massive enhanced COVID reactions.

          I totally get that ADE is a real thing and is a potential concern, but it doesn’t seem to be an actual real-world concern based on over a years worth of data (including those who got vax in clinical trials).

        • You’re ignoring the fact that currently hospitals are full of non-vax patients and have relatively few vaxxed patients.

          First, I would bet this is inaccurate or misleading. Do doctors tell vaccinated to stay home more often or they aren’t tested or it is just not being reported? Are many of the “unvaccinated” actually partially vaccinated and immunosuppressed, or are they unvaccinated because they had pre-existing conditions? What is the timeframe being used to generate these numbers?

          Anyway, that isn’t what is being reported elsewhere like Israel.

          But either way, one has been looking for ADE where its expected to happen. Which is when there are weak antibodies, because they waned and/or theres a new variant.

          In fact, last summer there were many papers reporting the neutralization curves showing negative % neutralization at high dilutions (low antibody concentrations). Then in new papers the y-axis started getting cut off at zero even though the curve went below, then the methods changed so the no-AB control saturated the signal making ADE impossible to see if it was there. Seems real trustworthy…

          And why does it seem more young people are getting severe covid this year, are these people who had mild undocumented cases last year?

          Finally, the delta 4+ variant created in japan clearly *can* enhance infection under certain conditions in vitro: https://www.biorxiv.org/content/10.1101/2021.08.22.457114v1

          Those are the types of studies we needed last year, not the ones that try to hide any ADE signal or only look in recently infected/vaccinated young healthy animals where no one thinks it would show up.

          tldr; We still have no idea what the ADE risk is.

        • I agree with you that data collection is very very wanting… But I think the signal is strong enough that it’s clear the vax is helpful.

          from ourworldindata.org

          if you look at Israel their cases per day today are at the highest they’ve ever been, but their hospitalizations are around half what they were at the previous peak and have flattened the last week or so.

          The UK has a wave of cases that in total appears to be as big as their winter wave (but a little more spread out into two peaks) but their current hospitalization is less than 1/5 of their peak in the winter.

          It’s incredibly dumb that people can’t seem to figure out what needs to be reported, but what you want is hospitalization as percent of cases in the two categories: vaxxed and unvaxxed… and it’s shockingly not that easy to find that data online or in graph form. In the US that’s probably partly political because CDC doesn’t want people reporting breakthrough cases to them (so the CDC doesn’t know how many breakthrough cases there are, only hospitalizations). But Israel probably has this data, and so do several European countries, including the UK.

          Anyway, I believe that the vaxxes are working as intended, the booster shots may be a good idea prior to another fall seasonal boom particularly in the elderly, and they need to hurry up and approve them for under 12 year olds. I don’t think Antibody mediated enhancement is a major concern.

        • Also see below where close to 50% of hospitalized vaxxed patients are immunocompromised.

          https://statmodeling.stat.columbia.edu/2021/08/29/a-study-comparing-young-and-middle-aged-adults-who-got-covid-to-similarly-aged-people-who-were-vaccinated/#comment-2007547

          indicating that in terms of hospitalizations *among non immunocompromised people* (2.7% are compromised so that’s 97.3% non-compromised) the vaxxes must be working extremely well.

        • Anoneuoid –

          > Do doctors tell vaccinated to stay home more often or they aren’t tested or it is just not being reported?

          So doctors aren’t basing their medical advice on the patients presentation but instead based on their vaccination status? They’re seeing patients sick enough to be hospitalized but aren’t hospitizomf them becise the patients have been vaccinatedo

          Oh, and they’re doing thst here. It not in Israel, of course. Doctors in Isreal aren’t as conspiratorial, I guess.

          > Are many of the “unvaccinated” actually partially vaccinated and immunosuppressed, or are they unvaccinated because they had pre-existing conditions? What is the timeframe being used to generate these numbers?

          Just asking questions right? But you have your answers and THEN you’re just asking questions?

          > Anyway, that isn’t what is being reported elsewhere like Israel.

          What isn’t what’s being reported eslwwhaee like Israel?

          > In fact, last summer there were many papers reporting the neutralization curves showing negative % neutralization at high dilutions (low antibody concentrations). Then in new papers the y-axis started getting cut off at zero even though the curve went below, then the methods changed so the no-AB control saturated the signal making ADE impossible to see if it was there. Seems real trustworthy…

          Prolly a vast conspiracy, looping in medical researchers, virologists, public health officials, and clinicians.

          But when you live in a fascist state you have to kinda expect that.

          > And why does it seem more young people are getting severe covid this year, are these people who had mild undocumented cases last year?

          Some new studies are reporting findings that Delta is more virulent – not based only on epidemiological analysis. I guess a conspiracy would explain that as well…

        • > The UK has a wave of cases that in total appears to be as big as their winter wave (but a little more spread out into two peaks) but their current hospitalization is less than 1/5 of their peak in the winter.

          If this turns out to be sustainable and occurs in other countries, and the worst case horror stores prove to be unfounded, then the vaccines are a remarkable success. Even that much more so if it’s confirmed that Delta is more virulent. Then the lower rate of serious illness per number of infections occurs against a virus that would likely have had HIGHER IFR without vaccinations.

          And even if it’s confirmed that natural infection confers more immunity against infection than vaccines, but the vaccines significantly lower fatality against a more virulent virus, that’s not really that much of a bad news story. It means that a lot of people will have enhanced defense from natural immunity + vaccine immunity with a lower chance of serious illness, as compared to relatively less immunity and higher chance of serious illness from natural infection alone.

          Continuously throwing shade on the vaccines still looks like a very odd thing to do.

        • BTW –

          The anti-vax crowd likes to point to the spike in Israel along with a high vax rate to argue that the vaxes are a profit-driven fraud scheme.

          They used to like to point to the high vaxed Seychelles, Gibralter, and Iceland also to make that case but seem to have been quite quiet about those countries more recently. I wonder why? And I wonder why they never talked much about high vax Chile.

          My sense is that whomever points to whichever country to try to make an example to support a strongly held argument about the pandemic winds up with egg on their face sooner or later. COVID is a bear and its fast moving and there’s still a lot that’s not understood.

        • “I find that when there is a large number of studies, they often find results that don’t agree”

          badabing!

          This is an interesting study for sure but the proof is in repetition and replication and almost never in a single study. It’s often reasonable to choose study constraints that are most likely to generate a detectable or notable result (e.g., “ideal” with respect to some postuatled outcome), with the idea that future work can expand upon that result and if no result is detected just move on. Whether that’s the case here I don’t know but it would be a mistake to accept this as anything approaching the final result. It’s a great preliminary examination, now it needs much more followup work.

        • None of theses studies are interventional studies – like if you took randomized samples of previously infected and non-infected cohorts, vaccinated the non-infrcted group group, controlled for exposure to the virus, and then exposed both groups to the virus under controlled conditions.

          Obviously, that’s not a realistic protocol. But then the results would have limited utility anyway because the conditions were so contrived.

          I think it’s really dicey to think you can confidently assess causality from these kinds of retrospective observations.

          They’re information. They help to assess probabilitie (in real world context).

          If you have lots of such studies it can help to narrow the probabilities. Imo, take any assumptions about “causality” with a grain of salt.

        • That was me (in case it wasn’t obvious. It takes special talent to misspell your own name).

          Anyway, I think what we have with that study are results from a particular location at a particular time. Generalizability to other contexts at other times? Hmmm. Seems to me more studies are needed because then you can see results in other places at other times. Assume they likewise have limited generalizability. Then in aggregate you get information about generalizability because then you can look for patterns in common across a variety of different contexts.

        • “take any assumptions about “causality” with a grain of salt.”

          Jishua: I would accept this as tentative evidence of greater protection from natural antibodies and await further results.

          “Generalizability to other contexts at other times? Hmmm.”

          Joshua: I’m not sure why not right off hand. From the extremely superficial point of view, I don’t see what about the context or the time period that would make it not generalize. But of course that awaits further study. If the study is sound – we should tentatively assume it is – it’s more a question of the relevance of the sample to the total population.

    • “Clearly the population included isn’t representative at all of, say, the general US population (and probably not globally either).”

      Kind of depends on what you need to generalize about. What we are concerned about is mostly biology mediated by social interaction. I don’t think you’re saying the virus is different enough or the population’s genetics are different enough to not generalize, so the main concern for generalizing these results is about social interaction. Well… Israelis have slightly larger families than Americans (3.7 members vs. 3.1). They seem to interact in a similar manner as many Americans. Is there some other reason these results wouldn’t generalize.

      Andrew’s comments about selection are important to consider but apply to most studies, so one could say the same things if the analysis were performed on a US sample. And they are only conjectures. Many vaccinated people I know behave very similarly to many previously infected people I know (though the vaccinated are definitely on the more cautious side of behavior). The follow-up period was June 1 to August 14, a period of time in which and before which the vaccine was widely available to all. Vaccinated people might actually behave more cautiously, which would suggest the results are an underestimate.

      Someone else mentioned the previously infected might get tested less than vaccinate people (again, another conjecture). I’ve heard that same comment in suggesting the differential results (relating to measuring break through infections) between vaccinated and unvaccinated is partly due to the vaccinated not getting tested as much as unvaccinated, so it’s not clear to me testing differences between vaccinated individuals and previously infected individuals would be driving the differences in this paper.

      Given the waning efficacy that has been seen in the Pfizer vaccine, why are large effects between only vaccinated and those with prior infection implausible?

      • I can’t say authoritatively why they are implausible. If the study found the opposite, that the vaccine was 10x better than natural immunity, I would be skeptical of that too (possible more skeptical!). I don’t have any reason to believe the immunity would be so vastly different, in general. Sure it’s possible. Maybe the immune system can identify other viral fragments besides just spike protein when trained naturally, whereas the vaccine only gives ability to identify spike protein. Maybe natural immunity was “harder won” and so is longer lasting. But a few months after vaccination with a viral variant such an extreme result seems unexpected.

        • You know what was 100% effective against reinfection? Dying of the first infection. Also if you had a strong reaction to the first infection, then you’re probably pretty likely to avoid other people and try not to get it again, hence potentially also that’s very effective against reinfection.

          So the group of “naturally infected people” is already selected heavily for people who didn’t have huge reactions to the first infection. For example they may have had cross-reacting antibodies from other coronaviruses that led to them fighting COVID much more efficiently, or they may have immune systems that don’t hyper-react to this virus, and therefore are already likely to have low levels of symptoms. There’s just a lot of complication in all of this.

  6. With the pre delta strain are there any analogous numbers?

    Ie if the differential is 8 to 21x what was it pre delta strain?

    Also, what about well studied diseased like the flu? Does natural infection offer the same huge protection advantage over a vaccine there?

    What’s out prior here? Somehow I am skeptical. 21x sounds like a huge number for effect size.

  7. The conclusion of this article does not mention an important effect of the disease. COVID-19 consequences depend on the overall state of the individual’s immune system, consequently it produces not only natural immunity but also “natural” deaths. The deaths after vaccination are rare and probably random. This means, the weak in the group of individuals “with previous covid” were sorted out. Many of them would survive today, because the new treatment protocols are much better.

    There is no “clean” control group with natural immunity without vaccinations and without previous COVID-19. Consequently, we do not know what part of the better herd immunity was produced by Darwin’s selection.

    An additional problem are the false positives. We do not know what part of the registered individuals with an RT-PCR confirmation were infected by other influenza viruses and were unable to create a natural immunity against COVID-19. The numbers are quite small and this fraction may be quite big.

    It is also reasonable to conclude that “individuals who were both previously infected with SARS-CoV-2 and given a single dose of the vaccine” have more cautious behavior.

    • I think this is important. That the comparison is natural immunity of survivors to general vaccine-induced immunity. Of course, given that the study population was relatively young (only 5% 60 years and older), the overall death rates would have been small. But, still, a number of vaccinated individuals might have died instead of surviving if they were infected instead of being vaccinated, a big unknown. Given the small case numbers and large sample size (~200 out of 14 thousand infections), maybe a small perturbation in something could drastically change the overall effect.

      I’d be interested to see a similar demographic matching study between those who were infected (whether survived or died) and those who were vaccinated. Do a mortality comparison. It would probably be like vaccine is 1 million % less risky for mortality.

      • Unfortunately, the demographic is not important by COVID-19. This is a disease of an abnormal response of a weak immune system. Consequently, COVID-19 is very selective to the health state of a specific person. You cannot make reasonable conclusion about it on base of demographic data only. (This is also means that a decision about vaccination must be made by a doctor and not by a bureaucrat.)

        Unfortunately, a weak immune system additionally means a weak ability to build a long-term immunity against COVID-19 as after a previous infection also after a full vaccination. All statistics I know show that the levels of antibodies rapidly fail in the groups that must be protected the most.

        The percentage of persons with a weak immune system — what usually means chronic illnesses — is small in younger age groups. A chronic illness usually means frequent doctor visits or frequent hospitalizations. These are now the virus distribution centers. This means, any person in a risk group remains to be in danger.

        The Russian health care system is insane, and the doctors visit their patients at home. The unnecessary hospitalizations are also very frequent. These are the most prominent ways of infections of the patients form risk groups. What is responsible for big numbers of COVID-19 deaths.

        Simply speaking, this article profs the obvious idea that letting people get ill can help the politicians to build a “broader-spectrum” immunity among survivors. This is the way of Sweden: If you let the weak people die, the nation gets stronger. Sweden’s strategy has failed because the price to pay is too high, and the immunity is high only temporary.

        This article says nothing about the probability to die or to get infected for a specific person.

        I think, it is impossible to answer this question, because the medical data that could allow to make such predictions is simply not collected.

        Additionally, it is not enough to consider deaths. The problem with the infection is the severe long COVID-19, and the problem with vaccines are unknown long-term side effects that we will learn only in the future. The known side effects are also problematic.

  8. Let’s suppose this effect is true. This would tell us natural protection post-infection is stronger than vaccine-induced protection. What does it change? Does it tell us that it is better to wait to get infected than to get vaccinated? Actually, no, because of the survivorship bias. Those in the natural infection arm are the ones who survived to get compared to the vaccinated.

    • I think it tells us what I’ve been telling my friends and Facebook followers, which is that the pandemic ends when the vast majority of people have been infected, so get your vaccines so that the infection is less risky, because there is no way to reliably get out of this without infection.

      • Given the covid delta variant’s transmissiveness, 2 years from now almost everyone will either have had covid, have had a vaccine, or both. The trick for personal decision making is to pick the least painful path through that chain of states.

      • An infection does not completely prevent a next infection even with the same SARS-CoV-2 version. It also may go to domestic animals and return to humans. The virus will remain with us in the future and new mutations could cause new outbreaks.

        • That’s all true, but it’s unlikely that you’ll have everyone with waning immunity all at once, and hence at some point we’ll enter an “endemic” phase where the danger of overwhelming hospitals is gone… it doesn’t mean the disease is gone, just the explosive nature of the infections.

          If we get a sufficiently different strain that immunity hardly helps, then we’ve got a different pandemic.

        • I think, you are too optimistic. The danger of overwhelming hospitals is only rising. Employees are active leaving intensive stations because of insane state politics and management’s economy measures. The number of ICU beds is falling in most countries.

          A new wave can rise anytime because of cold weather or because of school holidays. The reserves to respond to such rises are nearly absent.

          Additionally, experts say that the current brainless antivirus measures could guide the virus evolution in the direction of the Marek virus or the Dengue virus.

    • There is a big problem with all of this when it comes to natural immunity. Apparently there is a 100-fold difference in antibody titer among naturally infected, and without expensive and complex tests it is impossible to know who has what level of protection.
      That was one of the reasons for suggesting vaccination even for those who recovered from covid.

  9. Biggest and most relevant thing missing in that list is whether an individual is immunocompromised, who make up ~2.5% of the population and ~45% of breakthrough infections. Almost all of the immunocompromised will be in the vaccine group. So divide the vaccinated cases in half. Natural immunity would still be more effective, but vaccination is ~2x more effective than these results suggest.

    • All vaccines have a long list of chronic diseases that prevent vaccination because of high probability of side effects. This means, we must divide both groups on something. It is also very probably that the both groups contain only immunocompromised individuals.

        • To be clear, those numbers are for HOSPITALIZATIONS not infections. The CDC isn’t even tracking breakthrough infections, I suspect because they know it will be many many people and don’t want to have the data make the vaccines look bad.

          Vaxxes are moderately protective against infection, but VERY protective against serious disease. This is their main job when it comes to respiratory viruses

        • Daniel –

          Yes, thanks for the underlining. I forgot that my original question/dubiousness was regarding infections not hospitalizations!

          Still, I had no idea the prevalence of immuno-compromised among hospitalizations high was so high, although admittedly neither did I realize how high the prevalence of immuno-compromised is among the general population.

          Also, from the article:

          > In comparison, the rate of breakthrough cases among vaccinated people who are not immunocompromised was less than 1%.

          That stat always bugs me when given without mentioning the background rate, and because they’re “comparing” apples and oranges.

        • Regarding the efficacy of vaccines inferred from the data in Israel:

          > Now look at the population 50 and older. There are 2.1 million vaccinated Israelis over 50, and 290 were in the hospital Aug. 15. That’s 136 per million, a rate that dwarfs anything younger people are experiencing. And unvaccinated older Israelis? There are very few people in that category: just 186,000. But of that group, 171 were hospitalized — a grievously higher rate of 919 per million. In the older population, vaccinated people were less than one-sixth as likely to be hospitalized as the unvaccinated.

          https://www.washingtonpost.com/outlook/2021/08/31/covid-israel-hospitalization-rates-simpsons-paradox/

        • “As of July 2021, nearly half of the vaccinated people hospitalized with breakthrough COVID-19 infections were immunocompromised – despite making up only 2.7% of the U.S. adult population. In comparison, the rate of breakthrough cases among vaccinated people who are not immunocompromised was less than 1%.”

          Note that the “rate of breakthrough cases among vaccinated people who are not immunocompromised” *does not* specify hospitalization. Mistake or intentional?

        • Why do they give the job of writing articles about quantitative data to people who can’t add up how many fingers they have without using their toes?

          the quality of statistical reporting is so bad I think this is just incompetence as usual.

        • “this is just incompetence as usual.”

          Agreed, but it that does have to be assumed, so it leaves one wondering what’s missing elsewhere.

  10. Excuse the slow response but it has taken me some time to think through the issues and check facts as best I can. The study does not pass the sniff test – which of course is why Joe Stover brought it to Andrew’s attention. I am a biostatistician, my partner an infectious disease specialist. It doesn’t pass her sniff test either. Here’s her sniff test: the rate of breakthrough infections for the data in Model 1 is 238 infections among 16,215 fully vaccinated patients over 75 days; that’s 20 infections per 100,000 person days. She says that has to be an at-risk population. For example, the rate in the general Israeli population is estimated to be 2.1 breakthrough infections per 100,000 person days [1].

    And I think she’s right. Here’s why.[I am just focusing on Model 1 – I have only so much time to devote to this, and I want to be methodical.] Think about the way cases and controls were selected for Model 1. As I understand it – and really it is not clearly written so I could be wrong – in Model 1, the cases are 16,215 individuals who had a natural infection in January or February 2021. Table 1 gives the details – mean age 36, 60% under the age of 40, 5% aged 60 or over. To me, this sounds plausibly representative of the general Israeli population, minus the elderly. The Israeli population is described as relatively young, with only 12% aged 65 or over [2]. So the elderly didn’t as a general rule get infected in those two months. It’s reasonable to assume they were being protected.

    In Model 1, those cases were 1:1 matched to controls by age and sex. The controls were those receiving their second dose of vaccine during January or February 2021. Here’s the crux. “Israel launched its COVID-19 vaccination campaign on December 20th” and “the initial target groups for vaccination would be people aged 60 and over, nursing home residents, other people at high risk due to serious medical conditions, and front-line health care workers” [2]. So my guess is that the controls in the matched sample, receiving their second dose in January or February, were a high risk group – the immunocompromised and health care workers. Note that in one Israeli tertiary health care centre, the rate of asymptomatic breakthrough infections in health care workers was 11 per 100,000 person days [3].

    So I suspect Model 1 is not comparing apples with apples. Younger healthy people are the cases; the matched controls are younger people with poor immune systems or those constantly exposed to the virus – and frequently tested. Andrew’s guess is likely correct (“I don’t see any statistical reason to question the claims, other than possible concerns about selection of who was included in the studies”) and Joe is on the money (“the result is so extreme/unexpected (natural immunity is an order of magnitude better than a vaccine) that I would want to be cautious”).

    Those opposed to vaccination are using this pre-print to support their views (see [4]). Even if corrected in a final published paper, misleading pre-prints can do a lot of damage [5].

    [1] https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(21)00947-8/fulltext
    [2] https://ijhpr.biomedcentral.com/articles/10.1186/s13584-021-00440-6
    [3] https://jamanetwork.com/journals/jama/fullarticle/2779853
    [4] https://www.spectator.com.au/2021/09/from-vaccine-passports-to-vaccine-apartheid/
    [5] https://www.theguardian.com/world/2020/jun/04/unreliable-data-doubt-snowballed-covid-19-drug-research-surgisphere-coronavirus-hydroxychloroquine

Leave a Reply

Your email address will not be published. Required fields are marked *