## Relative vs. absolute risk reduction . . . 500 doctors want to know!

Some stranger writes:

What are your thoughts on this paper? Especially the paragraph on page 6 “Similar to the critical appraisal ….. respectively”.
There are many of us MD’s who are quite foxed.
If you blog about it, please don’t mention my name and just say a doctor on a 500-member listserv asked you about this. And send me the link to that blog article please. There are at least 500 of us doctors who would love to be enlightened.

The link is to an article called, “Outcome Reporting Bias in COVID-19 mRNA Vaccine Clinical Trials,” which argues that when reporting results from coronavirus vaccine trials, they should be giving absolute risk rather than relative risk. These have the same numerator, different denominators. Let X be the number of cases that would occur under the treatment, Y be the number of cases that would occur under the control, and Z be the number of people in the population. The relative risk reduction (which is what we usually see) is (Y – X)/Y and the absolute risk reduction is (Y – X)/Z. So, for example, if X = 50, Y = 1000, and Z = 1 million, then the relative risk reduction is 95% but the absolute risk reduction is only 0.00095, or about a tenth of one percent. Here’s the wikipedia page.

What do I think about the above-linked article? It’s basically rehashing the wikipedia page. I’m not talking about Weggy-style plagiarism here, just that this is standard material. There’s nothing wrong with publishing standard material if people need to be reminded of some important message.

That said, I don’t think this article is particularly useful. I don’t think it conveys an important reason for parameterizing in terms of relative risk, which is that it can be much more stable than absolute risk. If the vaccine prevents 95% of infections, then that’s 95% of however many occur, which is a more relevant number than comparing to how many happened to occur in the control group in some particular study. Conversely, if that 95% is not a stable number, so that the vaccine prevents a greater proportion of infections in some settings and a lesser proportion in others, then we’d want to know that—this is an interaction—but, again, the absolute risk difference isn’t so relevant. Absolute risk does matter in some settings—for example, we wouldn’t be so interested in a drug that prevents 50% of cases in a disease that only affects 2 people in the world (unless knowing this would give us a clue of how to treat other diseases that have higher prevalence), but of course coronavirus is not a rare disease. Presumably the rate of infection was so low in those studies only because the participants were keeping pretty careful, but the purpose of the vaccines is to give it to everyone so we don’t have to go around keeping so careful.

So, unless there’s something I’m missing (that happens!), I disagree with the claim in the linked paper that failures to report absolute risk reduction “mislead and distort the public’s interpretation of COVID-19 mRNA vaccine efficacy and violate the ethical and legal obligations of informed consent.”

P.S. I’d never heard the term “foxed” before. From the above message, I’m guessing that it means “confused.” I googled the word, and according to the internet dictionary, it means “deceived; tricked,” but that doesn’t seem to fit the context.

1. Jon Baron says:

At the risk of appearing to blow my own horn, there is an empirical literature about this issue. See

https://www.sas.upenn.edu/~baron/papers/liferat.pdf

and the citations in it.

• Andrew says:

Jon:

Thanks. The key difference between the coronavirus-vaccine example and the rare-disease example is that coronavirus is not a rare disease. It just looks rare in the data because of how the experiment was designed. Looking at it this way, the key flaw in the above-linked article is that it is focused on the data at hand rather than on the underlying quantities of interest. That happens a lot in statistics, that people get sucked into discussions of estimates and standard errors without stepping back to consider what questions to ask.

• Jonathan (another one) says:

How rare is “rare?” (I get varying empirical answers to this question when I ask for steak at a restaurant, so you don’t need to give a precise answer here.) While something on the order of 30 million people have gotten the disease in the US, when they were testing the virus in the US it was around 40,000 cases per day; given a two-week virulence period that suggests about 600,000 cases or so, or only about 0.2% of he population.

• Jonathan (another one) says:

… though you really shouldn’t type “virus” when you mean “vaccine” or nobody will understand what you’re talking about. Unfortunately, those errors (by me) are not rare. [Edit button!!!]

• Andrew says:

Jonathan:

I guess the vaccine is supposed to work for awhile, so I think the relevant rate is not the percentage of people who will get it in any particular two-week period, but rather the percentage of people who will get it ever.

• Jonathan (another one) says:

Sure, but take the Pfizer vaccine initial announcement: https://www.pfizer.com/news/press-release/press-release-detail/pfizer-and-biontech-conclude-phase-3-study-covid-19-vaccine
“The Phase 3 clinical trial of BNT162b2 began on July 27 and has enrolled 43,661 participants to date, 41,135 of whom have received a second dose of the vaccine candidate as of November 13, 2020.”

So roughly 20,000 unvaccinated people (I assume the 41,135 must mean either second dose of vaccine or placebo) 162 people got Covid over a 4 month period. That’s still only a rate of 0.8 percent. Obviously, if you extrapolate out in time then that rate rises, but that’s the data they used to approve the vaccine. (It’s a little more complicated than that because (a) many of the enrollees had less than 4 months of exposure and (b) the underlying rate by November was considerably higher than 0.2%. Those two effects work in opposite directions.)

• elin says:

Remember that Pfizer study was not proactively testing for infection, but only reporting symptomatic cases that were reported.

• Jeanne says:

Elin, That bothered me, too.

• IC says:

To date no vaccine has actually been approved. The premise of the article in discussion pertains to informed consent, so it seems important to not get sloppy with language around this point.

• Mike Bluett says:

Whether the vaccines have been approved or not is unimportant. What is important is whether they prove to be effective or not.

• name withheld by request says:

Like every attempt at predicting or modeling this disease, getting from the relative risk reduction in a fast-tracked clinical trial to absolute risk reduction long-term in the population means assuming facts not in evidence. Specifically 1)for how long will a vaccine reduce risk similarly to what was observed in the trial and 2) to what degree does the vaccine reduce transmission. Add the absolute risk reduction to the list of COVID “facts” that can take on ‘most any value you like depending on the assumptions you use for the unknown quantities!

• Hopefully the questions you raise are being pursued practically. It’s a hit and miss effort in some substantive sense.

The problem is that relative risk reduction has translated into a marketing strategy that exaggerate risks and benefits, as Welch notes in the article I posted.

• Fred says:

https://covid19-projections.com/infections/us
One estimate says ~29% (with a range of [19%, 43%]) of Americans were infected with COVID by Feb 2021.
Far from being rare, it might be one of the most common disease Americans had in the last twelve months.

• MarkD says:

as far as I know, (for risk assessment) usually it’s like:

– very common: P > 10%
– common: 10% > P > 1%
– not common: 1% > P > 0.1%
– rare: 0.1% > P > 0.01%
– very rare: P < 0.01%

on the topic, besides the other links I find this https://academic.oup.com/ndt/article/32/suppl_2/ii13/3056571#64437158 quite neat, it also briefly touches the 'number needed to treat' (NNT or NNV in this case) thing.
Absolute risk is quite important to know: the 'treatment' comes with a risk/cost too (it might even reduce 100% the desease fatality, but adds it's own).
My risk of dying (in a given time frame) is my expected context/situation risk of getting infected x my expected category IFR, versus the expected fatality risk of the vaccine (e.g. flu vax is some 3-4 ppm of severe/fatal outcome, < 1/100k overall o severe adverse effects). I'd want the latter well below (say, 10%?) the former to go for that jab.
Suppose my overall chance of getting infected in say 1y is 10% and my expected IFR is some 0.05% then my acceptable limit would be a risk for the jab < 10% x 10% x 0.05% = 5 ppm.
Anyway my overall expected 1y-chance of dying for that desease is some 0.005%, and even a 10-20 fold reduction from that level wouldn't look much impressive to me, regardless of the added risk.
As others have pointed out already, these figures are much time-varing.
Clearly, if (when) I were in the 5+% IFR category I'd likely rush for the jab.

2. Joshua says:

From the article:

> Such examples of outcome reporting bias mislead and distort the public’s interpretation of COVID-19 mRNA vaccine efficacy and violate the ethical and
legal obligations of informed consent.

What’s the non-biased baseline on which to determine bias and distortion? What is the baseline of ethical and legal obligations by which to determine what comprises a “violation.”

Seems that the authors believe that their opinions on these subjective form those factual baselines.

3. Javier says:

Efficacy of vaccines -as well as efficacy of any other treatment- has been always reported as relative risk reduction (RRR).
Absolute risk reduction (ARR) is intended to measure how many people would be benefited from the vaccine (or treatment) out of those receiving it.

The problem with ARR in covid-19 vaccines is that the follow-up had to be very short (just 2-3 months) in order to take decissions asap. That makes the infection rate very low.

Another point with covid-19 is the lack of independence between cases. As in any transmissible disease, to prevent one case by vaccination is not just to prevent one case: it is also to prevent the whole chain of generation of cases transmitted by the primary case.

• I am not clear as to why the FDA can’t include an ARR, as it issues not from the clinical trial data. It can be calculated now as a function of measuring the effectiveness of the vaccine. Maybe by June as then you will have had time monitor vaccine effectiveness.

That article is bit more technical. But I think Gilbert Welch has penned an article for patients and consumers of medical research

The Problem is Relative

https://www.huffpost.com/entry/health-risk_b_1613912

• BrianM says:

Agreed – I think the problem in the framing is the either/or. I think there’s a good case for having both, as Andrew’s comments implicitly draws out by giving some conditionality. I was about to write about the dangers of how these things are publicised, but that huffpost link says it better than I can.

BTW ‘foxed’ may be a geographic thing as I’m very familiar with it – my guess is the author might be British!

• Brian,

Hey hey, not to be outfoxed.

My understanding is that the Consolidated Standards of Reporting Trials [CONSORT} recommends that both the relative and absolutes effects be reported with their confidence intervals. Moreover it recommends that event rates along with their denominators so as to convey how that patients and physicians understand how the risks were calculated.

4. paul alper says:

Absolute vs. Relative has plagued statistics from the outset. Textbooks of years ago would often show real examples (such as the Pentagon budget of one year vs. the next) of histograms, with and without the origin pictured specifically, in order to illustrate how a seemingly large relative difference could be considered tiny in absolute terms. The choice of absolute or relative was psychological and/or situation dependent.
With regard to the Covid crisis, there is even disagreement regarding the numbers used to calculate anything. A depressing number of people in the U.S. still even doubt the existence of the disease, let alone the number who have been affected by the disease and cured by bleach.

5. Sorry, I meant

‘event rates along with their denominators be included so as to convey to patients and physicians how the risks were calculated’.

Sloppy Sameera

6. Dzhaughn says:

We are fortunate to live in a time when the ARR of a smallpox vaccine is zero. Even one with an RRR of 100%. But that is largely because a high-RRR vaccine was administered globally for decades when any measure of the ARR was vanishingly close to zero.

If the vaccines do work and are widely used, I might expect that repeating the study (except larger) next year would find the ARR about 100 times lower and RRR to be the same. So why fuss about ARR?

I think the paper itself is very misleading to call the 1% ARR “very much lower,” inviting comparison of the two.

7. Navigator says:

Medicine is full of similar examples of RRR vs. ARR. Processed meat consumption increases colon cancer chance by 5%. The catch is everybody’s baseline is 4%, regardless of diet. All of a sudden cold cuts don’t seem too bad. (I made up the figures, don’t remember exactly).

I believe all numbers need to be presented to get a full picture. The famous 94% or 95% figures that get thrown around for the two mRNA vaccines are simply based on a limited sample of 40K or so participants in each arm. Vast majority didn’t get infected, regardless of the vaccination status. It is a 20-fold difference in those that did get covid between the two arms that produced the often-quoted numbers.

Here are the figures for Pfizer 7 days after the second dose:

Vaccine: (N=18,198 not infected, 8 infected)
Placebo: (N=18,325 not infected, 162 infected)

I mean, when you only compare the number of non-infected, the difference is really trivial. It’s 8 vs. 162 part that make the headlines.

What really matters is the true protection among hundreds of millions already vaccinated (myself included).

8. TV Watcher says:

“Foxed” = Influenced by conservative rant on Fox News.

Just a guess.

9. Thanatos Savehn says:

Flummoxed?

10. Thomas says:

I think Javier gave the answer. For absolute risks (and AR reductions) to be useful one needs a meaningful time horizon, such as 5-year survival in cancer or a one-winter risk of seasonal flu. Covid is now endemic rather than epidemic, and fluctuating at that. A 3-month risk doesn’t mean anything when you live with the virus continually, especially as the current transmission level differs from that which applied when the trials were run. At least the RR has a simpler message (whatever your risk is, it’ll be reduced by 90%…)

• Anoneuoid says:

Waning and new variants ensure the RR (for infection) will drop with time since vaccination. The trials also excluded the most at-risk population and ignored the increase in illness seen during the first week after first dose while white blood cell counts were low (which has a “culling” effect).

The right way to interpret those numbers is as a maximum possible efficacy.

• Anoneuoid says:

Another issue was the rather noticable side effect profile after vaccination. The trials failed to do exit surveys but it seems unlikely the blinding was very effective.

• Carly says:

I’ve seen you talk about immune suppression in the first week after vaccination several times, but there is no evidence of this. The EUAs for J&J, Moderna, and Pfizer vaccines show no increase in COVID cases in the first week. Specifically, the cumulative incidence curves show no evidence of increase susceptibility to COVID between 0 and 7 days, or 0 and 14 days. If you were substantially more susceptible to COVID in that period of time, the vaccinated curve would exceed the placebo curve before then dipping below it after your immune system “woke up”. I’m also not aware of any immunological reason why temporary immune suppression would be expected. What evidence do you have that these vaccines suppress your immune system for a week?

Related, you state that leukopenia (low WBC counts) occurred during the first week. All three vaccine makers performed hematology panels that would have detected leukopenia. They each performed them before vaccination, after approx one week, and after approx. two weeks. So, what evidence do you have of leukopenia during the first week of vaccination?

To make it clear and easy to access the phase I/II articles and how they performed hematology, here are the links:
Pfizer: https://www.nature.com/articles/s41586-020-2639-4.pdf (safety assessment section in methods)
Moderna: https://www.nejm.org/doi/suppl/10.1056/NEJMoa2022483/suppl_file/nejmoa2022483_protocol.pdf (page 13, clearly states they perform WBC counts)
J&J: https://www.nejm.org/doi/suppl/10.1056/NEJMoa2034201/suppl_file/nejmoa2034201_protocol.pdf (pages 20-37 timing charts, page 114 lays out a reaction grades associated with reduction in WBC count)

It’s also not clear what you mean by “excluded the most at-risk population”. The J&J and Pfizer EUAs are clear that the vaccine was tested in hundreds of people over the age of 75. Moderna only gives the number of individuals over 65, but it seems likely they have approximately the same number as the other vaccines. At least 20% of the participants in each trial was 65 or older. With respect to comorbidities, thousands of individuals in these trials had co-morbidities, some mild and some serious. There are huge tables in each EUA showing the estimated vaccine efficacy for each comorbidity. So who, specifically, did they leave out of these trials that you think should have been included?

To make it clear and easy to access the EUA documents I’m talking about, here are the links:

• George says:

When you have zero deaths in the control group of Pfizer over a 6 month period during the worst part of the pandemic, it tells you something about the study design. It’s called “strategic selection” aka cheating.

11. Phil says:

There’s a phenomenon whose name I’ve forgotten: you think you’ve never seen a word before, then it is brought to your attention, and suddenly you start noticing it all the time.

Foxed…it’s not a common synonym for ‘deceived’, except in the form ‘outfoxed’, but I’ve seen it before, and Andrew I’m very sure you have too. It’s the kind of thing I see maybe once every year or two on average, usually in an old book or historical novel, so, OK, uncommon. Still, I started reading such books when I was in my early twenties, and now I’m 55. For sure I’m well into double digits in seeing that word used that way.

So, Andrew, you’re going to start noticing it.

12. The way to understand the difference between absolute vs relative risk reduction is this:

1) Relative risk reduction is approximately a constant property of the vaccine itself. Whatever the risk is of getting COVID, being vaccinated reduces it by some factor.

2) Absolute risk reduction is a fact that is a historical accident of the time and place in which the trial was run. Whatever prevalence there was, and behavior that obtained during the trial, that sets the absolute risk of infection among the control group, and with a large enough and randomized group, the same risk of exposure will obtain in the vaccinated group, which will be reduced by the relative risk reduction to some other value.

So the question is really this: Do we care about a property of the vaccine, or of the accidental facts of the time period when the trial was run?

Relative risk is the main and most important fact about the vaccines.

• Hi Daniel,

The way you framed the difference between absolute and relative risk does not compute for me. Admittedly, I’m not a statistician either. I think the contexts is very critical to evaluate. I am not convinced that calculation of absolute risk need to be restricted to the trial itself. It is a big jump to assert that because ARR is a historical accident temporally, therefore the relative risk is the most important fact. Strikes me as a hasty generalization

Gerd Gigerenzer has made a valiant effort to educate physicians and the public at large about statistics.

In cancer treatment studies, I think it is an imperative to report both ARR and RRR.

• When a disease is in an approximately “steady state” such as say tuberculosis, or heart disease, or colon cancer, the ARR from one period of time and the ARR from another would be similar, so then the ARR has meaning as it would apply to “next year” at least approximately as well as it applied to this year. That’s not at all the case for an epidemic disease spreading in exponential growth and decay phases.

• Andrew says:

+1. Recall that just over a year ago we had a respected expert suggesting that maybe only 1% of Americans would be infected with coronavirus.

• Hi Daniel,

I understand that it might not be the case with an epidemic in exponential growth & decay phases. But I gather that the RRR can exaggerates the benefits and downplay the harms. So without calculating and reporting the ARR in many contexts, RRR is not meaningful to patients, physicians, and researchers. See CONSORT guidelines below.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2844943/

• That’s right. For example suppose there’s a very rare genetic condition that makes eating avocado life threatening. Suppose there’s a pill you can take with an enzyme which digests the dangerous component reducing the chance of the effect by 99%. However, only 1/100,000 people has the genetic condition.

a 99% relative risk reduction would sound like a lot, but since it only affects 1/100,000 what’s happening is the risk is dropping from 1/100,000 to 1/10,000,000

But what’s worse is that people who’ve eaten avocado before know already that they aren’t at risk… so for them the pill is useless.

In these kinds of conditions, relative risk reduction of a rare issue can be very misleading…. almost no-one needs to have their avocado risk reduced, but among those who do the pill is a big deal.

That’s not like COVID at all… In the absence of vaccines, north of 65% of everyone on earth will likely eventually get COVID and be at risk for serious illness and death over the next few years. This means everyone needs the risk reduction. Therefore, the relative risk reduction is precisely the quantity of interest. Furthermore, this is just the individual risk reduction. In the presence of widespread vaccination the fraction of the population which will actually be infected drops to much lower than the raw 65%… Furthermore, the economic consequences of delaying vaccination and stabilization of the illness are very severe as well.

In the end for COVID vaccines, the relative risk reduction actually **understates** the value because it’s just considering the individual risk issue not the societal risk issue.

yes, perhaps boosters and variant versions of the vaccine will be needed at some point, but right now, the relevant information is that if you get Moderna or Pfizer’s vaccine your risk of hospitalization, severe disease, or death if infected drops by perhaps 99% or so, your risk of being infected in the first place drops by 80-90% and the risk of children, elderly, and others in contact with you drops by some large amount as well. The systemic hospital system stability risk drops by a lot. The economic risk drops by a lot… There’s no excuse to not vaccinate other than a known sensitivity to a vaccine component.

I don’t buy Anoneuoid’s suspicion that the vaccine doesn’t prevent transmission. The evidence seems to be overwhelming (from multiple sources) that it does at least for some time, just on the basis that highly vaccinated populations are not spreading the disease (such as LTC facility patients).

• Joshua says:

+1

• Joshua says:

Suspicion, or a conclusion with fairly high confidence? Seems to me like the latter. But I could be wrong.

• Anonymous says:

“I don’t buy Anoneuoid’s suspicion that the vaccine doesn’t prevent transmission”

You are too generous.

• Marie says:

“That’s not like COVID at all… In the absence of vaccines, north of 65% of everyone on earth will likely eventually get COVID and

be at risk for serious illness and death over the next few years. This means everyone needs the risk reduction.”

WAIT! Putting aside the first premise (which I question on whether we know this or not as well, that second line is definitely something that we do not know and in fact, it seems we know the opposite — SO FAR. In the most simplest form of what we know based on data since the beginning, and tracked and published in ways unseen until this particular pandemic, the chance of getting Covid-19 AND becoming severely ill and/or dying from it is extremely low up and until even 69 years of age. I don’t need to repeat those stats here, or the difference between CFR and IFR, etc. etc. The charts are everywhere and the risk of severe disease and dying, without even looking at how some key treatments are showing great results in reducing severity and hospitalization (another discussion) and are surely to become more and more prevalent.

Your argument that we must all get vaccinated is based on a false conclusion — that 65% of the population will likely get Covid AND be at risk for serious illness and death, making what seems to me to be a circular argument resting on an inaccurate premise which we know not to be true – so far.

Though I am neither a medical doctor nor a statistician, I can easily understand the risks of Covid-19, to me personally and to larger populations, overall, as the event tracking of this disease emanates from so many sources, including the CDC. Absolute Risk Reduction does seem to matter, quite a bit even, if one takes into account, which one should, the risk of severe disease and death. To say ARR does not matter is to miss the entire point, in my view. ARR absolutely matters when compared to the risk of severe disease and/or death.

• Joshua says:

Marie –

> In the most simplest form of what we know based on data since the beginning, and tracked and published in ways unseen until this particular pandemic, the chance of getting Covid-19 AND becoming severely ill and/or dying from it is extremely low up and until even 69 years of age.

Aggregating age stratified risk across ages 0-69 obscures the significant risk from, say, ages 50-70. The CDC’s latest IFR estimate is 0.6% for 50-64, and it obviously would be considerably higher for 50-69. There are a lot of people at that age, so 0.6%+ of all of them getting infected and dying is obviously a lot of death. And obviously, the risk of severe disease is much higher than that, and likewise, aggregating the risk of severe disease with the risk of death obscures the significant level of risk for severe disease.

And then focusing only on severe disease further narrows the window in a way that might be sub-optimal. Even a relatively mild infection may well have long term post-infection sequelae that can have considerable public health consequences for years going forward, including enormous costs to our healthcare system once you consider the impact of those sequelae at the population level.

And then there’s the economic and other disadvantages of a slower trajectory of reducing the infection rate.

> To say ARR does not matter is to miss the entire point, in my view. ARR absolutely matters when compared to the risk of severe disease and/or death.

Personally, I wouldn’t say that “ARR doesn’t matter” – but I would say that it’s important to evaluate COVID within a context that includes the relative risk, at the population level, in public health, economic, and other societal outcomes. A lot of individuals choosing not to vaccinate based on the rationale that it makes little difference in their individual level of absolute risk will have significant cost to our society.

• First off when I said “at risk” i meant something like “we don’t know which ones will be seriously ill or die” not that 65% of people will in fact get seriously ill and die. It’s like “children in houses with pools that have no fence or gate are at risk of drowning”. They won’t all drown, but many more of them will than outside this group.

Second even what’s considered “mild” COVID is extremely unpleasant and has long term effects in largish groups. Focusing on death rates is very misleading. Easily well over 10-20% of people who get the illness wind up with an extremely unpleasant illness whose effects last weeks. Maybe another 10% wind up hospitalized but not dead.

• Hi Daniel,

Thanks for such a thorough explanation. Post-vaccination durability & transmission are still open questions among COVID experts from my observation of their Twitter interchanges. It is a work in progress.

As for the value of calculating absolute risk, I gather that it is recommended for most all trials/studies. Peter Attia MD, has a Twitter thread on ARR as it pertains to the J&J vaccine.

• Mark says:

Joshua,
You write ‘The CDC’s latest IFR estimate is 0.6% for 50-64, and it obviously would be considerably higher for 50-69. There are a lot of people at that age, so 0.6%+ of all of them getting infected and dying is obviously a lot of death.’

That doesn’t seem right. An IFR of 0.6% in this age group does not mean that 0.6% of them will get infected and die. It means that 0.6% of infected people in this age group will die. It says nothing about what % of people will get infected to begin with. So the real world risk of dying is much much lower than 0.6% for this particular age group.

• Jon says:

The IFR is .15%. This is not a serious illness

• Andrew says:

Jon:

It’s a non-serious illness that’s killed 10 million people and disrupted the lives of hundreds of millions of others.

One way to look at this is that people react strongly to certain sorts of avoidable risks, including many risks that are much lower than that of covid.

• Anoneuoid says:

Relative risk reduction is approximately a constant property of the vaccine itself.

This is true for primarily viremic viruses (polio, measles, etc) but not those that mostly stay out of the blood (influenza, the various cold viruses, covid).

You can see the waning for Moderna *vs the original strain* in supplementary Figure 1 here (the best data is FRNT-mNG since that uses the actual virus):
https://www.nejm.org/doi/full/10.1056/NEJMc2103916

After 6 months the neutralizing activity has dropped to about 10-20% of the peak and back to the levels seen after a single vaccine dose in ~30% of ” healthy adult participants”. This waning seems to have accelerated between 3 and 6 months which is kind of odd, most people (including me) expected it to level off.

Based on other studies, if they had done the same assay against the variant viruses (which they would be crazy to have not done) that would lead to another 2-15x reduction in neutralizing activity.

• George says:

The ARR is far more important. It is what should dictate our behavior. The RRR is also relative. It’s not stable. It changes over time and with circumstance. For example, the Pfizer vaccine had an RRR in a population that wasn’t representative of any normal population since there were so few people who got covid, and so few people who died. The effectiveness of the vaccine changes over time as well. We know nAbs go down over time, titers in general go down over time and relative the person. The RRR changes depending on which variant, and the variants that are blocked by the vaccine don’t become predominant, so there is selection for escape variants which also affect the RRR dramatically. The ARR is more important because it gives you a larger perspective on how it impacts your life. If the ARR is 1% then a RRR of 95% vs an RRR of 40% won’t matter much. If the ARR is 70% then there’s a large difference between an RRR of 95% and of 40%

13. Joshua says:

Personally, I think that focusing on the question of whether relative or absolute risk is more informative, and by how much, mind of misses the point.

Lome IFR, both of those measures of risk have fairly limited utility because COVID risk interacts with so many important variables at the individual level. Do you have diabetes? How old are you? Have you had extended contact on close quarters with someone who has a high viral load? Etc. All of these variables interact with both relative and absolute risk in very meaningful ways.

More important, IMO, is to think from within a socio-ecologocal model of health, it am individual model. It’s important to think at the population level.

If you get vaccinated, you help reduce the spread, you make it less likely that you’ll get infected and be a burden to healthcare heroes. You’ll contribute to fewer people dying from COVID.

Javier says it well above:

> it is also to prevent the whole chain of generation of cases transmitted by the primary case.

• Anoneuoid says:

If you get vaccinated, you help reduce the spread,

In contrast to sars-2 infection, vaccination (Pfizer) does not seem to trigger mucosal immunity:

Altogether, our results suggest that vaccinees probably do not elicit an early humoral response detectable at mucosal surfaces. They strengthen the hypothesis that some vaccines may not protect against viral acquisition and infection of the oral–nasal region, but may prevent severe disease associated with viral dissemination in the lower respiratory tract. Our results are in line with those obtained in nonhuman primates, in which vaccinated and then challenged animals display detectable viral loads in nasal swabs but not in lower airways37.

https://www.nature.com/articles/s41591-021-01318-5

By what mechanism do you think it will reduce spread? That was how vaccination was possibly going to prevent infection of the upper airways.

• Joshua says:

I’m going on the many reports that differential comparisons of vaxed vs. non-vaxed shows an advantage in that not only do fewer people have severe infections, also fewer people get infected. While one study is certainly relevant, I”m leery of the “one study” problem, and anyway, I’d rather go with large scale stats in the real world.

That said, even if it were true that just as many people still get infected, or there’s no differential benefit in that those who do get infected are less likely to infect others, I’m still going to go with the compounding benefits at the population level of fewer people winding up in the hospital, less stress on healthcare workers, etc. The compounding benefits of that alone, make navel gazing about relative versus absolute risk at the individual level, in my opinion, pretty meaningless and rather self-absorbed. Doesn’t mean that I don’t understand it. It’s a human characteristic to be self-absorbed and focus on relatively meaningless stuff. But even though I understand the tendency, I also have an opinion about the tendency.

You should get vaccinated.

• Anoneuoid says:

I’m going on the many reports that differential comparisons of vaxed vs. non-vaxed shows an advantage in that not only do fewer people have severe infections, also fewer people get infected. While one study is certainly relevant, I”m leery of the “one study” problem, and anyway, I’d rather go with large scale stats in the real world.

Unfortunately you didn’t link to any of these studies so not much I can say other than “real world” data used to be referred to as “observational” data and considered less reliable. Eg, that argument has been used to not correct vitamin deficiencies in covid patients.

I do know of this study:
https://www.cdc.gov/mmwr/volumes/70/wr/mm7013e3.htm

They compare positive tests rates in healthcare workers from dec 15 to mar 13. When unvaccinated that was during the winter peak of 200k cases/day but by the time 2 weeks after second dose started happening the US number had already dropped to about half that.*

Very strange study that compares two different timeframes with very different baseline risk of being positive.

* And total cases generally have lagged HCW cases by a week or two.

• Joshua says:

Sure. I’ll go with “observation data,” along with the always important caveat that with observational data (particularly cross-sectional observational data or limited longitudinal data), cauality can’t be proven. There could, conceivably be other reasons why the rate of infections has dropped as it has in Isreal, why those who do get infected are increasingly likely to be unvaccinated, etc. But sometimes even when you don’t have enough tightly controlled studies to make an airtight conclusion of cauality, you still have to make decisions about risk, and in particular decisions about even low probability but high damage risk. So you do your best to give a range of uncertainty.

In this situation, I’m comfortable getting vaxed and telling other people to get vaxed, for reasons I consider more important than my own individual risk profile, be it relative or absolute risk (of course there might be individuals where their individual profile takes in a bigger weight).

Maybe in 5 years I’ll prove to have been mistaken. I accept that. As my father used to tell me, no one ever promised me a rose garden.

• Joshua says:

BTW, while I think is important to stipulate caveats about drawing causality from observational studies, I think I’m going to walk back my agreement about observational data ’cause as I think about it more I’d be hard pressed to think of any data that isn’t “observational.”

I think “real world data” might be subject to a similar criticism (i.e., is non-real world data from another planet?), but I think there’s a meaningful distinction to be made between data collected out there in the wild and data collected in contrived, lab or clinical conditions – and I think that “real world data” works as a descriptor.

• Anoneuoid says:

There could, conceivably be other reasons why the rate of infections has dropped as it has in Isreal, why those who do get infected are increasingly likely to be unvaccinated, etc.

What conceivable reason are you supposing it prevents infection without mucosal immunity?

Perhaps that one study is flawed but that is all the evidence we have besides the supporting animal studies that found the same.

And the same drop in infections has happened 2-3x in the last year without any vaccine involved:
https://www.worldometers.info/coronavirus/country/israel/

I don’t think you’ll have to wait 5 years to find out.

• Joshua says:

Anoneuoid –

> And the same drop in infections has happened 2-3x in the last year without any vaccine involved:

Really. The “same” drop?

Can you give a link to the evidence where the previous drops showed such a wide differential in infections comparing vaxed to non-vaxed? ‘Cause I haven’t seen that from back in November 2020 😉

• Joshua says:

Andrew –

Don’t worry, I’ll drop out now.

• Anoneuoid says:

I’ll bet before Dec 31st, 2021 the daily number of cases on worldometers for Israel rises above 5k at some point again.*

I am a stupid crazy crackpot who doesn’t understand science so you should give me at least 2:1 odds, right?

* Even if they just stop testing, or don’t use tests that can detect new strains, or anything like that, I still lose the bet. If worldometers stops reporting data like covid19tracking did last month then no one wins.

But yeah, don’t bother responding unless you are willing to bet at least \$100.

• Joshua says:

Anoneioid –

>I am a stupid crazy crackpot who doesn’t understand science so you should give me at least 2:1 odds, right?

At the risk of deepening my status as “one of those commenters,” and hoping that given the nature of what I’m going to say, Andrew won’t think that I’m breaking of my pledge above, I’ll comment not in response to our topic of previous discussion but in response to what seems to be your mistaken impression that I consider you to be a crazy crackpot who doesn’t understand science.

That isn’t my view. I apologize for giving you that impression.

• Hi Joshua,

RE: If you get vaccinated, you help reduce the spread, you make it less likely that you’ll get infected and be a burden to healthcare heroes. You’ll contribute to fewer people dying from COVID
—-

The durability and transmissibility questions after receiving the COVID vaccine still loom in social media, like Twitter especially. After all, we are wearing masks and advised to keep socially distance. This is all a work in progress

• Joshua says:

Hi Sameera,

> This is all a work in progress

No doubt. It’s easy to lose sight of that sometimes.

• confused says:

Definitely.

I personally think that the concerns have been overstated (IE vaccine efficacy has been “underplayed”) which probably contributes to vaccine hesitancy among people who aren’t otherwise anti-vaccine (IE compared to a situation where the CDC etc. message was “two weeks after your second shot, you might as well act like COVID doesn’t exist”).

But no one really knows.

I personally think possible variant issues are “overstated” because in past pandemics we didn’t have the genetic technology to track mutations (so this “looks new and weird” when in fact it could be completely normal for the late stage of a pandemic, we couldn’t track mutations in 1890, 1918, 1957, 1968…)

• David says:

The WHO has said there’s no evidence of the vaccines reducing transmission rates or infection rates. Do your research before posting.

14. Jon says:

I think one case where relative risk reduction is more significant than absolute risk reduction is where the numbers come from measurements over a short period of a recurring risk. For example suppose over a two week period being instructed to wear a mask reduced the risk of getting Covid from 2.5% to 2% . That is a relative risk reduction of 20%, but an absolute risk reduction of only 0.5%; however, the risk will be repeated until the individuals change their other behaviors, get vaccinated, or the pandemic essentially ceases. If the risks remain constant, over 1 year’s time, that becomes an absolute risk reduction of about 7.3% from people who would have about a 50% chance of getting COVID.

15. Alan says:

I review the Pfizer data and keep coming to the conclusion that during the trial period those in either group had a less than one percent chance of becoming infected. It seems to that studies should be done to better determine how and where the disease is transmitted and then provide better fact based recommendations as to how one should protect themselves. China has already proved the disease can be controlled quickly without vaccines and at a much lower cost to the population. Wuhan was reopened after less than 3 months yet the battle continues in most of the world.
During the Pfizer trial there were no reported deaths in either group from covid. The deaths reported were all in line with deaths routinely reported throughout any given year. How does one come to the conclusion the vaccine significantly reduces death? In the Moderna trial one person died from covid.
I saw statistics today claiming there have been over 150,000 infections in the US military yet only 21 deaths. Given their average age is considerably lower than the general population, is a vaccine even warranted?
I do not want to be grouped with the anti vaxxers; i just want to better understand the trials.

16. Dave says:

Simple, report the absolute risk to the public and convey the important aspect of relative risk. Thats all. To those trained in consuming the clinical trial reports absolute risk is common knowledge, to the layperson, their denominator is the 24×7 media reporting of the killer pandemic and the constantly updated death ticker standing at well over 500K (in the USA) people at the time I am writing this. I am relatively sure the authors of the clinical trial report did not intend to muddy the waters, however nobody who has even a smidgeon of the pulse of the general public would deny that the absolute risk reduction is not known, understood or considered in their decision to get a vaccine. Everyone would agree people should be educated as much as possible before making decisions that may effect their health. Reporting the absolute risk reduction in addition to relative risk is additive, and promotes more understanding, not less. This is a no brainer.

• > Reporting the absolute risk reduction in addition to relative risk is additive, and promotes more understanding, not less

The thing is, absolute risk reduction isn’t a number we know. Calculations involving it are essentially wild ass guesses which could vary over several orders of magnitude. Relative risk however is a number we know to within a few percent.

The “absolute risk” doesn’t add anything to the debate because it’s a meaningless number.

• Dave says:

I don’t think absolute risk can be considered more of a guess than relative risk considering the 95% efficacy rate was arrived at without a human challenge trial. Those in a placebo group would have roughly shown the same efficacy rate, there is no two ways around that. I think you can make the argument that absolute risk should not be given all that much weight in deciding the effectiveness of mass vaccination, but not the superiority of the way the figure was arrived at. The aspects that affect calculation of absolute risk reduction can be relatively controlled for. With the volume of cases vs population and clear patterns repeating for different populations based on location, age, etc. Analyzing a subset would almost certainly not stray in order of magnitude.

• I lean to Dave’s view. Among experts, Gilbert Welch gives the most understandable explanation of absolute risk that I come across. I have to find the relevant article or video.

17. Eyal says:

I prefer to see all the data. It seems to me that ARR and RRR are both contextualized and actually I am not convinced that enough control is applied in these studies at all. The decision should not be made only based on these numbers. The conversation has to go wider. The long term effect of the vaccine is not known, that risk is absent from the decision making process. The availability of treatment that avoids death and long term COVID is also absent. The evolutionary pressure on the virus (if you agree that it is prevalent) by the vaccines and other measures are also not factored into this. So the picture that emerges as if there is only one option and all should line up for this experiment is very worrisome. There is a clear agenda advanced by Pharma and Gov, but their bias and conflict of interest is unchecked.

18. Dave says:

I believe one needs to distinguish risk and benefits (harms and efficacy) when viewing absolute or relative risk, toward the individual, not the population. Will the intervention “matter” to me vs doing status quo (or other personal mitigation or health improvement effort)?
Speaking in terms of absolute or relative to a population is one thing. Determination based on a singular clinical trial, still in Phaee 3 motion, the personal impact +/- on “me” is wholly another. In either view, one cannot state “I” have a 95% lower chance of obtaining Covid once I am properly vaccinated.
Social morning and guilt complexes need not apply. Ironically, vaccines do not require informed consent under EUA.
The forging statements of 65% likelihood of obtaining Covid are a bit of a morning burger: who exactly are we talking about? The infirmed. The sickly? The elderly? 1 or 2 of 3 of these scenarios? CFA matters.
Ironically, no vaccine has been powered to tell us that: if I get Covid will I die and will this vaccine prevent that from
happening to people like ME? Moreover, a reduction from 4 to 2 deaths (Placebo to Active arm) isn’t quite the thrilling level of ARR of death (over that time frame, quite important), one would hope for and cannot be extrapolated to ME or the World when vaccinated.
Not much discussion, yet, on ARR or RRR of key safety events or sub pop analyses. Looking forward to the dialogue.
If we ascribed as much “what if…long term….potential badness” of the virus to the vaccine, we’d certainly be having a different dialogue.

19. Dave says:

Excellent, peer-reviewed article with succinct upshot: need to fully vaccinate well over 100 people to avoid a single Covid case.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7996517/pdf/medicina-57-00199.pdf

20. MD says:

Some general comments. I find it troubling that a clinician would seek advice on this topic from a statistics blog, even one hosted by Columbia University. Also troubling that when searching for “Absolute Risk Reduction”, this page appears in the results ahead of the paper in question. Such is the state of things.

For any other treatment or vaccine, there would be no controversy or concern regarding the admonition in the paper that ARR should be reported along with RRR. It is standard procedure and well understood by most clinicians, at least those who have had good training and keep current with literature.

ARR is no more of a trivial measure than is RRR a magical one. Both are important for decision-making, hence the FDA guidelines mentioned in the paper. It should be noted that these EUA medical products are not without risks.

For your own personal risk profile for SARS-CoV-2 / COVID-19 (age, co-morbidities, prior infection / immunity), one must determine how much risk is reduced by the intervention and balance that against how much risk you have without the intervention—PLUS you must factor in the risk(s) of the intervention itself.

Good luck doing that in any realistic and clinically unbiased way without incorporating ARR into the process. Personally, I do not want medical advice from any clinician who doesn’t know the ARR of these medical products—and I am troubled that many are not only not aware of the ARRs but think that they are unimportant when advising patients. This is a major failing.

• Andrew says:

Md:

He’s not seeking advice from “a statistics blog,” he’s seeking advice from a statistician! It’s a statistics question, so it seems very reasonable for this person to ask a professional statistician for thoughts on the topic. When I have a medical question, I’d ask a doctor.

• Layman Joe says:

Based on the responses so far, I don’t think he got what he wanted. All I see is a bunch of random posters are brainstorming and speculating. No definitive “advice”.

• Andrew says:

Layman:

I don’t think my advice was so bad! But, hey, free advice is worth what you pay for it, I guess.

21. A.D. says:

Can anyone point me to what the study’s participants were instructed to do after receiving the vaccine/placebo when they went on their respective ways regarding wearing masks, socially distancing, working from home, etc.? Also, did the study track if they were living with other people? Seems like these would all be relevant factors since relatively speaking the difference between the infection rates between the groups was not significant.

Where can you find the best CBD products? CBD gummies made with vegan ingredients and CBD oils that are lab tested and 100% organic? Click here.