(This post is by Yuling, not by/reviewed by Andrew)

Recently I (Yuling) have contributed to a public heath project with many great collaborates: The goal is to understand the excess mortality in potential relevance to Covid-19. Before recent case surge in south Asia, we have seen stories claiming that the pandemic might have hit some low-income countries less heavily than in the US or Europe, but many low-income countries were also lacking reliable official statistics such that the confirmed Covid-19 death counts might be questionable. To figure out how mortality changed in rural Bangladesh in 2020, my collaborates conducted repeated phone surveys in 135 villages near Dhaka, and collected death counts from Jan 2019 to Oct 2020 (the survey questions include “Was member X in your household deceased during some time range Y? When? What reason?”).

##### The findings

The statistical analysis appears to be straightforward: compute the mortality rate in 2019 and 2020 in the sample, subtract, obtain their difference, run a t-test, done: You don’t need a statistician. Well, not really. Some deaths were duplicately reported from multiple families, which needs preprocessing and manual inspection; the survey population dynamically changed overtime because of possible migration, newborns, deaths, and survey drop-out, so we need some survival analysis; The survey has large sample size but we also want to stratify across demographical features such as age, gender and education; The repeated survey is not random so we need to worry about selection bias from non-response and drop-out.

It is interesting to compare these data to the birthday model on the day-of-month-effect effect. We used to make fun of meaningless heatmap comparison of day times month of birth dates. But this time the day-of-month-effect is real: participants were more likely to report death dates of deceased family members in the first half of the month as well as on some integer days. Presumable I can model these “rounding errors” by a Gaussian process regression too, but I did not bother and simply aggregated death counts into months.

In the actual statistical modeling, we stratify observed date into cells defined by the interaction of age, gender, month and education, and fit a multilevel logistic model: the education is not likely to have a big impact on mortality rate, but we would like to include it to adjust for non-response or drop-out bias. We poststratify the inferred mortality rate to the population and make comparison between 2019 and 2020. The following graph shows the baseline age and month “effect”—not really causal effect though.

All coefficients are in the scale of monthly death log odd ratios. For interpretation, we cannot apply the “divide-by-four” rule as the monthly death rate is low. But because of this low rate, log odds is close to log probability hence we can directly take exponential of a small coefficient and treat that as a multiplicative factor, which further is approximated by exp(a)-1=a. Hence a logistics coefficient of 0.1 is approximately 10% multiplicatively increase in monthly death probability (it is like “divide-by-one”, except the genuine divide-by-four rule is on the additive probability scale).

One extra modeling piece is that we want to understand the excess mortality in finer granularity. So I also model this extra risk after Feb 2020 by another multilevel regression across age, gender and education. The following graph shows the excess mortality rate by age in females and from low-education families:

This type of modeling is natural or even default to anyone familiar with this blog. Yet what I want to discuss is two additional challenges in communication.

##### How do we communicate excess mortality?

The figure above shows the excess mortality in terms of log odds, which in this context is close to log probability. We may want to aggregate this log odds into monthly death probability. That is done by the post-stratification step: we simulate the before and after monthly death probability for all cells, and aggregate them by 2019 census weights (in accordance to age, gender and education) in the same area. Below is the resulted excess mortality, where the unit is death count per month,

We actually see a decline in mortality starting from Feb 2020, especially for high ages (80 and above), likely due to extra cautions or family companions. Because of their high baseline values, this high age group largely dominates the overall mortality comparison. Indeed we estimate a negative overall excess mortality in this area since Feb 2020. On the other hand, if we look into the average log odds change, the estimate is less constrained and largely overlaps with zero.

We report both results in a transparent way. You can play with our Stan code and data too. But my point here is more about how we communicate “excess mortality”. Think about an artificial make-up example: Say there are three age group, children, adults (age <59.5), elderly adults (age >59.5). Suppose they have a baseline mortality rate (0.5, 5, 30) per year per thousand people, and also assume the census proportion of these three age groups is (20%, 60%, 20%). If their mortality changes by -10%, -10%, 20%, the average percentage change is -4%. But the absolute mortality change from (0.5, 5, 30) * (20%, 60%, 20%)=9.1 to 9.99, which is 0.89 excess death per year per thousand people, or a 0.89/9.1=10% increase. Is it a 10% increase, or a 4% decrease? That all depends on your interpretation. I don’t want to call it a Simpson’s paradox. Here both numbers are meaningful on its own: the average excess log mortality measures the average individual risk change; the average excess mortality is related to death tolls. I think WHO and CDC report the latter number, but it also has the limitation of being driven by only one subgroup—we have discussed similar math problems in the context of simulated tempering and thermodynamics, and a similar story in age-adjustment over time.

##### The excess morality depends on the what comparison window to pick

Lastly, the question we would like to address from the evidence of “excess morality” is essentially a causal question: what we care is the difference in death tolls if covid had not occurred hypothetically. Except this causal question is even harder: we do not know when covid was likely to be a significant factor in this area. In other words, the treatment label is unobserved.

Instead of doing a change-point-detection, which I think would lead to non-indetfication if I would model the individual treatment effect, I view the starting month of the excess morality comparison window as part of the causal assumption. I fit the model with varying assumption and compute the excess mortality rate starting from Feb to Aug. The results are interesting: the overall excess mortality is nearly negative when we compare 2020 and 2019, which is the result we report in the title. But if we choose a only compare later months, such as starting from July, there is a gradually positive excess mortality, especially for people in the age group 50–79. Well, it is not significant, and we need multiple testing when we vary these assumptions, but I don’t think a hypothesis testing is relevant here. Overall, this increasing trend is concerning, amid a more serious third wave recently in South Asia at large.

We also report the relative changes (there is always a degree of freedom to choose between multiplicative and additive effect in causal inference).

This type of varying-causal-assumption-as-much-as-you-can is what we recommend in workflow: when we have multiple plausible causal assumptions, report fits from all models (Fig 22 of the workflow paper).

But this graph leads to further questions. Recall the birthday model: if the there are fewer newborns in Halloween, then there have to be more on the days before or after. The same baby-has-to-go-somewhere logic applies here. Despite the overall excess mortality was likely negative starting from Feb 2020, we infer there was an increasing trend of mortality since the second half of 2020, especially for the age group 50–79. What we do not know for sure is whether, this positive excess mortality was a real temporal effect (a signal of gradual Covid infections), or a compensation for fewer deaths in the first half the year. Likewise, in many counties where there was a strong positive excess mortality in 2020, there could as well be a mortality decline in the following short term. In the latter case, is this base-population change from the previous period (aka “dry tinder effect”, or the opposite wet tinder) part of the indirect effect, or mediation, of Covid-19? Or is it the confounder that we want to adjust for by fixing a constant population? The answer depends on how the “treatment” is defined, as akin to the ambiguity of a “race effect” in social science study.

“the survey population dynamically changed overtime because of possible migration, newborns, deaths, and survey drop-out, so we need some survival analysis”

In your description of what you did, I don’t see anything I’d usually classify as a “survival analysis”. Is there a missing piece somewhere? Or did you just mean some sort of regression modeling to account for the changes in survey population?

I guess it is related to the broad sense survival analysis in terms of dealing with censored/truncated data, or data-dependent-collection at large.

for birthday problems see https://staff.math.su.se/hoehle/blog/2017/02/13/bday.html

and

Klotz, J. 1979. “The Birthday Problem with Unequal Probabilities.” 591. Department of Statistics, University of Wisconsin, Madison. https://drive.google.com/file/d/1qUBp3arLpFTwZ_s9y_ftSoexOGsiiTbW/view?usp=sharing

Thanks for sharing Yuling, I always appreciate your blog posts.

Hi Yuling, this is very interesting. Did you continue this work into 2021? Are you aware of other similar works in other parts of the world or Bangladesh?

It will be interesting to have more data. We do not (yet) have followup data for 2021: it is hard to conduct additional surveys given the recent situation in South Asia.

I understand. Isn’t this an indication in and of itself of high mortality?