“There are no equal opportunity infectors: Epidemiological modelers must rethink our approach to inequality in infection risk”

Jon Zelner, Nina Masters, Ramya Naraharisetti, Sanyu Mojola, and Merlin Chowkwanyun write:

Mathematical models have come to play a key role in global pandemic preparedness and outbreak response . . . However, these models have systematically failed to account for the social and structural factors which lead to socioeconomic, racial, and geographic health disparities. . . . We evaluate potential historical and political explanations for the exclusion of drivers of disparity in infectious disease models for emerging infections, which have often been characterized as “equal opportunity infectors” despite ample evidence to the contrary. We look to examples from other disease systems (HIV, STIs) as a potential blueprint for how social connections, environmental, and structural factors can be integrated into a coherent, rigorous, and interpretable modeling framework. . . .

Zelner adds:

I think it touches on some of the issues in our Patterns piece, but from the perspective of saying that transmission models that omit the structural drivers of risk—almost analogous to hyperpriors on the model parameters—are inherently misspecified.

This sort of connection between statistics and politics always interests me.

58 thoughts on ““There are no equal opportunity infectors: Epidemiological modelers must rethink our approach to inequality in infection risk”

  1. Speaking of the second paper, this kinda stood out:

    > What if the challenges and failures of prediction and forecasting in this pandemic are not to be overcome by more elbow grease and ingenuity, but instead require moving the inferential and predictive goalposts to better align with what the available data can tell us?

    Do you have an example of a failed prediction/forecast and how we might move the goalpost? I liked the line but I’d like to have an example to think about.

    • “Do you have an example of a failed prediction/forecast”. That’s pretty much every single model that’s been used to predict anything longer than 2 weeks into the future, no?

      • No. At least, here’s how I see it. It’s much easier to predict what the virus will do GIVEN a set of human behaviors than it is to predict what human behavior will be.

        The idea that pandemic modelers will need to be able to predict that massive protests over the latest political outrage will break out, or that QAnon believers will hold an impromptu rally in which they purposefully spread the virus to show how it’s all a hoax, or that the governor of Florida will sign executive orders against basic infection control in schools or any of that sort of insanity in general seems to put the modeling efforts on far too omniscient a pedestal.

        I think it’s sufficient as a modeler to say what would happen if the social conditions you model continue in a certain manner.

        However this means inevitably that modeling is a short term thing. I’d say more like 6weeks than 2 but yeah, we need to constantly update models to match the social situation. I do think a feedback between model outputs and behavior would be an interesting kind of model though

      • I don’t doubt that predictions and forecasts have failed — but that second bit is important. Like what is a productive way to move the goalpost to suit the data (or in any of the many different Covid situations under which data is being collected)?

        Andrew had the paper with the Indiana hospital data — so maybe the message is there (https://statmodeling.stat.columbia.edu/2021/02/09/how-to-track-covid-using-hospital-data-collection-and-mrp/)? Like get specific? And if that is a moved goalpost, then what was the unmoved goalpost?

        And in what ways were we elbow greasing and ingenuity-ing our way through the data parts of the pandemic? I really don’t know it just seemed like an interesting line and I am hoping for more details. Maybe the answer is reread the paper lol. At this point I’ve probably spent more time wondering about the line than I did reading the paper the first time through :/.

    • You can see 28 of them here. Check the historical accuracy: https://www.cdc.gov/coronavirus/2019-ncov/science/forecasting/forecasts-cases.html

      They all assume there is no seasonality, just add that and you’ll get far more predictive skill. For example, it is a near certainty cases will be below 500k per week on Oct 1st, yet that looks like it will be below the lower bound of the ensemble 95% interval.

      If anyone disagrees, lets bet!

        • Instead, why don’t you just bet against someone who is clearly wrong like me? I’m even giving an extra few weeks since really I expect the trough to be closer to the end of Oct.

          Then after the results are in, it is worth discussing why there is/isn’t a huge seasonal effect.

          Writing a bunch about it beforehand is a waste of time.

        • is it because, globally, countries are in a different season?

          And yes Anoneuoid is a well-known imbecile in this comment section. I think he was also the one claiming it’s really easy to beat the stock market. Lolz.

        • Different seasonality does not appear to be a major factor. Peaks over the last year appear with quite different timing in countries with similar seasonality. Peaks appear throughout the year. Also, (a bit of a cheat) when the pandemic started I spoke with a researcher who had worked with Coronavirus for many years, including during the first Sars outbreak, and was informed that seasonality for Coronavirus is quite variable, depending on the type, and is not a strongly marked phenotype for any of the types yet identified. Or at least that was the short and uncomplicated version I got.

          The cycles seen look more like a feedback loop with significant hysteresis. The subsequent peaks appear at intervals with timing set more by initial onset. Or that is what I see with a cursory look over the data.

      • Anon, you say “For example, it is a near certainty cases will be below 500k per week on Oct 1st, yet that looks like it will be below the lower bound of the ensemble 95% interval.”

        Since it’s a ‘near certainty’, I assume you’re willing to offer, say, 12:1 odds? (I’m interpreting “near certainty” to mean about a 92% chance). And you say “on October 1st” but you are talking about the weekly rate. How about if we base the bet on 7 days of which October 1 is the 3rd day, that is just slightly in your favor.

        So, just to make it totally clear:
        How many new cases will be reported in the United States for September 29 through October 5 (inclusive)? We need to decide where to get the number we will use. I typically look at Worldometers “USA” numbers but if you have a specific website you would like to suggest, please do so. I suggest we settle the bet on October 12, so there’s a week-and-a-half in between during which the official numbers can be revised.

        If you’re offering 12:1 then my preferred wager is $1000.. So, just being crystal clear: if I wager $1000, and the number of ‘new cases’ summed over the seven days (Sept 29 through Oct 5) is greater than or equal to 50,000, then you pay me $12,000. Otherwise I pay you $1000. But $12K is quite a bit of risk for you. If you’d rather make it a more symbolic bet, say $10 on my side and $120 on yours, that’s fine.

        Hmm, it suddenly occurs to me that I don’t know you, or even know who you are, so if you simply refuse to pay then this is a foolish one-sided bet. Nothing personal — since I don’t even know who you are, it can’t really be personal! — but this does change things. Maybe we can each send Andrew our money and he can hold it for us until the bet settles?

        • Sheesh. “So, just being crystal clear: if I wager $1000, and the number of ‘new cases’ summed over the seven days (Sept 29 through Oct 5) is greater than or equal to 50,000, then you pay me $12,000.” That should be 500,000, not 50,000. Of course.

        • I’d say use the data shown on the cdc forecasting site. The only problem with your bet proposal is theres gotta be people who will give better odds. I mean, what do you think the odds are?

          And yea, keep it friendly amounts to start (~ $100 max payout). Do you, or do you know anyone, that uses crypto so we can keep it anon?

        • I know little about this issue or about how much to trust the models. Just looking at the historical peaks we have had thus far, I’d guess this week or next week will be the nationwide peak and that things will fall off about half as quickly as they climbed. My central estimate for the week in question would be something like 650K new cases, but with wide enough error bars that I’d put maybe 30 or 35% of the probability below 500,000. But I’d also have a substantial chunk of probability over $1M, which you must think is pretty much nuts.

          So, we definitely disagree and should be able to find a mutually satisfactory wager. I think a number over 500K is more likely than not, let’s say a 66% chance or so, whereas you think a number that high is extremely unlikely. There’s a sense in which I ‘should’ take an even-money bet, since I think that’s actually in my favor, but it feels wrong to me because then we are ending up much closer to my belief than to yours. It seems more fair to split the difference or something.

          I think there’s something like a 66% chance of a number over 500K, you think it’s, what, maybe an 8% chance or something, depending on what you mean by “near certainty.” I know there’s some theory or philosophy for ‘splitting the difference’ that suggests we don’t just take the mean of the averages — that would be around 37%, which is somehow much more consistent with my number than yours. How does 25% sound, i.e. 3:1 odds? If there are more than 500K new cases in the week in question, you pay me $100. If fewer, I pay you $33.33. Ah, hell, I’m a generous man, let’s make it $34.

          Could you post a link to the cdc page that shows the number of new cases by day? I’m happy to use such a page.
          Again, let’s wait about ten days to settle the bet, in case of reporting delays or whatever.

          As for how to settle, I don’t do bitcoin or any of the other crypto currencies and I will not do so, I think the wasted energy is appalling. Easiest thing would be just to send a few bills through the mail; if you don’t put a return address on it, that’s more anonymous than any crypto currency.

        • Triple entry bookkeeping is the future, it actually saves massive amounts of energy and allows all sorts of otherwise impossible economic interactions. You can start here if you want to learn:
          https://medium.com/dataseries/triple-entry-accounting-system-a-revolution-with-blockchain-768f4d8cabd8

          Also look up the byzantine generals problem.

          That said, if you refuse to take advantage then I’m sure something else could be figured out. It is more just why?

        • I think I’d be interested in taking this bet too, at the same odds as Phil. Not sure if Anoneuoid would want to risk losing another $100 bucks, so I thought of another condition. I’d happily send $33 to an intermediary to go to Anoneuoid’s charity of choice if I lose, and if I win, until February 2022, if Anoneuoid chooses to post here, they would need to use the handle “Anoneuoid Got ‘Basic’ Science Wrong” in every post or reply. This is worth about $100 to me.

        • Anon,
          Here’s the proposed bet: If there are more than 500K new cases in the week Sept 29 through Oct 5, you pay me $100. If fewer, I pay you $34.

          Do we have a bet? If I lose I’m sure we can find a way to get you your money.

          I see you re-posted the link to the cdc’s projected future cases, but I don’t see where we would look to check the actual number new cases in the period in question. The ‘past’ numbers in that cdc page obviously come from somewhere, and I’m sure we can figure it out when the time comes, but if you happen to have that link handy then please post it.

          Phil

        • Crypto is less anonymous than anything else unless you’re on a layer-2 network or your first buy-in was anonymous. Probably the best way is to buy a greendot moneypak in cash, then exchange them online through secure messaging. Comes with a $5 flat fee though.

        • Phil:

          I’m with you on just sending mail, but I think Anon’s concern is that Anon would then have to reveal his or her address (or P.O. box number or whatever) if he or she wins.

          Another option would be that loser has to pay the specified amount to the winner’s specified charity. Maybe to be on the safe side we agree on the charities ahead of time.

        • Daniel,
          Anon isn’t the one who will be collecting money, he’ll be sending it, and he can do that anonymously through the US postal service!

        • Phil: But if you can convince him of that, then he won’t take the bet!

          Here’s my suggestion. Anon buys a burner phone (a thing he seems likely to want anyway). He registers the Burner phone to his Zelle account with his bank (This service is owned by BofA, BB&T, Capital One, JPM Chase, PNC, US Bank, and WellsFargo so most major US banks). He sends the phone number to Andrew, and you do the same. Whoever loses uses Zelle to send the money.

        • Anon,
          I don’t want to use cryptocurrency because of its absurdly wasteful energy requirements. Bitcoin currently consumes something over 1500 kWh per transaction (there’s some debate about how/whether to count bitcoin mining as distinct from a bitcoin ‘transaction’, but since validating a transaction requires mining bitcoin it seems to me that it’s a distinction without a difference. At any rate the number you’d get is not wildly different however you calculate it). That would power my house for two months. I just think that’s ridiculous.

          If you are willing to accept a gift card, I could buy one and send the serial number etc. to one of your email accounts. Or I can encrypt the card information and post it here, if you don’t have an email account that you consider suitably anonymous.

        • @Phil

          It would be dumb to use bitcoin for a $10-100 transaction, you would use a cheaper crypto. Eg, doge.

          But there is nothing appalling about securing larger transactions with a month or two worth of household energy. 1500 kWh works out to ~$150 per transaction. Currect tx fees are more like $2, since miners are subsidized by the block reward for now. But eventually you will be paying the cost of energy to transact on the network.

          Much more than that is spent all the time to secure larger transactions. Just the typical credit card fee is about 2%, so for a $7.5k purchase you are “wasting” the same amount of resources.

          It seems you are not interested in informing yourself on this topic though, so we need to figure out a workaround.

          I’m currently travelling but still interested in making this bet somehow.

        • I’m not talking about the cost, I’m talking about the energy usage. A bitcoin transaction takes more energy than a million VISA transactions. I find that absurd and very close to being immoral.

        • It seems you are not interested in informing yourself on this topic though, so we need to figure out a workaround.

          That’s rich coming from someone who proposed the use of a public ledger based transaction in order to “keep it anon.” Hey crypto expert, as your own link points out, the point of blockchain is “it’s tough to lie when everybody is watching.”

          “The concept has come into focus in recent years when Ian Grigg who associated it with blockchain technology and popularized it as he believed that accounting should no longer be completely private.”

          The entire premise is that transactions are recorded on a public ledger, mirrored tens of thousands of time by any node that wants to. It’s trivial to track chains of transactions through public keys, and any half decent state actor will have no trouble pinning it down to an irl identity through an initial buy in. It‘s by far the least private way to transact ever invented. I certainly hope you haven’t been operating on that premise. If you value your anonymity, buy a burner phone, register a signal account, and take the goddamn gift card. It’s how actually competent criminals operate.

          Much more than that is spent all the time to secure larger transactions. Just the typical credit card fee is about 2%, so for a $7.5k purchase you are “wasting” the same amount of resources.

          Those aren’t “resources”. The credit card company can increase or decrease their fees arbitrarily—if they were charging 1.5% as they do on charitable donations, or if a large merchant does a good job negotiating for a lower rate, the cost of the transaction in terms of human labor, electricity, rare earth metals in chipsets hasn’t magically decreased. It *is* money being handed to banks, and it’s concerning how much is diverted through banks in this way, but it’s not the same as the massive volume of literal coal being set on fire for a bitcoin transaction.

          Unrelated: your supposed technology of the future can accommodate a whopping 800,000 transactions per DAY at its theoretical upper bound. No you can’t increase the block generation time faster than the settling time. No you can’t increase the block size—that increases the settling time. No you can’t route everything through a layer-2, it’d take at least a half decade of the main chain doing absolutely nothing but onboarding users for every banked person to have bought in. Maybe PoS and sharded chains fix this, but until they actually exist, the technology simply does not work. It only kind-of works right now so long as almost everyone doesn’t use it.

        • Anoneuoid –

          > It seems you are not interested in informing yourself on this topic though…

          Here we go again…

          It’s because you do this so frequently that I point it out. Again, you seem to think you have mind-reading capabilities – and that leads you to a bad faith interpretation of what other people do, when in fact there are many plausible explanations other than the one you come up with.

          It’s a basic logic problem, unless you do in fact have mind-reading abilities – in which case I think you have an obligation to explain to all of us how you got them.

          Aside from being gratuitous and unnecessary at best, this kind of bad faith comment also only undermines your value as a contributor those times when you do make interesting comments.

        • I’d just like to note that this bet is hardly a bet about “seasonality” in COVID, as (1) the metric you’re betting on may or may not be an outgrowth of seasonality pretty much whatever the definition is and, (2) the term in itself (without further clarification) is so vague as to be unquantifiable

        • +1

          part of this seems more like a (pardon the French) pissing match than establishing a causal relationship. It is one time bet, and there are so many factors that affect COVID numbers, that regardless of which side wins the bet, it neither proves nor disproves the underlying point.

          Ah, and now I can settle back with full knowledge that I am about to get flamed!!!

        • Agreed. It seems like this would be a roughly 50:50 bet, given the cycle has already started. All that is being inferred is the approximate shape and width of the curve leading to the decrease. A better set up would be to create a model with seasonality as the major factor driving change in case numbers and see if it can predict when the peaks occur to within about 2-2.5 weeks over 3 cycles and 10-15 countries spread around the world.

        • A problem here is that the influence of “seasonality” in infection rates, and maybe even severity of infection outcomes, can be closely tied to behavior and doesn’t reflect “season” per se. Behavior changes largely associated variance in infection rate can occur across different seasons, such as people traveling from dense urban areas to vacation homes in remote regions. The there’s the influence of timing of NPIs, or frequency of large events or school schedules – potential confounds.

          Often when people talk about “seasonality” and covid they’re tying it to simplistic notions of causality like weather/humidity or immunity and vitamin D from sunlight, without any consideration of confounds or interaction effects or moderators/mediators.

        • It will likely be part of a series of bets.

          I’m still travelling but hopefully by tomorrow I can figure out the best way to do the payments without using crypto.*

          *Cypto is sufficiently anonymous for this purpose and does not waste more energy than the current swift/cc/bank system. Unless you misuse very secure blockchains to do microtransactions or something similarly dumb (that you are automatically punished for via tx fees), its a non issue. You can even use a proof of stake coin if you want… It is funny to read all the NPR level posts here. But either way, I wouldn’t expect phil/etc to overcome the learning curve for this purpose.

        • Anon,
          Please confirm that we have a bet!

          As for your claims about crypto: if you were arguing that crypto won’t necessarily consume enormous amounts of energy in the future, I’d agree. But if you’re arguing that a crypto transaction doesn’t currently consume a lot of energy, that’s just wrong. At any rate I am not willing to do it, so you need to find another way.

        • @Phil

          I don’t think we do but I’m still on mobile in the process of fixing my PC. What are the odds, etc you are proposing? Because I’m not gunna do 9:1 in your favor. We also need to figure out the method of payment.

          Besides that, it is just check the cdc forecast site for the first week of october. I say it’ll be below 500k cases per week.*

          *By last week of Oct, this is even more certain. And then it should be back over 1 million by Jan 2022, unless testing becomes rare or a giant flu season blocks covid infections via viral interference.

        • Anon, if it’s a “near certainty cases will be below 500k per week on Oct 1st” why wouldn’t you accept 9:1? Should be easy money for you (if you decide to use a real currency instead of the worthless stuff).

        • @Unanon

          Why would I do that?

          In betting you want to take advantage of information asymmetries to max your expected profit, not break even!

        • I suppose it depends what one thinks the underlying probability behind “near certainty” is. I interpret this to be greater than, say, 95% or 99%. You’re doing fine at 9:1 if that’s the case. This doesn’t really matter too much. Phil’s giving you 3:1, as he mentioned above. Maybe you should read the responses to your proposal.

        • It isn’t about “doing fine”, according to 28 (seasonality-ignoring) models used by the cdc there is only ~2.5% chance.

          So people who trust the cdc should be giving me 9:1 odds…

          Is there no one here who trusts the cdc enough to give better than 3:1 against me?

        • Anoneuoid –

          To go from this:

          “…it is a near certainty cases will be below 500k per week on Oct 1st, …”

          to

          “…Is there no one here who trusts the cdc enough to give better than 3:1 against me?”

          Looks to me like a total walkback. This didn’t start out about people having “trust” in the CDC, but your claims about what’s nearly certain.

          There’s a long way between “trusting” the CDC and your claims. That’s where Phil’s offer of a bet comes in.

          Should we take it that you’re walking back your assertion about what’s nearly certain?

          And I’m wondering if you could explain how traveling prevents you from settling the terms of the bet with Phil?

        • Anon, quoting myself from two days ago: “Here’s the proposed bet: If there are more than 500K new cases in the week Sept 29 through Oct 5, you pay me $100. If fewer, I pay you $34. Do we have a bet? If I lose I’m sure we can find a way to get you your money.”

          You’ve said you think there’s virtually no chance that the number of new cases that week will be more than 500K, so I don’t understand why you’re balking at this; from your perspective it’s an easy $34.

        • Yes, Josh, it appears Anoneuoid is getting cold feet.

          I wonder if they’ll be traveling until Mid-October. Or maybe have mysterious computer issues?

          Or make up more nonsense reasons to not accept a 9:1 or 3:1 bet, despite “near certainty” of the outcome based on “basic science.”

          Maybe there should be a separate pool to bet on whether Anon goes through with the wager as it currently stands.

  2. I skimmed the second paper. Yes, it was useful. As someone who has built two stochastic models of Covid-19, the paper doesn’t address what’s most important to me: differentiating between independent and dependent variables. For instance, as a verification step of R-0, a non-linear mixed effects model is used on a data set of excess deaths. The purpose is to infer R-0 using independent variables. Depending upon the parameters used, different independent variables can be found, resulting in differing values of R-0.

    Also, the stress on stochastic modeling, in the paper, ignores that solving differential equations requires correctly, first, identifying independent variables.

  3. “these models have systematically failed to account for the social and structural factors which lead to socioeconomic, racial, and geographic health disparities.”

    Seems like the sentence could have ended at “factors”. since one should account for those social and structural factors whether or not they lead to disparities in any specific category. You should be concerned with how those factors affect the spread of disease.

      • I think you actually have to think about the outcome of interest, i.e. infection disparities, when deciding what to leave in and what to leave out. Otherwise you are stuck modeling everything that could impact the outcome as a mechanistic input, when maybe you would have been better off either leaving it to some kind of unstructured random effect or the demographic stochasticity of the model?

  4. > “Anon, quoting myself from two days ago: “Here’s the proposed bet: If there are more than 500K new cases in the week Sept 29 through Oct 5, you pay me $100. If fewer, I pay you $34. Do we have a bet? If I lose I’m sure we can find a way to get you your money.”

    You’ve said you think there’s virtually no chance that the number of new cases that week will be more than 500K, so I don’t understand why you’re balking at this; from your perspective it’s an easy $34.”

    Thanks, it was missed in some paragraph above. The reason for balking at 3:1 is the CDC is saying more like 1:9, so I think I can get much better odds. Apparently there is no one who puts any faith in these models here, your implied prediction interval is something like twice as wide.

    I still haven’t gotten a chance to look into the best alternative for anon transfers though, but yea we can figure something out I’m sure. Enough has been written about it at this point lets just agree to it.

    If anyone wants to gives better than 3:1 odds I’ll consider that too.

  5. BTW, there’s also this.

    You say:

    > They all assume there is no seasonality,

    You provided a link. Following that link, the CDC provides links to the modeling groups whose work m they include in their ensemble. I’m confused whether you examined the link you m provided? This is from 2 minutes of following up the first link I ttried (Columbia University group)

    To characterize their viral and epidemiological properties in support of public health planning, we develop and apply a model-inference system to estimate the changes in transmissibility and immune escape for each variant, based on case and mortality data from the country where each variant emerged. Accounting for under-detection of infection, disease seasonality, concurrent non-pharmaceutical interventions, and mass-vaccination…

    https://www.medrxiv.org/content/10.1101/2021.05.19.21257476v1

    It seems to me that you’ve been arguing that seasonality has some dominantly predictive power for explaining the pandemic’s trajectory, or that it moderates (or mediates?) all other predictive factors – and that in turn leads you to think that not ascribing to seasonality dominantly/moderating (mediating) predictive power = “assum[ing] there is no seasonality?”

    Clearly you were wrong in your statement.

    • I looked into your claims. In the end it amounts to you saying because some of the authors included seasonality in a paper about other countries they are also using that same model for what they submit to the cdc, which is contrary to the documentation provided by the cdc.

Leave a Reply to Roy Cancel reply

Your email address will not be published. Required fields are marked *