Resolving the cathedral/bazaar problem in coronavirus research (and science more generally): Could we follow the model of genetics research (as suggested by some psychology researchers)?

The other day I wrote about the challenge in addressing the pandemic—a worldwide science/engineering problem—using our existing science and engineering infrastructure, which is some mix of government labs and regulatory agencies, private mega-companies, smaller companies, university researchers, and media entities and rich people who can direct attention and resources.

The current system might be the best option available, but one problem is that we see lots of small, messy studies—experiments with small samples and no control groups, surveys with questionable sampling plans and incomplete writeups, a need to balance evidence across similar but not quite comparable studies, and a general unavailability of data.

This all sounds a lot like various subfields of cargo-cult science: the world of ESP studies, embodied cognition, anything-and-sex-ratio, air rage, himmicanes, pizzagate, etc., the PNAS or Psychological Science world in which everything gets publicity, careful and sloppy work alike, where everything’s a conceptual replication, where failures never are acknowledged and battles are fought at land, at sea, in the air, and in the news media.

I’m not saying coronavirus research is junk science. Not at all. Some of it may be bad science, but it all seems to be addressing real questions.

What I’m saying is that coronavirus research, in all its varying levels of quality, is being developed and transmitted in the same sociological matrix that also handles junk science. A system that thrived on the PNAS-NPR-Ted-Gladwell circuit is not being adapted to a much more serious purpose. And, again, even if the current decentralized system driven by career ambitions is the best we can do, it’s clearly flawed, as we already knew.

In my earlier post I referred to the classic dichotomy of “the cathedral”—centralized, closed-source development—versus “the bazaar,” which is decentralized and open source. But, as we discussed in comments, the analogy is imperfect, as one notable flaw of the current “bazaar” is that it’s not open source. People keep coming out with papers and not releasing their data or even their code.

Relevant to all this is a new article, “Psychological Science is Not Yet a Crisis-Ready Discipline,” by Hans IJzerman, Neil Lewis, Netta Weinstein, Lisa DeBruine, Stuart Ritchie, Simine Vazire, Patrick Forscher, Richard Morey, James Ivory, Farid Anvari, Andrew Przybylski, who write:

The threat presented by the new coronavirus disease (henceforth COVID-19) has mobilised public health practitioners, epidemiologists, and policymakers. It has also motivated scientists across a range of disciplines, including psychology, to ask whether their expertise might facilitate effective pandemic responses . . . That said, if we are to empirically understand and intervene in global pandemics, and steer policy in decisions where human lives hang in the balance, humility, caution, and realism are warranted. . . .

Before application can occur, we must first establish systematic guidelines for flagging trustworthy and actionable research findings. . . .

Much of their article is about psychology research and is not so relevant to work in biomedicine or epidemiology, two fields which, whatever their flaws, have questions and research methods that are more settled than those in much of psychology. For example, there has been lots of discussion recently about false-positive antibody tests and variation in the infection fatality ratio, but nobody doubts that antibodies, infections, and fatalities exist (in comparison to, say, implicit bias testing, embodied cognition, or terror management theory, three subfields of social psychology where there is doubt not just on certain empirical findings but of the entire theoretical structure). This is not a slam on psychology: it just means that psychology is a very difficult field to study. Biomedicine and epidemiology are easier.

Anyway, right now I’m not so interested in the particular problems of psychology addressed by IJzerman et al., but I did notice this quote in their paper:

As a start, we should look towards allied fields that have grappled with scaling their evidence to learn from their expertise and experience.

The field of genetics started from a similar position with small, independently collected samples that produced unreliable findings. Attempts to identify candidate genes for many constructs of interest kept stalling . . . In one prominent example, 52 patients provided genetic material for an analysis of the relationship between the 5-HTT gene and major depression, a finding that spurred enormous interest in the biological mechanisms underlying depression. Unfortunately, as with the current situation in psychology, these early results were contradicted by failed replication studies.

Technological advances in genotyping unlocked different approaches for geneticists. Instead of working in isolated teams, geneticists pooled resources via consortium studies and thereby accelerated scientific progress and quality. Their recent studies (with, e.g., N>1,000,000) dwarf previous candidate gene studies in terms of sample size. To accomplish this, geneticists devoted considerable time to developing research workflows, data harmonization systems, and processes that increased the accuracy of their measurements.

This seems very relevant to our problems right now with coronavirus research: lots of small studies, no coordination, no data sharing, scientific papers being unleashed on the world and then fighting each other in social media and the news media like some sort of killer robot drones going at each other . . .

But if the genetics researchers could get together and solve this problem, maybe the biomedical researchers and epidemiologists (and all the non-epidemiologists who are doing bad epidemiology research which then gets unfairly blamed on epidemiologists) could get together too in some way to develop research workflows, data harmonization systems, and processes that increase the accuracy of their measurements.

I don’t think it’s about to happen, but maybe it’s a model for the future?

I don’t think genetics research is perfect—in particular, I have the impression that they may have “harmonized” on some problematic statistical methods, and any standardization has tradeoffs—still, maybe what they do in genetics could represent a way forward for public health research, which is now such a mess.

84 thoughts on “Resolving the cathedral/bazaar problem in coronavirus research (and science more generally): Could we follow the model of genetics research (as suggested by some psychology researchers)?

  1. Andrew – it happening.

    I gave instances in a past comment:

    Wide collaborations are taking place e.g. https://covid.cd2h.org/ (which includes the OHDSI Covid19 group) and in my country, Canada’s COVID-19 immunity task force and I am sure many more.

    But everyone pretty much has to use and put up with current resources and skills – habits are hard to break and most are in the habit of advancing their career and interests.

    The one big break being people working from home have more flexibility.

    However, this remains the challenge having to put up with current resources and skills and existing habits are hard to break and most are in the habit of advancing their career and interests. Along with the risk of eventual group think.

    Maybe others could point out other collaborating groups.

    • > However, this remains the challenge having to put up with current resources and skills and existing habits are hard to break and most are in the habit of advancing their career and interests

      I think this is key. There’s a huge explosion of Covid-19 papers and a sizable fraction of them will be gambles at going viral in the media / amongst political decision-makers. I don’t think carefully putting together data sets and sharing them widely is conducive towards that goal. I think many researchers right now have a fear of missing out

  2. Andrew –

    > maybe what they do in genetics could represent a way forward for public health research, which is now such a mess.

    Can you speculate, mechanistically, a bit more about how you see the “what they do” in genetics might pan out in pandemic-related research?

  3. “52 patients provided genetic material for an analysis of the relationship between the 5-HTT gene and major depression, a finding that spurred enormous interest in the biological mechanisms underlying depression. Unfortunately, as with the current situation in psychology, these early results were contradicted by failed replication studies.”

    Only 52 patients? D:

  4. I find myself facing these issues on many levels. As a researcher, I can’t keep up with the continual release of new data, new models, new critiques. The data and code are often not available, and when available, there is simply too much there, too many voices, and too little time. I am simply overwhelmed by the quantity of information and I start resorting to crude (and probably ineffective) screens – like ignoring everything released on Twitter, paying attention to only a few select places (such that this blog), looking at credentials, etc.

    With my students, this is a great opportunity for teaching moments. But where to start? I have an introductory class with under-prepared and largely disinterested students – do I give them bad data and analysis to look at or do I give them data and studies which they can’t hope to understand, given the few skills they currently have? Or do I just ignore COVID and teach other data and problems that I had been planning on using?

    As a citizen, how do I make sense out of all this? I can’t even decide whether the lockdown was an over-reaction or doing too little. And, the more I see politics at play (it looks a lot like a bandwagon when decisions were made to lock everything down and now it looks like a bandwagon deciding to open things up), the less I trust any of the decisions being made. Skepticism (which comes naturally to me, as well as through my training) is healthy – but it is rapidly becoming unmanageable.

    I think you need a new metaphor. This is certainly not a cathedral. And, most bazaars are more coherent (order through disorder) than this. Perhaps a World Cup match? or this image: https://kikn.com/cowboys-herding-cats/ ?

    • > I can’t even decide whether the lockdown was an over-reaction or doing too little.

      I recommend the articles by Tomas Pueyo on medium.com:
      Coronavirus: Why You Must Act Now
      Coronavirus: The Hammer and the Dance

      The lockdown was an over-reaction, but lack of information left no choice: the spread needed to be stopped early or the hospital system would be overwhelmed, see Wuhan, Italy, New York for proof. We can now adjust this reaction to find a balance between economics, education, and public health, by relaxing some of these measures. Since there is no data on which measure does how much exactly, deciding which measures to select for relaxing is a political process that can have different results in different states. It takes about two weeks to see the effects of any such decision in the statistics, so that has to be a careful process; otherwise, we might incur another two weeks of exponential growth.

      • New York, Wuhan and Italy did lockdowns and their hospital systems were overwhelmed.

        Many other places did lockdowns of similar or lower thoroughness with similar or later timing and did not experience hospital system crashes.

        It almost makes you think that there are factors more important than lockdowns in determining whether hospital systems are overwhelmed. Or is that just crazy talk?

        • The majority of small to medium sized cities in USA have, as far as I can determine, been able to handle the influx of COVID-19 patients without maxing out their ICU capability. Charlotte, NC for example. Springfield, IL for another. Charleston, SC and Columbus, OH.

          And I could go on for dozens more. The places where the hospital system crashed are easier to list than those where it didn’t happen.

          That’s in part because people stayed home, businesses closed, yada, yada, yada. But all those measures were taken in the places where hospitals were overwhelmed, too.

        • NYC was overwhelmed because it took those measures late in terms of doubling times since the Nth community case. The earlier the virus arrived, and the more initial cases there were landing at the airports etc, and the more tightly packed the city, the faster the growth rate and the more cases there were.

          People seem to be thinking about this thing in terms of a calendar date: all these cities closed on the same date, and yet NYC was overwhelmed while others weren’t… so it’s just NYC…

          early on in the whole thing, the number of cases was:

          N0 * 2^((t-t0)/td)

          Pick N0 = 500 for example, then t0 is the day on which 500 people were community transmitting, and td is the doubling time.

          t0 obviously varies widely based on travel patterns and td varies widely based on environmental factors (are people indoors, on subways, in large apartment complexes, vs outdoors, single family homes, small spread out office parks, etc)

          what matters is the quantity ((t-t0)/td) not t which is the calendar date.

        • You also asserted this was done with similar or worse timing.

          North Caroline started cancelling events on March 12, with 15 confirmed cases total in the state. New York State had 1000 case when it went into lockdown, that’s 6 doublings later than North Carolina. North Carolina instituted measures substantially earlier with respect to the spread of the epidemic than New York.

          On March 27, at 763 confirmed cases, North Caroline went into lockdown and finally pushed the growth of the daily cases down to a single digit percentage 2 weeks later, but they haven’t passed the peak yet. The data looks like the measures before the lockdown brought the growth of daily cases down to ~20%.

          I’m not inclined to examine your other examples after the first one clearly failed to support your point.

          OBVIOUSLY lockdown isn’t the only factor affecting the spread, but it is a major factor, and it seems to have an observable effect. Germany staggered its response over 3 weeks, and you can see each measure in the slope of the new infections curve.

          Your point seems to be that lockdowns have no effect, but neither theory nor data support this.

        • Of course lockdowns have an effect, how could they not. But we need to know what, exactly, are the other reasons for health systems to collapse in some places while not collapsing in others. There isn’t a pill called “lockdown” that can be prescribed to cure every outbreak of COVID-19 disease.

          Estimating where, when and under what conditions the next overwhelming outbreak will occur is necessary before workable policies can even be suggested (assuming there’s political will to implement workable policies). Right now the default model seems to be that where, when and why is “anywhere that dares relax the lockdown even a little bit”. Which is nonsense, the reality is more complicated than just “lock everything down until a miracle vaccine arrives”.

        • > OBVIOUSLY lockdown isn’t the only factor affecting the spread, but it is a major factor, and it seems to have an observable effect. Germany staggered its response over 3 weeks, and you can see each measure in the slope of the new infections curve.

          Observing a decrease in the curve does not mean shutdown alone causes the issue. How can you control for other factors exclusive to it, including weather, hygiene, masks, immunity? It is impossible to do so by measuring cases alone.

          Sure, you can fit the data to the SEIR model and get R0 or similar measures, but as others have described in detail (https://www.medrxiv.org/content/10.1101/2020.04.14.20048025v1.full.pdf) such fitting alone is not enough; you cannot delineate effects from other factors besides the shutdown.

        • The most important factor determining whether hospital systems are overwhelmed is the number of infected people (more precisely infected people requiring hospital care relative to hospital capacity). I think most people here understand that the spread of the infection doesn’t happen at the same time or speed everywhere and that the timing of interventions is not measured looking up the date in a calendar.

      • > > I can’t even decide whether the lockdown was an over-reaction or doing too little.

        How about both?

        I think that with public health policy, if you have unambiguously good results people will say you were alarmist, if you have unambiguously bad results people will say you reacted too slowly. If your results can in any way be seen as mixed, you will be accused of both.

    • Dale,

      Something to consider:

      Assume it takes 11d to develop symptoms after infection with SARS-CoV-2 (source: https://annals.org/aim/fullarticle/2762808/incubation-period-coronavirus-disease-2019-covid-19-from-publicly-reported).

      Consider data from NJ. Their peak deaths/case occurred on April 6 with only marginal increases in per day counts for the past week (source: https://www.nj.gov/health/cd/documents/topics/NCOV/COVID_Confirmed_Case_Summary.pdf). Therefore, peak infections occurred on March 26.

      NJ enacted mass shutdown on March 21. Assuming it took ~1-2 days to enforce effectively (some counties, like Lakewood, were major problem), then true Social Distancing occurred on March 23.

      Therefore, it seems NJ enacted their shutdown too late. They curve was already flattening itself. If otherwise, there would have been a peak in deaths/cases later than April 6.

      If anyone seems gaps in the above logic, please say so! I don’t want to be misleading with the above analysis.

      • “The median incubation period was estimated to be 5.1 days (95% CI, 4.5 to 5.8 days)”, says your paper. You say “assume it takes 11d”. WHY?

        The “Illness Onset Date” graph shows new cases flattening on March 30.
        March 19 businesses closed
        March 21 stay at home order
        Assume 5 days for household infections, then 5 more days for the new cases to show an effect?

        The curve didn’t “flatten itself”. You have zero evidence that the curves flatten themselves.

      • According to wikipedia: “On March 16 […] Gatherings of 50 or more people were prohibited […] All bars, casinos, gyms, movie theaters, and restaurants were closed indefinitely […] [The Governor] announced that all of the state’s schools, colleges, and universities would close indefinitely on the 18th. By this time, most of the state’s school districts had closed already.”

      • Mendel,

        > “The median incubation period was estimated to be 5.1 days (95% CI, 4.5 to 5.8 days)”, says your paper. You say “assume it takes 11d”. WHY?

        Per the paper: “Fitting the log-normal model to all cases, we estimated the median incubation period of COVID-19 to be 5.1 days (CI, 4.5 to 5.8 days) (Figure 2). We estimated that fewer than 2.5% of infected persons will show symptoms within 2.2 days (CI, 1.8 to 2.9 days) of exposure, and symptom onset will occur within 11.5 days (CI, 8.2 to 15.6 days) for 97.5% of infected persons. The estimate of the dispersion parameter was 1.52 (CI, 1.32 to 1.72), and the estimated mean incubation period was 5.5 days.”

        It seems the paper, per Figure 2, defines “incubation period” as period it takes for 50% of infections to show symptoms. So if April 1, for example, was the date of maximum infection, then why would the curve start to decline only 5d later? Should it not continue to increase beyond April 6 and peak later? Maybe I am missing something or oversimplifying the analysis too much.

        Although, the COVIDView Weekly Summary for the CDC states 2020W12 (March 16 – March 22) was the peak (106,972 cases) of COVID Like Illnesses (CLI) reporting to ERs nationwide. Since the first cases in NJ were mid-February, my prior is that since this curve starts to increase dramatically mid-February, it approximates the cases in NJ. (source: https://www.cdc.gov/coronavirus/2019-ncov/covid-data/covidview/04242020/covid-like-illness.html) Maybe this is too broad an assumption, though. I’d like to see COVIDView data for NJ only, but it doesn’t seem to be available

        Carlos,

        A major caveat to the March 16, March 18, and March 20 dates — they coincided with a mass influx of college students returning homes from across the US, whom could have brought the infection with them. So while said closings of schools, businesses, universities in NJ may have helped, there were other factors at play until most universities closed nationwide by ~March 23.

    • Addendum:

      I do understand that NJ had no way, at the time, of determine true number of infections since mass testing started too late. So I’m not trying to place blame. My above point is more retrospective analysis to consider moving forward (“hindsight is always 20/20” as they say).

      • Retrospectively, according to both Mendel and Carlos above, the NJ curve flattened precisely because of a gradual move towards lockdown starting March 16 and continuing onward, including *VOLUNTARY* rather than mandated closures and restrictions by individuals and local schools etc.

        curves don’t flatten themselves, unless you mean because people start restricting their motions even before its mandated.

      • Daniel,

        > Retrospectively, according to both Mendel and Carlos above, the NJ curve flattened precisely because of a gradual move towards lockdown starting March 16 and continuing onward, including *VOLUNTARY* rather than mandated closures and restrictions by individuals and local schools etc.

        What you say here is basically the question I am asking: Voluntary and common-sense practices like better hygiene, avoiding major gatherings, etc., — how people behave when any bad illness is spreading (like the damned norovirus, agh) — started ~2-3 weeks earlier than the shutdown; therefore, the timing of the peak suggests that these effects, not necessarily the shutdown, reduced infections. Maybe they didn’t, though. I haven’t seen data that convinces me yes or no to date.

        • Twian –

          > therefore, the timing of the peak suggests that these effects, not necessarily the shutdown, reduced infections. Maybe they didn’t, though. I haven’t seen data that convinces me yes or no to date.

          I agree. I have been making the same argument in the other direction: That unless you did some kind of very sophisticated sensitive analysis on the different variables, you can’t legitimately say that rhe mandated social distancing *hasn’t* had a measurable impact.

          That said, I personally think the mandates were justifiable on the basis of the precautionary principle.

        • I know I’m really being snarky here but I’m tempted to agree with your statement, “you can’t legitimately say that rhe mandated social distancing *hasn’t* had a measurable impact.” but add that there’s never going to be a “measurable effect” unless we start actually measuring.

        • Brent –

          I don’t get why that’s snarky. It’s true.

          But the same does go the other way.

          We’ll never know the impact either way unless a very sophisticated sensitivity analysis is conducted to weigh the relative impact of the various interventions and non-interventional behavioral changes.

          And even then, we’ll only have an educated guess.

        • Joshua,

          > I agree. I have been making the same argument in the other direction: That unless you did some kind of very sophisticated sensitive analysis on the different variables, you can’t legitimately say that rhe mandated social distancing *hasn’t* had a measurable impact.

          Aye — this is the crux of my argument. I am personally seeing more and more scientific discourse attempting to justify the shutdowns from available data; but you simple can’t do it; the resolution is not there to deconvolve the multiple other factors and their affects.

          I too personally agree that in the short-term the shutdowns and such were reasonable: we had limited data and had to mitigate risk or potentially face severe consequences.

          The problem, I think, is what seems to be the popular rhetoric used to justify said shutdowns: “The shutdown worked. Here are some qualitative observations and esoteric modeling (to the average person) that explains why it works. Based on this information, shutdowns are going to be the new norm for future pandemics” Such rhetoric can seem didactic and dogmatic and therefore lead to dissent among some.

          I don’t think that discourse, be it among scientists, politicians, etc., is productive. Why not instead say: “Hey, we understand this shutdown was difficult; maybe it was excessive in hindsight. But we did what we thought was best to save lives and learned lessons that will improve our response for next time. This includes mitigating effects to daily life and the economy as best we can while not sacrificing safety.”

          I know many sensible people, who don’t have any extreme leanings Right or Left, frustrated by the first rhetoric and would like to see the second.

        • Twain –

          You probably could guess I’m going to spin that back the other way.

          > Aye — this is the crux of my argument. I am personally seeing more and more scientific discourse attempting to justify the shutdowns from available data; but you simple can’t do it; the resolution is not there to deconvolve the multiple other factors and their affects.

          I too personally agree that in the short-term the shutdowns and such were reasonable: we had limited data and had to mitigate risk or potentially face severe consequences.

          The problem, I think, is what seems to be the popular rhetoric used to justify said shutdowns: “The shutdown worked. Here are some qualitative observations and esoteric modeling (to the average person) that explains why it works. Based on this information, shutdowns are going to be the new norm for future pandemics” Such rhetoric can seem didactic and dogmatic and therefore lead to dissent among some.

          I don’t think that discourse, be it among scientists, politicians, etc., is productive. Why not instead say: “Hey, we understand this shutdown was difficult; maybe it was excessive in hindsight. But we did what we thought was best to save lives and learned lessons that will improve our response for next time. This includes mitigating effects to daily life and the economy as best we can while not sacrificing safety.”

          I know many sensible people, who don’t have any extreme leanings Right or Left, frustrated by the first rhetoric and would like to see the second.

        • Something weird about this website and my phone keyboard…

          To spin that the other way:

          > Aye — this is the crux of my argument. I am personally seeing more and more scientific discourse attempting to justify the shutdowns from available data; but you simple can’t do it; the resolution is not there to deconvolve the multiple other factors and their affects.

          I am seeing more and more scientific and non-scientific discourse trying to support the argument that the mandated social distancing (I reflexively don’t like “shutdowns” as a descriptor) was a panic move that will unnecessarily hurt people, cause undue deaths, etc.; but you imply can’t do that, IMO – unless you do a sensitivity analysis.

          I read one post at a “skeptic” website where a “skeptic” much ballyhooed for his statistical chops tried to just add up the different interventions to say that the different curves weren’t associated with the different absolute numbers of interventions. I couldn’t believe it. He didn’t even consider if some of the interventions were universal, or if some that they all did had more of an effect than some that only some of them did, etc. I was amazed at how bad his analysis was.

          > The problem, I think, is what seems to be the popular rhetoric used to justify said shutdowns: “The shutdown worked. Here are some qualitative observations and esoteric modeling (to the average person) that explains why it works. Based on this information, shutdowns are going to be the new norm for future pandemics” Such rhetoric can seem didactic and dogmatic and therefore lead to dissent among some.

          But I see that as a tribalistic process – not one of simple and objective analysis. I see that reaction as largely driven by ideological proclivity. But even more – I see that also running in the other direction: “The shutdowns are a panic move by totalitarian libz who just want to hurt Trump and who want to run a nanny state and rob us of our fundamental freedoms. They wanted to do this anyway, and this ‘pandemic’ is just an excuse. Open the damn country and let us get back to work.”

          > I don’t think that discourse, be it among scientists, politicians, etc., is productive.

          I’ll agree and run it back at you.

          > Why not instead say: “Hey, we understand this shutdown was difficult; maybe it was excessive in hindsight. But we did what we thought was best to save lives and learned lessons that will improve our response for next time. This includes mitigating effects to daily life and the economy as best we can while not sacrificing safety.”

          Or, “Hey, we understand that the shutdown was hard on many people, but maybe it has actually lessened harm significantly in many ways. But we recognize that these are tough calls, and maybe…..well, you get where I’m going with that.

          > I know many sensible people, who don’t have any extreme leanings Right or Left, frustrated by the first rhetoric and would like to see the second.

          And you know where I’m going with that.

        • As we’ve discussed, part of my default “prior” is that extra weight is given to the welfare of people on the front line. Hospital workers, doctors, nurses, grocery store workers, transportation workers, etc., many of whom work for very limited wages. IMO, that needs to be factored into a sensitivity analysis.

          I can only reduce my risk, effectively, at their expense.

        • Joshua,

          I figured you would ;-).

          Daniel,

          Bill Gates and others claiming vaccines are our only hope need to read Chapter 16 of Janeway’s Immunobiology or something similar to see just how difficult and complex developing vaccines can be.

        • Twain –

          Related to evaluating the influences of different policies and different behavioral adaptations:

          > Also, the social distancing policy is not a problem in Sweden, because that is how it works normally. It is more than a joke when quoting the swedish reaction to the distancing rule of 2m: “that close?!!”. We don’t adress or greet strangers. Chit-chatting about nothing in public is not heard of except from the large immigrant minorities and the occasional “boomer” missing the good old days of demonstrations for some obscure communist guerilla, made up of three letters…We are fullfilling the demands of social distancing without effort.

          https://judithcurry.com/2020/04/26/a-sensible-covid-19-exit-strategy-for-the-uk/#comment-915893

          That joke is pretty funny, even if it is more than a joke.

        • Joshua,

          It is interesting (and funny, in a sense) how societal norms that seem inconsequential could have great consequence.

          NJ, NYC, Boston, Philly, are all similar in that you don’t really acknowledge people for even the time of day…in contrast, other regions of the US are much more friendly, open, etc. Then there is Italy — and that was a problem.

          Proving this is almost impossible, but its intriguing to consider. Nice point!

        • Entire public health establishment: “We’re going to lock down because we know this is the only thing that will control the spread. If it isn’t too late, people afterwards will call us overreacting and ask us to apologize, if it’s too late hundreds of thousands to millions of people will die… it’s a good thing for all those guys that we know something about how disease works… let’s hope they call us overreactors”

          Terry (several weeks later):”maybe it was excessive in hindsight”

        • ack, I should have said “Twain” not “Terry” sorry, the relevant attribution had scrolled up off my screen.

        • Daniel,

          Saying “maybe it was excessive in hindsight” and also agreeing with the quote you provided are not exclusive. There is no reason someone can’t believe both.

          We may look back and say the shutdown, lockdown, etc., was excessive. We should learn from that: How can we do better the future? What data could we have collected that could have mitigated the length or severity of the shutdown? How did our models succeed and fail? How did our systems for reporting, aggregating, and analyzing data fail? With all the effort, how did we still fail to protect LTCFs (or did we not fail)?

          And I’m not claiming we won’t.

        • I think there is zero evidence the shutdown lockdown was excessive. All the evidence says that death rates skyrocketed even with the shutdowns.

          What makes you think they were excessive? You have to have an estimate of what would have happened without such “excessive” lockdowns in order to make that claim. What is your estimate of what would have happened under some counterfactual, and what counterfactual would you think was “not excessive”?

        • Well – what level of prevented bad outcome would justify what’s been done?

          Given that we did basically nothing during 1957 and 1968 pandemics, and did some social distancing but less than this during the 1918 one, I would argue that what we’ve done was definitely excessive – at least by historical standards – if the “unmitigated” effect of the disease would be closer to 1957/1968 than to 1918, and possibly excessive if comparable to 1918.

        • Daniel,

          You misunderstand. I’m not making any claims for what I believe; rather, I’m stating potential counterpoints for the sake of socratic method.

          > I think there is zero evidence the shutdown lockdown was excessive. All the evidence says that death rates skyrocketed even with the shutdowns.

          If all the evidence shows deathrates skyrocketed even with the shutdowns, it therefore questions how effective the shutdowns were at achieving their purpose (e.g., “crushing the curve”).

          > What is your estimate of what would have happened under some counterfactual, and what counterfactual would you think was “not excessive”?

          We can never know what truly would have happened if we didn’t shutdown — therefore someone can produce no counterfactual with real, sampled data.

        • Twain, I’m not asking you “what truly would have happened” I’m asking you what is your prediction for what would have happened which might cause you to believe that the shutdown was excessive… but then you’re saying in the same post that you don’t necessarily believe that the shutdown was excessive… I consider this all to be essentially fluff.

          You’re saying “if someone were to figure out in the future that the shutdowns were excessive, we should all admit to it” which has the logical structure “If A is true then admit A is true”… fine.

          But it is just a rhetorical device to bring in the idea that “I think the shutdowns were excessive but I’m not going to admit it out loud”

        • Sorry — I was being a bit dense there. Now I see what you mean. Below is my stance on how I think the shutdown was excessive in some ways, but not entirely. Be warned, its long.

          > You’re saying “if someone were to figure out in the future that the shutdowns were excessive, we should all admit to it” which has the logical structure “If A is true then admit A is true”… fine.

          Exactly. Accountability is important. Without it, people often use the Post Hoc fallacy, e.g., “Since event Y followed event X, event Y must have been caused by event X.”

          Now for my thoughts on the shutdown.

          How I think the shutdown was excessive:

          1. Closing universities was not necessary.

          Data from Italy already clearly showed risk for young people, even those with underlying conditions, was very low: 8 deaths age 0-29; that sample is so low, its very likely its outliers with already severe morbidity not representative of the average person age 0-29; could not find data on hospitalization by age. Using NYC as proxy and assuming the 20% seroprevalence of infection is accurate, the likelihood of hospitalization for anyone age 0-17 (the lump age 18-44 together for some reason…) is 0.4% — again, low (if you exclude underling conditions) then it is 0.01%. Using NYC again,

          Therefore, sending students home from their universities — most of which occurred between March 16-23 — caused a super-spreading event nationwide that counteracted the shutdowns simultaneously taking place. Keeping students at schools would have eliminate this issue while also promoting immunity so some degree.

          2. Preventing healthy people age 0-49 from working was not necessary.

          Same reasons as above. Those who are healthy or have well-managed underlying conditions have minimal risk of severe outcomes: 0.007%. This predicates, however, that companies request all elderly and at-risk employees work from home for 2-3 weeks. As with above, this promotes immunity at low-risk while also keeping some parts of the economy moving.

          Calculation: 68 deaths no underlying conditions / (8,400,000 people * 0.88 age 0-64 * 0.60 healthy * 0.20 infection seroprevelance) = 0.007%. Note: This uses obesity as proxy for underlying conditions (so 40% with underlying conditions)

          3. Closing K-12 schools was not necessary and could have offered an excellent tool for tracking COVID-19.

          This paper offers some interesting thoughts on why closing schools is useful beyond building immunity (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2862335/). Children are easy to sample from: have them receive weekly inspections for any symptoms. Also have teachers be vigilant for symptoms and immediately send students to isolation and for testing if symptoms present. You could even have children complete a simple form each day on their symptoms. Once serology testing is available, mandate it for students. The paper discusses some of these more.

          Schools also guarantee sanitation, via nightly cleanings or similar, not possible to guarantee at home.

          Of course, some modifications may be necessary: Since COVID-19 is airborne (to what degree, still unknown), mandating masks, adjusting classroom setups, prohibiting any crowds, having half students come in morning and half in afternoon, etc.

          4. Close non-essential businesses but keeping essential ones, like supermarkets, open and relatively unmonitored for weeks into the pandemic.

          Supermarkets and big-box stores are super-spreading sites. Many did not mandate masks until 1-3 weeks ago in my locales; this seems to be the case nationwide based on media reports (if they are reliable, thought…). Even with masks and distancing — all its takes is one person, the wrong luck with HVAC or cleaning, and a super-spreading event occurs.

          I understand these places must remain open, but the laissez-fair way they were allowed to operate, often into mid-Aptril, likely hampered the efforts of the shutdown.

          How I think the shutdown was reasonable:

          1. Closing restaurants, large gatherings, etc., and not opening them until we fully understood the prevalence and risk of the virus.

          2. Mandating that companies have at-risk employees work from home until we understand the prevalence and risk of the virus. Also allow anyone else who wants to isolate at home do so.

          3. Strictly monitoring travel for those with symptoms. (This seemed to vary much across the US, though; a more cohesive response like that in Taiwan and South Korea would have been much better.)

          How I think the shutdown failed:

          1. We did not protect the most vulnerable — those in LTCFs. For 23 reporting states, ~34% of all deaths were residents of LTFCs. This is not acceptable and should be a focal point for improving our future responses.

        • Twain –

          Just one piece:

          > Data from Italy already clearly showed risk for young people, even those with underlying conditions, was very low: 8 deaths age 0-29; that sample is so low, its very likely its outliers with already severe morbidity not representative of the average person age 0-29; could not find data on hospitalization by age. Using NYC as proxy and assuming the 20% seroprevalence of infection is accurate, the likelihood of hospitalization for anyone age 0-17 (the lump age 18-44 together for some reason…) is 0.4% — again, low (if you exclude underling conditions) then it is 0.01%. Using NYC again,

          Therefore, sending students home from their universities — most of which occurred between March 16-23 — caused a super-spreading event nationwide that counteracted the shutdowns simultaneously taking place. Keeping students at schools would have eliminate this issue while also promoting immunity so some degree.

          _____

          Focusing only on mortality risk misses much of the impact of the virus. Imagine dorms full of students in very close contact with each other. In classrooms with very close contact with each other. Now maybe a high % were already infected when they were sent home, and they then went on to infect their families. If we had any contact tracing in this country we might have some idea, but we don’t.

          But imagine that many of them weren’t infected at the time they were sent home, and those who were sent home isolated themselves (I happen to know many who did), then their spread to their families would have been somewhat limited. And if they had stayed, there would be rather shortly a student body of largely infected students. Guess what, students have teachers. They tend to be somewhat aged. And schools have many employees who aren’t teachers, and many of them aren’t necessarily aged, but they live in relatively high risk communities. And students don’t just limit themselves to the classroom and the dorm. Particularly if the school is in a city, you then have a group of people infected at a very high prevalence, walking around cities and taking public transportation and going into stores – and they are very likely a group that is particularly prone to ignore instructions for social distancing…

          Hmmm. That doesn’t sound like a pretty picture to me. What I do know is that even in a place like Sweden, while they kept younger students in school they closed many higher ed institutions. And I have heard people who model this stuff talking about data which recommends closing higher ed institutions but keeping schools for younger students open. I don’t know what their reasoning was…but maybe I managed to stumble on one or two of the reasons (blind squirrels, stopped clock, you know).

        • Joshua,

          > But imagine that many of them weren’t infected at the time they were sent home […] Particularly if the school is in a city, you then have a group of people infected at a very high prevalence, walking around cities and taking public transportation and going into stores – and they are very likely a group that is particularly prone to ignore instructions for social distancing…

          You make a valid point, but a few caveats to highlight:

          1. My point about super-spreading comes from its timing: Closing of universities coincided with shutdowns. Sure, students would only spread to their families — but imagine if even 5-10% of students infected their parents? That would be a mass event of infections. (Its hard to contain infection within the home.)

          2. I did not mean continuing universities per usual; sorry for not being more precise. Mandate curfews. Close local bars. Close gyms, etc. Survey students for symptoms via surveys, etc. Mandate masks in-class. Monitor dorms for parties, apartments, houses, etc., for large parties (most PD should be very adept at this!). This allows for interaction among the low-risk population while mitigating spread to outside population.

          3. “And if they had stayed, there would be rather shortly a student body of largely infected students.” I’m not sure the evidence supports this. Especially if seroprevlance but no symptoms is high among young people (current data and immunology seem to support this; but without actual seroprevalence data, we’ll never really know).

        • Twain –

          > 2. I did not mean continuing universities per usual; sorry for not being more precise. Mandate curfews. Close local bars. Close gyms, etc. Survey students for symptoms via surveys, etc. Mandate masks in-class. Monitor dorms for parties, apartments, houses, etc., for large parties (most PD should be very adept at this!). This allows for interaction among the low-risk population while mitigating spread to outside population.

          What kind of curfews? You’re going to mandate that they stay in their dorms at night and ensure that the prevalence of infection in that cohort is even higher? And then they’re going to go outside in public during the day. And you close all the stores are don’t let them walk around in public? Can you imagine thousands of students siting around in dorms all day and not being let out? Have you ever rented an apartment to a group of students? They would demolish the school in a matter of days. I can only imagine a nightmare.

          Sorry, I don’t think that your scenario is very realistic. And I’m someone who when I first heard that they were shipping students home who thought that was nuts. And I still think that the short timeline students were given to vacate housing was nuts – especially because from what I saw they didn’t provide alternative housing for those students who had nowhere to go or no means to get anywhere. But when I thought about it more it seemed to me like the less worse of the options available.

          > 3. “And if they had stayed, there would be rather shortly a student body of largely infected students.” I’m not sure the evidence supports this. Especially if seroprevlance but no symptoms is high among young people (current data and immunology seem to support this; but without actual seroprevalence data, we’ll never really know).

          I’m confused by that comment. My understanding is that even those who are asymptomatic are likely infectious (and most infectious before they develop symptoms). Am I wrong about that?

    • “I can’t even decide whether the lockdown was an over-reaction or doing too little. ”

      IMO the early lockdown was necessary to assess the situation. Unfortunately, the assessment was poor and the lockdowns have been extended to compensate for the stupendous incompetence of leadership in thinking through the problem.

      The flood of models has been scientific and public health disaster. No one is thinking about how to solve the problem. Everyone is thinking about how to get better data so a model can solve the problem for us. WE NEED SCIENTISTS TO THINK ABOUT HOW TO SOLVE THE PROBLEM, NOT ABOUT HOW TO BUILD MODELS AND GET DATA. It’s obvious the problem is solvable without models because many countries have managed it well.

      Every jurisdiction in lock-down or with a stay-home order has had nearly two months to gather its shit and work out the opening plan, so if they don’t have one yet extending the lockdown is substitute for competent planning. So on the one hand I don’t really support the protests, but OTOH they have a point: it’s time for a clearly articulated and justified plan with a timeline.

      • I have no faith at all in USA politicians doing anything in the foreseeable future other than over-reacting in first one direction then another with no coherent, data-driven decision rules. It appears those of every “side” or tribe believe two things. One is if/when the lockdown is relaxed there will be another worsening in the COVID-19 crisis. The other is if they do not soon relax the lockdown there will be a complete economic collapse and rioting in the streets.

        Both beliefs are justified. But for some reason it is causing “paralysis by politics” at both the national and state levels. I have seen no movement toward a principled, cautious course of action that navigates a middle course accepting both some moderate level of continued COVID-19 and some moderate level of economic and societal damage.

        It’s like the whole world is holding its breath until some miracle occurs that will stop people from dying of COVID-19 while also allowing the economy to recover quickly. Wishful thinking and fear instead of reasoned action.

      • > WE NEED SCIENTISTS TO THINK ABOUT HOW TO SOLVE THE PROBLEM

        You need to stop assiuming things you don’t know about don’t exist.

        https://www.pandemictesting.org/

        (APRIL 20, 2020) — Today, a bipartisan group of experts in economics, public health, technology and ethics from across the country released the nation’s first comprehensive operational roadmap for mobilizing and reopening the U.S. economy in the midst of the COVID-19 crisis.

        Connect to better sources of information!

        • Thanks for this! I think it supports part of jim’s point though. Here’s a group saying what seems reasonable to me: ramp up testing to 5-20M tests a day, hire a gazillion contact tracers, and isolate cases to hotels and things…

          None of those are actual government plans, and will continue to not be government plans for the full duration of the time Trump is in office IMHO. States don’t have the resources, only the Feds can print/borrow the money needed to reallocate the resources.

          So, we will do something else, it will involve return to work, unless there’s a massive seasonal change in severity and transmission, many people will get sick and die, and the economy will tank after wave 2.

          By the time we’ve got those resources online, we’ll have herd immunity from the debacle anyway and we won’t really need them, and it’ll be too late to save all the people who died or had long term disability.

        • > None of those are actual government plans

          Yes. Because the talking point was plans by scientists, not plans by government.

          Inasmuch as CDC and NIH are government agencies, they had (and still have) plans for pandemics. They did these occasional table simulations in the White House, once shortly before the inauguration, and I think once more later. And then the White House ignored the plans when the pandemic struck. Reading the PDF with the “Red Dawn” emails that the NYT published is painful: there are experts saying in real time, back in February, that the very same mistakes that everyone makes when they do these simulation sessions for the first time are being repeated, and they’re helpless to do anything. Nancy Messenier tried her best late February/early March, but couldn’t handle Trump and hasn’t been a public voice since.

          But there are plenty of pandemic plans, just dip into the conspiracy theory community, they’ve found them all and are using them as evidence that the epidemic was engineered, forgetting that planning for a probable catastrophe proves foresight, not intent. But they exist.

          The CDC and the NIH know which indicators to watch for when it comes to relaxing restrictions. They can advice the government, as the WHO does across the world. I do not doubt there is a lack of plans going into politics, but to use the freedom inherent in these plans to craft a sensible policy, and to coordinate policies across the country to not confuse the citizens and effect better buy-in is a political issue that must be negotiated. But giovernment needs to be willing to listen.

        • Great, thanks for pointing that out.

          But I can see right away why states aren’t using this “model” (sensu lato):

          1) it’s dependent on testing capacity that won’t be available in the relevant time frame.

          2) it’s time frame for economic relief isn’t acceptable for most people: 10% of non-essential workers by late July?

          So some scientists are thinking about it, but in terms that aren’t remotely practical.

        • Yes, there are all kinds of plans floating about that assume away the intractable parts of the problems.

          As it was explained by several people to me in the early days of the lockdown frenzy (circa March 10) here was the consensus among the supposedly enlightened planners at the national and state level:

          1) Lock everything down quickly and severely
          2) Consider some relaxing of lockdown once warm weather causes a downturn in the rate of spread
          3) By summer have people back at work and school, watched over by a universal test/trace/quarantine regime to idenify and isolate newly infected persons
          4) Continue watchfulness and test/trace/quarantine for a year or two until a vaccine arrives

          My immediate reaction to this “plan” has never really changed. Items 2, 3 and 4 were pure wishful thinking. The fallback plan when these dreams didn’t materialize was to continue the lockdown for as long as it takes.

          So the whole “plan” boils down to let’s just tell everyone to hunker down and wait for the cavalry to arrive.

        • You have this “immediate reaction” but you never ever ever ever ever ever ever give a quantitative reason why we can’t do whatever it is you assume is both essential to the plan and absolutely impossible.

          It’s always just “that plan sucks and is impossible… next”

      • “It’s obvious the problem is solvable without models because many countries have managed it well.”
        1) Given that countries were ‘seeded’ by the virus by different routes and different amounts, and given that governments adopted a variety of strategies at different points in time, some inevitably would see better progressions than others.
        2) There is always a model. Sometimes it is implicit, but revealed by the actions taken. One takes certain actions if one believes the virus to be highly contagious but not especially deadly; a different set of actions if one believes it to be deadly but not particularly contagious. And of course actions will be very different depending on one’s beliefs about the mode of transmission, from what vectors, etc.

  5. The sharing of data and research is happening.

    WHO: The Unity Studies: Early Investigations Protocols

    The emergence of a new virus means that understanding transmission patterns, severity, clinical features and risk factors for infection will be limited at the start of an outbreak. To address these unknowns, WHO has provided Four Early Investigation Protocols (rebranded the WHO Unity Studies).

    These protocols are designed to rapidly and systematically collect and share data in a format that facilitates aggregation, tabulation and analysis across different settings globally. One additional study to evaluate environmental contamination of COVID-19 is also provided.

    Data collected using these investigation protocols will be critical to refine recommendations for case definitions and surveillance, characterize key epidemiological features of COVID-19, help understand spread, severity, spectrum of disease, and impact on the community and to inform guidance for application of countermeasures such as case isolation and contact tracing.

    https://www.who.int/emergencies/diseases/novel-coronavirus-2019/technical-guidance/early-investigations

    WHO: “Solidarity” clinical trial for COVID-19 treatments

    “Solidarity” is an international clinical trial to help find an effective treatment for COVID-19, launched by the World Health Organization and partners.

    The Solidarity Trial will compare four treatment options against standard of care, to assess their relative effectiveness against COVID-19. By enrolling patients in multiple countries, the Solidarity Trial aims to rapidly discover whether any of the drugs slow disease progression or improve survival. Other drugs can be added based on emerging evidence.
    [..]
    Enrolling patients in one single randomized trial will help facilitate the rapid worldwide comparison of unproven treatments. This will overcome the risk of multiple small trials not generating the strong evidence needed to determine the relative effectiveness of potential treatments.
    [..]
    The Solidarity Trial provides simplified procedures to enable even overloaded hospitals to participate, with no paperwork required. As of April 21 2020, over 100 countries are working together to find effective therapeutics as soon as possible, via the trial.

    https://www.who.int/emergencies/diseases/novel-coronavirus-2019/global-research-on-novel-coronavirus-2019-ncov/solidarity-clinical-trial-for-covid-19-treatments

    In the April 29 WHO press briefing, after 36:40, Maria van Kerkhove talked a little about the way the WHO assimilates new evidence using “living reviews”.

    There are studies that are coming out, particularly on therapeutics but in all aspects of this novel coronavirus, this new coronavirus causing this pandemic. It’s something within the scientific community; what we try to do when we evaluate the evidence is we look at what we call the weight of the evidence where we pull together all available evidence, all available studies on any particular topic, whether these studies are conducted in a laboratory through experimental conditions, whether they’re done in observational studies or epidemiologic studies in people, whether they’re done through clinical trials.
    [..]
    Then we go one step beyond that. Then we debate the results with our global expert networks and this is a healthy debate, it’s a constructive debate where we actually look again and we say, what does this tell us and what does this mean in terms of our guidance to our member states, to all people all over the world.
    [..]
    Right now what we are doing at WHO is we’re working with our science division and we’re working with partners at GOARN who are conducting living reviews for us so every day we are looking at the publications that come out in the peer-reviewed journals and the publications that are being sent to us before they reach the journals and we’re conducting living reviews on about 30 topics right now so that we can stay in tune with what is coming out.

    https://www.who.int/emergencies/diseases/novel-coronavirus-2019/media-resources/press-briefings

    “GOARN has grown to now have over 600 partners in the form of public health institutions, networks, laboratories, and United Nations and non-governmental organizations.” (Wikipedia)

    So, the public may see “scientific papers being unleashed on the world and then fighting each other in social media and the news media like some sort of killer robot drones going at each other”, but behind the scenes, it does look somewhat different. “Public Health Instutions worldwide doing their job” just doesn’t make for good news.

    • I don’t think people question this is occurring. But there is a dissemination market which didn’t exist in the past. That market is highly media driven: can you print today’s click grabber is now can you print today’s ‘scientific’ click grabber. We eventually end up with a science/medical result but that is largely separate from the self-hype mechanisms people use to get out their versions of stories (like it’s air pollution) or the self-hype methods people use to gain attention (like my lab found this result). And it really does matter. The markets went nuts upward on the report that a drug whose name my computer constantly autocorrects may help people who otherwise would recover recover a bit faster in certain circumstances. The media hype machine makes us teeter back and forth. And that is not a good environment for making decisions.

  6. +++ this

    Examples: many straw horses, some as stupid as the NYT’s widely disseminating article with not good graphs of excess deaths that contains this line right at the top: ‘These increases belie arguments that the virus is only killing people who would have died anyway from other causes.’ Do people actually believe something like that? Even a person 100 years old has a life expectancy of a year or more. If deaths were lower or the same, then why would anyone even notice a new disease? I mean sure we all die ‘anyway’ but tell that to life insurers when they price policies. I’m noting this though the source isn’t a ‘researcher’ or ‘scientist’ because the idea behind it is deeply confused unless you take it as erecting a straw horse on purpose, which I doubt.

    A researcher connects cases and deaths in Chelsea, MA to air pollution. There may be a connection at some point, but aren’t there a few other readily available explanations, like density, like occupations that don’t allow working at home or which place people in repeated contact with others? But that’s the point they wanted to make. So they did.

    A number of researchers have tested for the molecular signature of the virus. Saying that it can be detected is not the same as understanding whether that matters. But if what you do is that kind of work, then you do that kind of work. It isn’t junk work except it might not be meaningful and we don’t know.

    • I hit submit while not paying attention. I meant to say that it’s not possible to organize research in the manner suggested because times have changed. There used to be a small group working on whatever issues seized the moment. You could get nearly all the top physicists working on what became quantum mechanics into a room. The genetic code was worked out by small groups that knew each other.

      And a lot of the people doing work are doctors or medical researchers who are used to doing their own work, partly because that’s always been rooted in clinical work and the data which comes out of clinical settings. There’s a difference between clinical observation and measuring energies from colliding particles: you explain what you’re doing with the latter and see if others replicate or have a more complete explanation. You can’t replicate clinical experiences. And you rely on what are often highly imprecise if not inaccurate estimations and observations rather than actual numbers and actual math.

    • Jonathan, trust me, it’s not a strawman. There’s a German ex-parlamentarian and physician who is convinced the swine flu was a hoax (because it didn’t reach Europe in numbers), and stated in a *widely* shared video that I have seen myself that SARS-CoV-2 was just another cold virus that we’re only noticing because we’re specifically looking for it. Obviously, that was in early March, if I recall that correctly. So his idea was that this was just a new strain of “common cold” virus like HCoV-NL63 or HCoV-HKU1 which were discovered in 2004 and 2005 and didn’t cause an epidemic.

      • Even that is not the same as expecting deaths to be the same. We project deaths and we regularly are near that number, usually a few points low. Even if it’s bad flu, they would be under-projecting by some amount because even 100% would be 2 points higher than usual. I’ve never met or heard a person say people aren’t dying. So phrasing an argument that people think deaths wouldn’t rise is either deeply confused or mendacious.

        • > I’ve never met or heard a person say people aren’t dying.

          I don’t know if it qualifies as “people aren’t dying”, but there are people here who have argued over the last month that numbers are inflated because governments have a vested interest in keeping the death and case count at panic levels as long as possible and hospitals have financial incentive to get as many of their patients as possible labeled positive because they are going to be paid by the federal government for each one marked “COVID-19”; that there is not a single official datapoint indicating there is a real problem due to this virus; that people with weakened immune systems are dying of respiratory failure instead of from whatever other thing it would have been a few weeks/months down the line; that many deaths will be attributed to the virus just because people near death often get taken to the hospital, etc.

        • Carlos –

          Indeed.

          I don’t know how anyone who is following this in popular or social media could *not* have run across people making exactly those arguments. They’re ubiquitous.

          Not to make an argument from incredulity, but I really think that if you haven’t seen those arguments being made, you must have had limited exposure, and thus saying that no one could believe those things means you have been looking in only a very tight circle

    • Jonathan –

      > Do people actually believe something like that?

      As to what people really believe, I can’t say. But I see that kind of argument all over the blogosohere in right wing circles. Check out the comments sections at “skeptic” blogs.

      They are an outlier, I have no idea pervasive the pervasive is in the “real woeld.” But it is certainly out there. And it is alluded to when people break down the “average years lost” with a COVID 19 death.

  7. Excellent post. I’ve wondered for a while *how* genetics moved towards large-scale studies. (This is a simplification — there are still a lot of small, noisy studies, but I want to keep this comment brief.) I don’t know the answer to this, and it would be good to get insight from a real geneticist. Some possibilities:
    (1) A more widespread quantitative understanding of noise, statistics, etc. in the field;
    (2) A broader sense that the answers to the questions actually matter, if one wants to make predictions and not just write papers;
    (3) As noted in the post, really amazing technological advances in genotyping that make large-scale studies possible. (I’ve been reading / writing about this recently — it’s fascinating!)

  8. “Early on, in February, there were many interesting preprints [scientific papers that have not yet been peer-reviewed] around. Now you can read through 50 before you find something that’s actually solid and interesting. A lot of research resources are being wasted.”
    -German Coronavirus Expert Christian Drosten

    Despite the quote, the Germans themselves have made some progress on this. Here in the US, capitalism is once again the elephant in the room in terms of crisis management. If your private lab develops a treatment or vaccine that makes a profound difference, it will make you rich beyond your wildest dreams. So you start trying stuff, but all the stuff you are trying has already been tested and found to not work by your rivals. If you happen to find something, your paper has to walk a tight rope, overhyping results while trying to not give away enough that rivals can steal ideas.

  9. COVID-19 Host Genetics Initiative. An example of the international genetics collaboration, based on principles from the Human Genome Project, pioneered at the Broad Institute – shared data, tools, methods, results, analyses. Still early days, obviously. But lots or precedent. And lots of promise to understand the host genetic basis for susceptibility, severity, sex-bias, comorbidities, et al. If genetics is relevant, this Initiative has a great opportunity to make a useful contribution.

    • Cathedral or bazaar does not need to be ‘or’. During the hey-day of the Human Genome Project, a factory-cottage industry model worked exceedingly well, simultaneously enabling scale and innovation. (And back in the medieval day, bazaars (markets) surrounded cathedrals and cathedrals were often built near the market. – the life of a city relied on both.)

  10. Certainly, I think that finding ways for researchers to scale up projects and share efficiently is crucial. The development of open standards, and common resources, and resource hubs is crucial for this kind of thing. My outside understanding is that there are significant efforts going in this direction.

    One particular in this direction from the molecular science side is being made by MolSSI (Molecular Science Software Institute) and BioExcel.

    From the MolSSI site:
    “MolSSI is coordinating and has built the COVID-19 Molecular Structure and Therapeutics Hub (covid.molssi.org) to gather and disseminate the molecular sciences’ community efforts on the pandemic in one location. The main goals of this Hub are to provide a single source for researchers in the Computer Aided Drug Design (CADD), the Drug Discovery (DD), molecular modeling, and molecular simulation communities to not only share their efforts, but benefit from the advances of others looking to flex their scientific prowess and help tackle the ongoing pandemic. We and our partner, BioExcel, are amassing critical information to accelerate drug discovery for the molecular modeling and simulation community in support of COVID-19 research. This critical effort is coordinated with the large, grass-roots effort described in this open letter to the biomolecular simulation community.”

    Links:
    molssi: https://molssi.org/2020/04/17/molssis-response-to-the-covid-19-pandemic/
    bioexcel: https://bioexcel.eu/bioexcel-center-of-excellence-in-support-of-covid-19-research/

  11. I got interested in ‘self-describing data’ a while back, and a little Internet searching turned up that meteorologists have worked on the data-harmonization issue quite successfully. I assume this is one of the elements in the weather/storm forecasts that report both European and GFS model predictions. Geographers have OpenGIS, to mention another. I’m sure there is much to learn from these.

    • Thanks for this. Their Common Data Model is exactly the kind of thing to which I was referring. It would seem that some potential small-molecule pharmaceuticals for Covid-19 could be identified from purely observational data such as they are collecting. The possible therapeutic value of famotidine was noticed by less extensive and less formal observation, reportedly.

  12. Thanks for the links, Keith. Looks like an excellent initiative. Seems like this could be a source for identifying currently-in-use drugs as possibly of therapeutic value for Covid-19, as well as evaluation, as was done with chloroquine.

  13. I think part of the development of genetics depended on the early structure of the genetics community and the nature of the research material. I am reconstructing from stuff learned a while ago and guessing at the relation to the current community, but here goes. For genetics in model organisms (Drosophila, yeast) the early work was done when the (modern) science world was young. Generating the stocks used for research was quite difficult and comparing strains from different labs was very difficult. There was an initial investment in creating useful mutants for analysis by a small number of labs and this created a platform from which to build the research field. My guess is starting in this way created a degree of coordination that has persisted in the group as it has grown. Having a small number of groups with these stocks would naturally provide a source of centralized information about what was being done and a way of disseminating the information through the group as a whole and thus a way of coordinating efforts. I don’t think development of other areas of biology had comparable processes of branching from and retaining links back to a small number of sources and so did not have a chance to develop the habits of interaction seen in genetics.

  14. So as a statrter a statistical checking function somewhere where stuff can be submitted for review pre or post publish / draft – whatever either directly is as a link. Like a sort of agglomeration of sites like this. Then some sort of cataloguing or indexing.

    Its not that weird at one of my old we did just this for some projects and the function was called the Research Dept.

    On another note if projects/reviews were live at any stage we use Slack to track and update project progress. Something like that might contribute. Too.

Leave a Reply to Joshua Cancel reply

Your email address will not be published. Required fields are marked *