Skip to content
 

NPR’s gonna NPR (special coronavirus junk science edition)

1. The news!

Zad’s cat, pictured above, is not impressed by this bit of cargo-cult science that two people sent to me:

No vaccine or effective treatment has yet been found for people suffering from COVID-19. Under the circumstances, a physician in Kansas City wonders whether prayer might make a difference, and he has launched a scientific study to find out. . . . “It has to be a true supernatural intervention,” says Dr. Dhanunjaya Lakkireddy.

Lakkireddy is the principal investigator in a clinical trial involving 1000 patients . . . The four-month study, launched on May 1, will investigate “the role of remote intercessory multi-denominational prayer on clinical outcomes in COVID-19 patients” . . . Lakkireddy has assembled a steering committee of medical professionals to oversee the study.

That makes sense. It’s not like medical professionals would be busy with anything else right now.

The NPR story continues:

The investigators will assess how long the patients remain on ventilators, how many suffer from organ failure, how quickly they are released from intensive care and how many die. . . .

Lakkireddy says he can not explain how people praying remotely for someone they don’t know (or a group of people,) could actually make a difference in their health outcomes . . . Lakkireddy says he has no idea what he will find. “But it’s not like we’re putting anyone at risk,” he says. “A miracle could happen. There’s always hope, right?”

OK, NPR got their wish. It’s all about clicks, right? The attention economy. And they got my attention. Instead of writing, say, on recommendations for experimental design for coronavirus studies, I’m posting on this. So they’ve already done some damage by running this story. Sure, you could blame me for this—shouldn’t I take responsibility for what I post on this blog? Not really, I reply. I’m a creature of endorphins just like the rest of you. NPR pushed my buttons, and here I go.

Actually, I’ve been showing a lot of self-control this week. Just the other day someone pointed me to a horrible new paper by some dude I’ve written a lot about in the past, and I was like, no, I’m not touching that one at all!

2. Intercessory prayer

Intercessory prayer, though, that’s an old one. Deb Nolan and I wrote about it in the first edition of our book, way back in 2002:

We were reading the Web-based magazine Salon one day and saw an article reporting on a study of the effectiveness of prayer on the health outcomes of 990 patients at a critical-care unit. The article continues:

But does it do any good? Everybody’s got an opinion but nobody knows for sure, because the faith-health dichotomy has never received much in the way of serious scientific scrutiny.

Until now. A massive study published in the Oct. 25 issue of the Archives of Internal Medicine (a journal of the American Medical Association) showed that heart patients who had someone praying for them suffered fewer complications than other patients.

. . .

Dr. Harold G. Koenig, director of Duke University’s Center for the Study of Religion/Spirituality and Health, has spent his entire professional life looking at how spirituality affects a person’s physical well-being. This particular study is significant, he says, “because it’s published in an AMA journal, it has a huge sample, and it shows significant results.”

The prayed-for patients had better results, on average, than the control patients, and the difference was significant at the 4% level. On the other hand, three tests were performed on the data, and one might be skeptical about the findings, given that neither the patients in the study nor their doctors knew about the prayers being conducted on their behalf. . . .

Back then, we thought of studies of implausible phenomena reporting “p less than 0.05” from multiple hypothesis tests were just an amusing oddity. In those days before Gregory Francis, Uri Simonsohn, etc., we had no idea how prevalent they were in much of the published scientific literature.

We’ve discussed intercessory prayer on the blog before. As I wrote a few years ago,

What amazed me about this study—beyond the philosophically untenable (to me) idea that God is unable to interfere with the randomization but will go to the trouble of improving the health of the prayed-for people by some small amount, just enough to assure publication—was the effort the researchers put in to diminish any possible treatment effect.

It’s reasonable to think that prayer could help people in many ways, for example it is comforting to know that your friends and family care enough about your health to pray for it. But in this study they chose people to pray who had no connection to the people prayed for—and the latter group were not even told of the intervention. The experiment was explicitly designed to remove all but supernatural effects, somewhat in the manner that a magician elaborately demonstrates that there are no hidden wires, nothing hidden in the sleeves, etc. Similarly with that famous embodied cognition study: the elderly words were slipped into the study so unobtrusively as to almost remove any chance they could have an effect.

See here for further discussion of this point.

3. NPR

So how does this appear on the national news? Just as they run fake-science stories about football the week before the Super Bowl, and fake-science stories about basketball during March Madness and the NBA playoffs, now’s the time to run fake-science stories about the coronavirus now. And it’s a human-interest story.

I guess NPR justifies it by some mixture of (1) lots of their listeners are religious so this sort of story shows them respect, (2) even if it doesn’t work, little harm is done, and (3) hey, maybe it really will work and this is a big story! It’s hard for me to argue against #1 and #2—after all, there are lots of news items on sports, Hollywood, etc., and we can think of junk science as just another form of entertainment or human-interest story. #3 bothers me, though. To think, “Hey, this might work,” is like thinking that that lottery ticket you just bought might win the hundred million dollar jackpot. As we’ve discussed elsewhere in the context of Pascal’s wager, once you accept the possibility that intercessory prayer might “work” and cure people, you have to accept the possibility that it could have a negative effect on survival.

For example, maybe you and your friends all pray really hard for Grandma, and God’s listening, sees all your love and decides to do her a favor, put her out of her misery, and send her to heaven right away! Your prayer worked. But it looks like a negative outcome in your study.

4. God’s mighty thumb on the scale

And that doesn’t even get into the possibility that God is screwing around with the randomization.

I’m not joking here. Conditional on a belief on a directly-intervening God, one of the most striking things is the lack of direct evidence. No photos of the guy, no videos of a burning bush, no fun and games with the stars in the night sky. Dude’s so boring, he just operates through the laws of physics. Shroud of Turin? C’mon—that’s about as convincing as those photos of the Loch Ness monster. This is not to say that God doesn’t exist—I have no idea—but just that, if he does exist, and he does have the habit of making supernatural interventions, he seems pretty careful about covering his tracks and never doing it in a way that leaves unambiguous evidence of supernatural activity. There’s always some alternative explanation.

So, now to the prayer study. If it’s direct prayer—get all the cousins to call Grandma on Skype and everyone prays together—then it’s no problem, God can intervene, and there’s an alternative explanation involving her mood, hormones, etc. But intercessory prayer—that’s another story. Suppose God really does want to help these people, but he wants to continue covering his tracks and keeping all evidence ambiguous. How can he do this? One way is to manipulate the random assignment! Too much unbalance on pre-treatment predictors—that can be detected (an all-knowing God will be aware of modern econometrics, balance tests, etc.)—but God can induce imbalance on the potential outcomes, and that’s fine, it’s undetectable in principle. So I anticipate that this new study will be a failure, just like all the others (although they might be able to p-hack their way into a publication, more publicity, hundreds of citations, a Ted talk or two, a book contract, an endorsement from Columbia University’s very own Dr. Oz, and maybe some funding from NIH’s office of alternative medicine)—but maybe it will only look like a failure. Maybe it will be a stunning success, manipulated into looking like no effect through the workings of a self-effacing God.

I guess this is why they never put statisticians on NPR. We’re such party poopers.

P.S. For more on this topic, Paul Alper points us to this 2001 parody article by Leonard Leibovici and these online comments.

P.P.S. Am I being mean to mock? I don’t think so. If we want to take this study seriously, I guess we could think of it as a form of performance art. Generating random numbers and telling stories about them can be fun. Randomness can inspire creative thinking. Just look at Jackson Pollock. Or Philip K. Dick. Or Rorschach. Kinda too bad that medical professionals are wasting time on this, but, hey, they need to blow off steam too. They’re not on call 24 hours a day. If I’m allowed to goof off and blog like this, they should be allowed to goof off and play with random numbers.

Let me emphasize that I have no objection to people studying the empirical implications of religious models. My problem is not with the idea that God can intervene in people’s lives. Who am I to question other people’s religious experiences? I only know what I’ve seen; I can’t judge the visions of others. I recommend they study the effects of prayer directly, without the randomization and all the other trappings of cargo cult science.

It’s hard to even write about topics such as religion out of the concern that people will say you’re disrespecting them. All I can say is . . . I’m not disrespecting your religion. I’m disrespecting the idea of studying a supernatural phenomenon using a randomized clinical trial—but, hey, do it if you want. It’s not that much worse than studying embodied cognition, beauty and sex ratio, ESP, subliminal smiley faces, and all sorts of other things that might as well be supernatural given their noise-to-signal ratio.

In all seriousness, I think these discussions do have statistical value, in the same way that we can deepen our understanding of inference by thinking about problems such as Newcomb’s paradox, as it can force us to think more carefully about what do we mean by treatment effects, randomization, etc. If you work it through carefully, you can figure out exactly what assumptions of the conventional causal model don’t work in the God-can-screw-with-the-randomization design.

Also, I’m legitimately annoyed at this doctor, and his enablers in the science establishment and the news media, who on one hand claim this is serious science and could improve people’s lives, and on the other hand say, “it’s not like we’re putting anyone at risk.” You can’t have it both ways. For one thing, there’s the opportunity cost of what the hospital staff could be doing if they weren’t wasting their time doing paperwork on this silly experiment. But, besides that, once you do consider the possibility that this push-button prayer really does something, then of course you could be putting people at risk. How can you be so sure the effects would be positive? It’s all just such a mess.

113 Comments

  1. Anoneuoid says:

    No vaccine or effective treatment has yet been found for people suffering from COVID-19. Under the circumstances, a physician in Kansas City wonders whether prayer might make a difference, and he has launched a scientific study to find out. . . . “It has to be a true supernatural intervention,” says Dr. Dhanunjaya Lakkireddy.

    It’s cheap and safe so even an unlikely minor benefit would make it worthwhile. But I suspect they just hope to hit the p < 0.05 jackpot. Maybe they can even get away with sampling to a foregone conclusion like the NIH remdesiver trial.

    Also, HBOT and IV vitamin C both appear to be safe and effective treatments (not sure how cheap HBOT is, but vitamin c is very cheap) where immediate improvement is seen after administration. Unfortunately we need to get this info from interviews with the doctors in podcasts and youtube videos for some reason:

    https://www.hbotnews.org/the-role-of-hyperbaric-oxygen-in-the-treatment-of-covid-19/
    http://www.drwlc.com/blog/2020/04/16/video-conference-with-dr-zy-peng-of-the-worlds-first-high-dose-ivc-trial/

    My ideal solution would be give everyone pulse oximeters and access to hyperbaric chambers and HPLC to get their serum ascorbate levels measured.

    If you feel sick and your psO2 starts dropping, you go to local hyperbaric chamber for an 1-2 hrs per day where they check your vitamin c levels, and if deficient, give you it at whatever route of administration and dosing schedule is required to correct the deficiency.

  2. Paul Hayes says:

    It’s not like medical professionals would be busy with anything else right now.

    Medical research is supposed to be constrained by ethical principles too.

  3. Daniel Rogoff says:

    “God can mess with the randomization” is a brilliant rebuttal that I wish I had thought of myself.

    • billo says:

      My favorite “God messes with randomization” study is the AWARE study — a prospective study of near death experiences in cardiac arrest. This study over multiple institutions looked at about 2000 cardiac events with 140 survivors who were able to be interviewed. The researchers placed pictures on the top of shelves in the emergency rooms that could be seen if there were so-called “out of body” experiences where the disembodied patient looks down from above at the event. Ideally, this disembodied person would be able to see the picture on top of the shelf, but would not be able to see it looking up from the table. However, even though the researchers put pictures on over 1000 shelves, only 22% of the events occurred in rooms with a labeled shelf, and neither of the 2 out of body experiences occurred in these rooms.

      Swing and a miss.

      See: Parnia S, Spearpoint K, deVos G, et al. AWARE – Awareness during Resuscitation – A prospective study. Resuscitation 2014 85(12):1799-1805.

  4. Terry says:

    The correspondent, Tom Gjelten, is NPR’s religious correspondent, so pretty much everything he covers is imaginary. This story is probably no loopier than many of his other stories.

    https://en.wikipedia.org/wiki/Tom_Gjelten

    But, just in case this remote prayer thing actually works, I hope they organized the study so different gods were prayed to for different patients, so we can have scientific proof on which gods are the most powerful. (The remote prayers are being made to many different gods.) Let’s keep an open mind here. This study may be bogus, but if its not, you have to admit it will be pretty darn important.

    • Wade says:

      terry says:
      “But, just in case this remote prayer thing actually works…”

      yeah, Think BIG.

      if we can get omnipotent supernatural beings to do favors for us — don’t waste that awesome resource on trivial stuff like flu viruses.

    • jrkrideau says:

      Does the design account for “interference” across prayer groups? If i have a crew praying hard to Osiris for me are prayers to another god going to scramble tho prayer waves?

      And what about informed consent?

      • Nick Adams says:

        From God? We know s/he’s informed. The consent part is implicit (no smiting).

        • The patients! Suppose you’re a wiccan, and you later find out that your doctors had arranged a whole busload of people to pray to a Christian god for your health. Wouldn’t you be pretty damn angry? particularly if it turned out that you’d had a really hard time recovering.

          • Andrew says:

            Daniel:

            This reminds me of this story a bunch of years ago when it came out that the Mormon church was organizing people to pray to retroactively convert a bunch of dead people to Mormonism. There was a big outcry, and some people were outraged that someone was trying to convert their ancestors. I could never understand the fuss. If you’re not a Mormon yourself, then presumably you don’t believe that their conversion efforts are working anyway, so why should you care? Or maybe it makes sense. For example, suppose someone in another city (Columbus, Ohio, maybe?) made a voodoo doll of one of my friends and stuck pins in it. That would annoy me! It’s not that I think voodoo “works”; still, I don’t like the idea of someone trying to hurt my friend, even misguidedly.

            • Terry says:

              A wife calls her husband and asks “do you feel a sharp pain like someone is sticking a pin in a voodoo doll of you?” The husband says “No.” There is a pause then the wife says “How about now?”

            • Divalent says:

              IIRC, the story was the Mormons were having young people stand in for unmarried dead people, and “marrying” them to other dead people in an actual ceremony in the church.

            • Roy says:

              The real controversy came when they retroactively baptized Holocaust victims. The victims are too dead to care, but given that real-life Jews have undergone forced conversion, the outrage is understandable. Sure, it’s only a symbolic conversion, but symbols matter.

          • jim says:

            But what if the patient miraculously recovered from death’s doorstep? Would s/he be angry that the Christian God saved h/him? :) Maybe instead h/she would become an evangelical Christian and spread the word!

          • Anoneuoid says:

            Trump gave me an idea the other day when he was talking about getting UV light into the body.

            It should be possible to create a chimeric protein consisting of luciferase enzyme linked to something that binds to the virus like recombinant hACE2 or an antibody. Upon binding to the virus a conformational change activates the luciferase which then emits a short burst of UV light and damages whatever is bound to it (and probably itself too). AFAIK there are no UV luciferases but that doesn’t seem like something impossible to create. Start with eg cypridina luciferase (peak ~450 nm) and try to move more towards UV.

            Anyway, its interesting to think of how religious people would respond to having something named after lucifer injected into them. And actually if you look it up someone has already made a SARS-HIV-luciferase pseudovirus, which must sound like the scariest thing in existence to a lot of people at this point.

            • Nick Adams says:

              = internal sunburn

              • Anoneuoid says:

                Obviously theres a tradeoff there. Maybe the emission spectrum can be tuned just right to be mostly purple and give it with the right antioxident to clean up any collateral damage quickly. Maybe also add a “shielding” around the emission site so the light only escapes in the direction of the virus binding site.

            • jim says:

              Why not just eat some plutonium

              • Anoneuoid says:

                Because plutonium emits whenever it wants, not only when close to [bad thing]. It also accumulates in your bone marrow.

                This thing should self destruct after a few emissions and stay in the blood for the most part.

            • So my understanding is that there is a lot of research on using these kind of fancy techniques to deliver drugs. More likely it’s a drug that encapsulates a toxic chemical, when it binds to the virus, it releases the chemical, sometimes this release might be mediated by light emission, or by other biochemical events that are triggered by binding. But this whole class of drug delivery mechanisms is highly speculative and an active research field in university level pharmaceutical research.

              • Anoneuoid says:

                Well Ive never heard of the light emitting part, but I dont see any problem in principle. Getting something to “only” emit light upon binding sounds way simpler than that plus a trigger to release a drug though.

              • The most common thing I’ve heard is a light responsive drug, one where the presence of some particular wavelength of light causes a chemical to be released or activated. So you could imagine a light responsive drug throughout the body, which is inert until it absorbs a particular kind of photon, and then a second drug which binds to the targeted cells and releases light, thereby activating the drug delivery locally in the neighbor of that cell.

              • Twain says:

                Anoneuoid,

                This type of therapy is called “Photodynamic Therapy”: https://en.wikipedia.org/wiki/Photodynamic_therapy#Photochemical_processes

                It’s been developing for ~2 decades or so now. The major issue impeding progress is optical loss (mainly in power or elongated point-spread-function) from scattering, absorbance, etc. — reaching beneath superficial tissue has proven very difficult in large animal models and humans.

              • Anoneuoid says:

                This is not photodynamic therapy. Here the drug itself releases the light when it comes into contact with the virus. There isn’t any outside stimulus.

              • Twain says:

                Oh duh, I missed your initial comment.

                It’s an interesting idea. I wonder how much output you could generate from a change in conformation alone. If it only threw a few photons, then I’m not sure how well it would work. Then again, I’m not a chemist and what chemistry I do know well isn’t related to this idea much…so I’ll stop here!

              • Anoneuoid says:

                Look into how a luciferase works. This is the blue-purple one: https://en.m.wikipedia.org/wiki/Vargulin

                It’s a type of enzyme that catalyzes the oxidation of a substrate molecule (the luciferin). That reaction then releases light. Thats what makes fireflies glow.

            • Kevin says:

              I think you’ve fundamentally misunderstood the difficulties in treating an active viral infection. You brushed over the hard part: somehow interacting with enough of the virus before it kills too many cells, and over complicated the easy part: neutralizing the virus once you’ve found it. Once an infection is active, the virus is distributed throughout the body in extracellular and intracellular spaces. You need to get your treatment into these same spaces and have enough of it available and for long enough time to neutralize basically all of the virus because it self replicates. That is a very difficult challenge, especially for a virus that seems to infect many different tissues. For instance, how does something that is delivered in the blood stream get to the surface cells of the lung that the virus infects? You are obviously not coughing up blood on every breath, so clearly there is a barrier between the blood stream and your lung cells. You need to figure out how to make a molecule that can get into the infected tissue spaces from the blood. Your immune cells are very good at doing that, but any arbitrary protein is not. Plus, your engineered protein needs to somehow avoid being itself cleared by the immune system or filtered by the liver or kidneys. It is also very difficult to make a molecule that binds a specific virus and nothing else, so you suggested using an antibody. Well, if you have that antibody, you dont need anything else (assuming you were able to distribute it as above, and that the viruses isnt in so many cells that the tissues are significantly damaged). Any antibody that someone would isolate to develop a treatment with would most likely be in the form of a neutralizing antibody that binds to the virus and prevents it from entering a cell. That alone is “enough”, because the virus is only dangerous once it enters a cell, creating more virus and destroying that cell. Your body can already clear antibody bound viruses without needing to “kill” them, hence the term neutralizing antibody and not “toxic antibody”. There are also a number of fundamental problems with the specific idea of using a UV emitting luciferase to locally kill viruses in vivo (which, again, is totally unnecessary) even aside from the dosing healthy tissue aspect, so its difficult to know where to begin a productive discussion. Well, first, where does the ~2x extra energy needed for producing UV light instead of blue light come from? Its ironic that you used the word impossible for the part of your idea that could actually violate a law of nature. I don’t intend to be critical of you, just the actual feasibility of your idea. I do think that its a very interesting idea to learn from because it involves a number of different fields of science: virology, anatomy, pathology, pharmacodynamics, biochemistry, quantum mechanics, immunology, protein engineering. A feasibility study of the idea would probably make an interesting qualifying exam for a biomedical engineering doctoral student.

              • Anoneuoid says:

                I think you’ve fundamentally misunderstood the difficulties in treating an active viral infection.

                Sorry, it was difficult to follow without paragraphs. But from what I read of the issues you take with the idea all the same difficulties will be present for a treatment like this: https://clinicaltrials.gov/ct2/show/NCT04287686

                Or this: https://www.jci.org/articles/view/138003

                Those treatments are going to stay in the blood just as much as the one with luciferase attached… So it was difficult for me to continue to put forth the effort to read the rest of your post.

              • Anoneuoid says:

                where does the ~2x extra energy needed for producing UV light instead of blue light come from? Its ironic that you used the word impossible for the part of your idea that could actually violate a law of nature.

                I noticed this. Where does the extra energy to get a blue (450 nm peak) cypridina luciferase photon instead of a red (625 nm) firefly luciferase photon come from? Photon energy is E = hv/wavelength. so E_blue/E_red = 1.4x. To get to UV (starts at 400 nm) from blue (450 nm) you need only a 1.125 increase.

                Not sure where your 2x came from, but I don’t see why there would need to be a law of nature violated.

  5. I often wonder why in the heck my Dad continued in the comparative religion field. He was an agnostic. His father was an atheist. I have a hypothesis for why Dad did. But to fair, I’ve come across some highly insightful comparative religionists too. I actually think Rawlsian political philosophy was rooted in Protestantism. I guess what I’m saying that social sciences and philosophy are normative, of which dimensions are rooted in religions. That has been trend as a function of cross-disciplinary emphasis too.

    But I am not so judgmental about what others believe as long as they are decent to others and don’t foist their beliefs unduly.

  6. Joshua says:

    Andrew –

    I really like this post.

    > Lakkireddy says he can not explain how people praying remotely for someone they don’t know (or a group of people,) could actually make a difference in their health outcomes . . .

    My own belief is that if people are looking for associations without being able to propose a plausible mechanism of causation, they’re looking for trouble.

    That said, I’m not sure why NPR gets a finger pointed at them. If someone is going to do this research, is NPR just supposed to ignore it? Is there some guiding principle by which they should choose what stories to report on? Seems to me that their function is to report on what people find interesting. Yeah, so “clicks” might be a motivation. What motivation should be the replacement – who gets to decide?

    > So they’ve already done some damage by running this story. Sure, you could blame me for this—shouldn’t I take responsibility for what I post on this blog?

    Well, then I am at fault for reading your post – and only doubled down by commenting on it!

    > beyond the philosophically untenable (to me) idea that God is unable to interfere with the randomization but will go to the trouble of improving the health of the prayed-for people by some small amount, just enough to assure publication

    Love it.

    • Andrew says:

      Joshua:

      You write, “If someone is going to do this research, is NPR just supposed to ignore it?”

      Sure, they can ignore it! There are a million scientific papers published every year. NPR can’t cover all of them, or even one-thousandth of them.

      • Joshua says:

        Andrew –

        > Sure, they can ignore it!

        Sure. But what criteria (or criteria matrix) should they use for deciding what to cover? They likely decide based on their assessment of audience interest. Not sure what other basis they should use. Given the criticisms of peer review, following the advice of “experts” would be inherently problematic.

        Seriously, what criteria should they use other than your subjective judgment?

        • Andrew says:

          Joshua:

          The criterion they should use is not my subjective judgment; it’s the editor’s subjective judgment that’s relevant. If there are 1,000,000 papers published every year, and NPR reports on 1,000 of them, then the editor is using his or her judgment to not report on the remaining 9,999,000. They can report on whatever they want. And I can share my impression, which is that NPR reports on a lot of junk science.

          • Joshua says:

            Andrew –

            But maybe the editor thinks she should report on what the audience finds intersting! Or some combination of what the audience finds intersting and what the editor thinks is quality science. I presume (maybe I’m wrong) that you think they *sometimes* report on quality science. Does it have to be 100% what you think is sufficient quality? Is there some cutoff?

            Here’s what’s behind what I’m getting at. I think there’s relatively little evidence that media choice in what to report drives public opinion as opposed to media choice on what to report on reflects public opinion.

            But criticism if media making bad choices is ubiquitous. And people on both sides of issues blame the media for the prevalence of views on the other side. The media is a convenient target, but in reality, people get the media they consume, for the most part.

            People choose their media outlets based on compatibility with their worldview. If I don’t like a particular outlet’s choices, I can just pick a different media outlet.

            Seems to me like you’re setting up unfalsiable critique.

          • Joshua says:

            Andrew –

            > And I can share my impression, which is that NPR reports on a lot of junk science.

            By no means am I suggesting that you not share your impression. Quite to the contrary, I enjoyed your post very much.

            But I think to blame NPR for making bad choices, it helps to outline an alternative set of inclusion/exclusion criteria. And so I’m curious as to what your would be.

    • Curious says:

      Joshua:

      “My own belief is that if people are looking for associations without being able to propose a plausible mechanism of causation, they’re looking for trouble.”

      +1

    • “Is there some guiding principle by which they should choose what stories to report on?”

      NPR and People Magazine publish different stories. This is perhaps because they each use a random number generator to decide what to publish, and this happens by chance to consistently give one type of story for People and another for NPR, but more likely it is because they have different “guiding principles” on what their content should be. Even without explicitly articulating these principles, most people would agree that NPR’s intersect with concepts of scientific validity. I would be surprised and upset, for example, if NPR started broadcasting horoscopes. It is therefore perfectly valid to call out situations in which we judge NPR to be doing a poor job.

      Quickly looking at the article: one could imagine a better article that dove more into what people who really think about the scientific method, or who really think about theology, have to say about stuff like this.

      • Joshua says:

        Raghuveer –

        I know it not seem like it, but I’m really not trying to be a gadfly here…but…

        > It is therefore perfectly valid to call out situations in which we judge NPR to be doing a poor job.

        I am not suggesting that it’s invalid to criticize the study (Andrew’s critique seems spot on to me), nor am I suggesting that you shouldn’t say that you wish they’d more frequently choose topics to report on that you find more interesting. All fair game.

        But I’m asking what it means to say that you judge NPR to be doing a “poor job”? Perhaps their one of their standards is to choose topics that their audience finds interesting, and this story fits that criterion. I don’t know if it does or not. But I’d imagine that might well be one of their criteria.

        I don’t really think it’s NPR’s job to determine which studies are of the highest scientific value, and anyway, I think that such an assessment would always be subjective. And if they only chose topics that some select group of people determined to be of the highest scientific value, I’d imagine it’s quite possible that a large segment of their audience would find their selection boring.

        What set of criteria would you suggest that they use to determine which studies to report on?

        I’m interested in this as a larger phenomenon. A lot of people criticize “the media” for covering sensationalist and shallow trash. Yet, (1) I question what the actual effect is of what they choose to report on and (2) my guess is that there were a clear signal from their audience that their choices were “poor,” then they would start using a different sent of inclusion/exclusion criteria.

        I also read a lot of people saying that it isn’t the media’s job to decide what should and shouldn’t be reported on. A lot of people think that the media adopting such an attitude is “elitist” and is part of the reason that “trust in the media” has diminished. I don’t happen to agree with those views, but I do think that there is a legitimate question as to how to handle the influence of ‘subjectivity” in evaluating what he media does and doesn’t choose to report. I think it’s easy to trash the media and to say they’re doing a poor job. What I think is more difficult is to come up with a practical solution.

    • Anonymous says:

      “My own belief is that if people are looking for associations without being able to propose a plausible mechanism of causation, they’re looking for trouble.”

      If science was restricted reporting observations supported by a “plausible” mechanism, we’d still be in the freakin’ stone age.

      – There was no known or “plausible” mechanism inheritance and indeed that was a powerful argument against evolution for over a century.
      – The lack of a plausible mechanism for moving continents was long an argument against continental motion

      Many mountain belts are characterized by thin, laterally extensive sheets of rock that have been structurally stacked on one another (aka “thrust sheets”); geophysicists argued for decades that it was mechanically impossible to move such thin sheets because the force needed to move the sheet exceeded the strength of the rock. In the 1950s the riddle was solved by the recognition that fluid pressure in the fault zone dramatically reduced frictional resistance on the faults and thus the force required to move the sheets, proving that the empirical observation had been right all along, even though there was no “plausible” mechanism.

      All science starts with empirical observation.

      • Joshua says:

        Anonymous –

        > If science was restricted reporting observations supported by a “plausible” mechanism, we’d still be in the freakin’ stone age.

        and

        > – There was no known or “plausible” mechanism inheritance and indeed that was a powerful argument against evolution for over a century.
        – The lack of a plausible mechanism for moving continents was long an argument against continental motion

        Sure. But compare your examples (which are often cited – usually people add peptic ulcer disease to the list also) to the myriad examples of people finding associations and attributing a causation without any explanation of a plausible mechanism, to only find later that in fact that the association was spurious. The fact that people often cite the same examples over and over, IMO, is indicative of the “exception proving the rule” process.

        I’m not saying don’t ever do it. It’s unavoidable that we do it. Looking for associations is how we make sense of the world.

        I’m saying that when you do it you’re looking for trouble. Recognizing that, with acknowledgement that you are the easiest person for you to fool, then if you find such an association you can follow up. Trouble isn’t necessarily a bad think.

        We are pattern finding machines. Often we find patterns that don’t exist. I’m not remotely suggesting that you not start with empirical observation.

  7. Jay Quimby says:

    Do bad hypotheses equate to junk science? If the null was that prayer does *not* help recovery, would you view the study as more legitimate? (Ideally, the experimental design would not change at all.)

    What if you consider the possible benefits of showing prayer has no effect? If it leads a religious person to believe prayer will be less effective, maybe they would try other pathways to helping (donating time or money) or adopt healthy behaviors (physical distancing).

    Remote prayer will not help those suffering from the coronavirus. But that doesn’t make this study junk science.

    • Anonymous says:

      Correction: “null” should be “H1”

    • Joshua says:

      Anonymous –

      > What if you consider the possible benefits of showing prayer has no effect? If it leads a religious person to believe prayer will be less effective, maybe they would try other pathways to helping (donating time or money) or adopt healthy behaviors (physical distancing).

      Problem is, it may have no effect in this experiment, but that doesn’t mean it could never have any effect. So those who think they’ve experienced an effect from prayer previously wouldn’t likely change their view.

      Seems to me it’s inherently more interesting to show that it does have an effect in a particular experimental condition.

      But then again, I think unless you can describe a plausible mechanism of causality, or at least show things like that more prayer over time has a greater effect (or other changes in the effect in association with changes in variables), then a study like this one does little other than give people ways to confirm their biases.

  8. Torquemada in Training says:

    I gather the principals anticipate a result that is so modest that its presence can only be revealed by sophisticated statistical analysis. Suppose there is a God, and suppose He responds to prayers, and suppose He heals just enough of a cohort of patients for us mortals to conclude that He helped at a level p lt 0.05, but barely. In other words He didn’t save them all, or save half, or save just the worthy. No, he saved a handful so that His presence, while real – that’s the null hypothesis – is only faintly distinguishable from natural.

    Now you’ve got two choices. The evidence is not really strong enough to support supernatural effects, or, second, God is half-assing it.

  9. Defender says:

    I have no problem with the hypothesis so long as the science is right. I am all for critiquing the method. For that reason, I take the point that a God interested in interfering with the randomization can do so. And result would be valid only on the assumption that God didn’t mess with the study. But maybe it IS an important question for some people. Whether resources should be invested in studies like this is a different story, but I will support efforts to ask questions scientifically regardless of my own very strong priors. If prayers work, I’ll have to rethink my worldview. And if it’s a tightly estimated zero effect, some other people might change their beliefs. But yes, informed consent from patients might not be a bad idea.

    • Curious says:

      Randomization is not magical. It does not confer causality on that which is not mechanistically causal. THIS is one of the fundamental problems in the replication crisis — a misplaced belief in a misapplied method.

      • Defender says:

        But then it would raise the question: if not causation then what? Just because I don’t know a mechanism doesn’t mean there isn’t one. No?

        • Curious says:

          Is that a serious question?

        • Curious says:

          Ok, I’ll take you question at face value:

          1. It could be random chance, in the same way that it’s possible to get 8 heads in 10 flips of a fair coin by chance. It could be a tail event. If we concluded that the binary heads/tails had a p/q of 0.8/0.2 based on that data, our inference would be incorrect relative to the reality of 0.5/0.5.

          2. It could be correlative such that the variable (x) with which the correlation was derived is not directly causal with the outcome (y). This would mean that it would be a waste of time, effort, and money to try to manipulate x to effect y as it would be akin to manipulating the weight of people in an effort to effect their height.

          3. If (2.) is true, it could be that a third variable z causes both x and y, say genetic material inherited from the combination of egg and sperm with which we were created that creates some likely interval to which we will grow, etc.

          • Andrew says:

            Curious:

            Randomization eliminates your explanations 2 and 3. But explanation 1 is a big one. Other explanations are selection bias in how the data are coded and analyzed, who drops out of the study, etc.

            • Curious says:

              Yes. Thank you. For an experiment, of course you are right. I drifted into an example for observational data.

              • Defender says:

                I replied late, but yes it was a serious question. I am not here to troll. And I was assuming an experiment where a treatment is randomized. And let’s also put aside Andrew’s concerns about differential attrition (though the point about data coding I don’t understand). Then the explanation is either (a) the treatment causes the effect, or (b) what Curious said in point 1– the treatment, though randomized, was somehow correlated to an unobserved factor. (b) is what we learn to live with.

                Thanks for the responses.

  10. Tom M says:

    What sort of religious person refuses to pray for the control group? As an aside, I first read the phrase in the post as “the shroud of Turing”. It was an arresting image.

  11. Joshua says:

    Some here might find this interesting – too technical for me to evaluate:

    > Although estimates vary, it is currently believed that herd immunity to SARS-CoV-2 requires 60-70% of the population to be immune. Here we show that variation in susceptibility or exposure to infection can reduce these estimates.

    https://www.medrxiv.org/content/10.1101/2020.04.27.20081893v1

    I’d love to read reactions (provided they’re not too technical, or at least have a plain language summation).

    • Mendel says:

      What was your reaction?

      The thesis isn’t new: assume some subpopulation who is more susceptible, get them out if the way, and the spread collapses. Didn’t we talk about “epidemiologists don’t understand variance” here?
      There isn’t any evidence who might be more susceptible medically; they have a second prong to their argument where they talk about people who are better connected, but don’t actually model any connections, or reference any sociological data.
      Basically, “if the real world works like our fantasy land, this epidemic could be as good as over. Trust us, we used maths.”

      • Joshua says:

        Mendel –

        Thanks.

        > What was your reaction?

        First some background. The link was posted at a climate “skeptic” blog – in the comment thread of a post put by a rather well known “skeptic” – who regularly does very sophisticated mathematical analyses of climate science, and I guess just by pure coincidence always finds that the climate scientists have overestimated warming (funny how that works, eh?). So, when the commenter posted the link, the sophisticated “skeptical” analyst posted a comment that the article “made sense” to him. Now whenever I respond to critiquing the logic of one of his posts, he just ignores my comments – I assume because they aren’t mathematically sophisticated (for example, when he posted on the data on the cruise ship passengers I commented that it’s dubious to try to extrapolate from sampling of an extreme outlier and he just ignored my comments to that effect).

        At any rate…for my reaction…

        From the article:

        > Individuals that are frailer, and therefore more susceptible or more exposed, have higher probabilities of being infected, depleting the susceptible subpopulation of those who are at higher risk of infection, and thus intensifying the deceleration in occurrence of new cases. Eventually, susceptible numbers become low enough to prevent epidemic growth or, in other words, herd immunity is attained.

        Intuitively, given the total number of non-frail vs. frail people there are out there, and the relatively low levels of mortality even among the frail (not relative to the non-frail, but relative to the # who don’t die), it’s hard for me to believe that the impact on levels needed reach herd immunity would be felt until almost the typically estimated herd immunity levels were reached. I don’t know the correct fatality rate numbers but say that 8% of people infected over 65 die (assuming that everyone over 65 fits the descriptor of “frail,” which seems a bit dubious), and all people over 65 get infected, and 16% of the US population is over 65…. Is that 1.3% (seems like an upward bound?) of the US population really going to significantly shift the herd immunity numbers? But I can’t evaluate their math, and I guess I should assume that they’re right and I’m missing something.

        > There isn’t any evidence who might be more susceptible medically;

        Yeah. That would be my first question.

        > they have a second prong to their argument where they talk about people who are better connected, but don’t actually model any connections, or reference any sociological data.

        Seems that the “more connected” aspect should be proven. There are lot of elderly or otherwise vulnerable people, in certain demographic sectors, that don’t seem particularly well “connected” to me. And what good is a model that doesn’t allow for the impact of interventions which are actually taking place?

        Armchair epidemiology, that doesn’t rely on the literature…seems to be a lot of that going around.

        > Basically, “if the real world works like our fantasy land, this epidemic could be as good as over. Trust us, we used maths.”

        There’s a lot of that going around too. That said:

        > It would therefore be imperative to conduct longitudinal serological studies in representative samples of the population, as control measures are relaxed.

        I always like to see calls for longitudinal data. Not enough of that going around, IMO.

        • Joshua says:

          > Armchair epidemiology,

          I take that back. I’m not used to papers that don’t reference the literature extensively – but maybe that’s not a reasonable requirement in this context? At any rate, from just looking at their affiliations it looks like they’re well-qualified….

        • mendel says:

          Thank you!
          Mortality doesn’t come into it, and neither does age. The “elimiation” from the susceptioble pool is by immunity mostly. (Btw, this is a SEIR model with just different buckets of S.)
          We have data that all age groups (including children) are pretty much equally susceptible.
          And there’s no data I’m aware of that suggests “frailty” (i.e. likelyhood of a worse outcome) is associated with susceptibility (likelihood to get infected), unless maybe we’re talking about doctoirs and nurses.

          The “social connectivity” thing is exemplified by a care home: the workers there come into close contact with many older people, the residence not so much. So once the care workers are no longer susceptible, the older people become safer. That’s what this model seems to say. The sensible approach is to reduce the connectivity, i.e. to segregate the care home into separate groups of residents and caregivers that don’t socialize, and reduce the transmission rate that the workers have by provindg them with PPE.

          Social distancing measures are moving people from the “high chance to get it” buckets into “low chance to get it” buckets, and we see a “herd immunity” effect under lockdown conditions when the number of cases goes down.
          If you’re infecting one in 20 people you come in close contact with (actual data), then only having contact with 10 people has the same effect as contacting 40 people, 75% of whom are immune.

          Social distancing is proven to work.
          “Wait for the highly connected folks to become immune” isn’t because the study gives us no clue whether we live in a world where it would work.

          If we have 10% susceptible people, how many others are they going to infect? If they’re infecting 9 people not in this bucket (high number because they are socially well-connected), then getting all of the 10%ers immune means we’re probably infecting everyone else along the way. If we live in that kind of world, this strategy can’t work.
          I don’t understand the model in the paper well enough to check whether they’re considering that kind of relationship, or whether they’re just talking in terms of susceptibility as a one-way-street, but I thought they did the latter. And that’s a big flaw then if you’re using the connectivity argument.
          And there’s just no data for the other argument, that some people are biologically more easily infected except for maybe some small subgroups with certain medical conditions.

          • Joshua says:

            Mendel –

            Thanks.

            > Mortality doesn’t come into it, and neither does age. The “elimiation” from the susceptioble pool is by immunity mostly.

            Thanks for correcting my confusion. It’s ironic because the person who posted the link (I think he considers himself as a kind of expert) had a follow up comment where he largely talked about mortality (how few children are dying) and I asked him why he was talking about mortality.

            He claims there’s evidence regarding associations of age to infectiousness, claiming with children both less likelihood of being infected and less likelihood of acting as an agent to infect others. He linked to his website where he posted on the evidence. I didn’t chase any of it down because of the high probability that it’s one of those climate “skeptic” rabbit holes.

            https://healthy-skeptic.com/2020/05/03/children-and-coronavirus/

          • Twain says:

            > And there’s just no data for the other argument, that some people are biologically more easily infected except for maybe some small subgroups with certain medical conditions.

            A fundamental tenet of immunobiology is that some people are more or less susceptible to certain infections — namely, innate immunity is very heterogenous among humans (for reasons we still don’t understand well). Some people, by luck of environment, genetics, etc., may have just the right immune system to quickly quell infection by SARS-CoV-2; others may not.

            Here is recent review explaining this topic and recent data: https://www.nature.com/articles/nri.2016.125.pdf. Per the paper ” Young children and elderly individuals are more susceptible to infections than other age groups” — see “Immune system variation with age.” sub-section.

            Something to consider.

            • Joshua L Brooks says:

              When the pandemic was just a little news item, I remember hearing that children were likely siperspreaders of coronaviruses.

              Hmmm, Mrs. Google tells me this: https://www.google.com/amp/s/www.cbc.ca/amp/1.5552099

            • Twain says:

              Joshua Brooks,

              If true this report would not surprise me — young children/teenagers almost always have a stronger innate immunity that adults. So it would make sense that if SARS-CoV-2 infects them, the innate immune response quickly eliminates said infection within a few hours to a day or so.

              Of course, the opposite it also possible — as is the case with influenza. But we don’t know and can’t know because we closed all schools in the US.

              Here is the report for those inferested: http://www.ncirs.org.au/covid-19-in-schools

              • Joshua says:

                Twain –

                Maybe you missed this? (I left a question for you – no obligation, but 2nd time I wrote a fairly long response that became a dead end).

                https://statmodeling.stat.columbia.edu/2020/05/03/nprs-gonna-npr-special-coronavirus-junk-science-edition/#comment-1327894

              • Twain says:

                Joshua,

                Apologies! I missed your post.

                > Except there are still high numbers after the peak […] Flattening the curve is the main target – but is it the only target?

                Nope! As I mentioned, it is possible that even if peak infections already occurred, then the shutdowns drove it to low infectious faster than it would have otherwise. Hard to know yay/nay at this point given aforementioned issues with deconvolving effects of shutdowns from other factors influencing infection.

                > Behaviors were changed due to the “shutdown” even if supermarkets weren’t closed and everyone wasn’t wearing masks.

                Yes and no. Some people are less risk-adverse than others. Personally, I saw wide variance — some people wore masks early and were careful, but many did not do so. I remember walking through my local supermarket during the first three weeks of the shutdown in my location and wondering, “Huh, not many masks and people really aren’t avoiding each other. Humans are interesting.”

                > Do you think that the shutdowns, even if they came late, didn’t prevent many more infections from occurring, at least ion some areas, over the short term?

                I’m not sure.

                IMO, the answer may be both “yes” and “no”. In some areas, the shutdowns likely did much good; in others, they may have not. There are many factors influencing their efficacy (enforcement, societal norms, etc.), that vary from location to location, person to person, etc. And without decent data on seroprevalence, it’s hard to make any concrete conclusions.

              • Joshua says:

                Twain –

                >Yes and no. Some people are less risk-adverse than others. Personally, I saw wide variance — some people wore masks early and were careful, but many did not do so. I remember walking through my local supermarket during the first three weeks of the shutdown in my location and wondering, “Huh, not many masks and people really aren’t avoiding each other. Humans are interesting.”

                I had similar experiences. But my behaviors changed over time, as did those of mostly everyone I observed. The “lockdowns” added a level of seriousness that had a gradual impact that couldn’t be measured by a global survey of whether everyone was wearing a mask the day after it was put into place.

                > I’m not sure.

                IMO, the answer may be both “yes” and “no”. In some areas, the shutdowns likely did much good; in others, they may have not. There are many factors influencing their efficacy (enforcement, societal norms, etc.), that vary from location to location, person to person, etc. And without decent data on seroprevalence, it’s hard to make any concrete conclusions.

                If they did much good in some areas (in terms of #’s of infections), even if they didn’t do much good in others – the net result in terms of health impact is “much good.” Then the next question is how is that weight against the economics. Did they do much harm in the areas where they did much good, and how do you weigh the much harm they did there versus the total much good from everywhere. But then you have to also tease out the exact amount of harm that resulted from the lockdowns themselves as opposed to many of the similar economic effects that didn’t arise from the lockdowns per se but the general reaction to pandemic spread of a contagious and dangerous virus. And of course, as you know, I add extra weight in the “good” category for every single front line healthcare worker who didn’t get infected or wasn’t mentally/psychologically/emotionally overwhelmed by being stretched all out for an even more extended period of time. And I give extra weight to every grocery worker who might have been spared infection because the lockdowns increased, even of only marginally at first, th e# of people wearing masks. Those are the people who enable me to eat even as I reduce my risk.

              • Twain says:

                Joshua,

                > I had similar experiences. But my behaviors changed over time, as did those of mostly everyone I observed. The “lockdowns” added a level of seriousness that had a gradual impact that couldn’t be measured by a global survey of whether everyone was wearing a mask the day after it was put into place.

                Agreed. How much and low long this seriousness took to develop likely varied — but by how much is very hard to measure. I personally would have liked to see more immediate enforcing of masks, but that poses difficult logistics so I understand why it took longer.

                > . And I give extra weight to every grocery worker who might have been spared infection because the lockdowns increased, even of only marginally at first, the # of people wearing masks. Those are the people who enable me to eat even as I reduce my risk.

                This was part of why people not wearing masks (and place not enforcing masks earlier) and being more careful overall surprised me — I had the same logic as you did, e.g., “Huh, these people are risking themselves so I can get food and such. Let me take these small, low-cost measures to protect them.”

          • Joshua says:

            Mendel –

            Don’t know if you’ll see this…

            Seems that as yet, there is uncertainty in the evidence:

            > study released by Germany’s chief virologist Christian Drosten and others this week backs Zerr’s skepticism. Here’s what they found: “Analysis of variance of viral loads in patients of different age categories found no significant difference between any pair of age categories including children,” the researchers wrote. “In particular, these data indicate that viral loads in the very young do not differ significantly from those of adults.”

            https://www.google.com/amp/s/www.vox.com/platform/amp/2020/5/2/21241636/coronavirus-children-kids-spread-transmit-switzerland

            The article I excerpted is linked at the above URL, where the uncertainty in the evidence is discussed broadly.

    • Zhou Fang says:

      If we accept the New York antibody study as being correct, then the fact that infection level in New York exceeded 20% *under lockdown* seems pretty good evidence that herd immunity threshold for covid19 in the absence of a lockdown would be quite a lot higher than that number.

      • The fact that NYC might have gotten as high as 20% to me suggests that lockdown just came way too late.

        • Twain says:

          This data from the CDC has me wondering just that: https://www.cdc.gov/coronavirus/2019-ncov/covid-data/covidview/04102020/nssp-regions.html.

          According this data, the peak value of COVID-Like Illnesses (CLI) reported at ERs in NJ, NY, and PR was March 23 to March 29, with reported cases decreasing since then in that region and nationwide (albeit, there is ~3-4 wk lag in some data; but CDC now has CSV files to download!). Assuming it takes 5-11d for symptoms to present (DOI: 10.7326/M20-0504; 50% present in 5d, 95% present in 11d), that means peak infections occurred 1w prior, so March 15 to March 22 — right when shutdowns were starting. The nationwide values are similar, with the peak occurring March 16 to March 22, so maximum infections 1w prior would be March 9 to March 15 — again right when or just before most states shutdown en masse.

          A major argument for shutdowns was to “flatten the curve” — namely, implement when infection was occurring in the exponential-growth phase (and before the point-of-maximum growth if following a logistic model) to maximize the reduction of peak numbers.

          But if said curve was near its peak or beyond, then shutdowns came too late.

          Thoughts?

          • Joshua says:

            Twain –

            > But if said curve was near its peak or beyond, then shutdowns came too late.

            Not going to comment on the math – except to ask to what degree might the shutdowns occurring when they occurred help move the peak earlier than it would have been otherwise…

            But I certainly was saying “What the hell are they waiting for?” before they shut stuff down. Seemed to me that more or less after community spread was established there was sufficient rationale.

            Seemed to me that if think you’re going to have to shut it down, then you’re better off risking shutting it down early rather than late. That said, prolly better late than never.

            • Twain says:

              Joshua,

              > except to ask to what degree might the shutdowns occurring when they occurred help move the peak earlier than it would have been otherwise

              If the shutdowns occurred during the growth-phase, there should have been *some* continued increase in cases after the shutdowns started since they cannot prevent all spreading (especially with supermarkets and other super-spreading locations open and not mandating masks for ~1-3 weeks into the shutdown). But said continued increase in cases did not occur, which suggests the peak infections had already occurred or were close to occurring.

              > Seemed to me that if think you’re going to have to shut it down, then you’re better off risking shutting it down early rather than late. That said, prolly better late than never.

              Not necessarily. If the virus had already reached peak infections naturally before the shutdown, then said shutdown would not be able to serve its purpose — there is no curve to flatten. You could argue that the shutdowns may cause a faster decrease toward 0 cases — since we sequestered everyone at or just after the time of peak infections, it would run out of new hosts faster. But given the current implementation of shutdowns (see above parenthesized comment), IMO it is not likely the case. 

              Also consider that implementing a shutdown *too late* or *not well enough* has a fat-tail risk of incurring massive economic/social damage while not doing much to curb the pandemic.

              • Joshua says:

                > But said continued increase in cases did not occur, which suggests the peak infections had already occurred or were close to occurring.

                I was looking at the earliest dates from when you said the shut downs were starting and the latest dates for when the peaks became apparent, and it seemed there was some wiggle room.

                > f the virus had already reached peak infections naturally before the shutdown, then said shutdown would not be able to serve its purpose — there is no curve to flatten.

              • Joshua says:

                Again, with the accidental posts…

                > You could argue that the shutdowns may cause a faster decrease toward 0 cases — since we sequestered everyone at or just after the time of peak infections, it would run out of new hosts faster

                Except there are still high numbers after the peak – which has an accumulating impact of the healthcare system, doctors, nurses, etc. Seems like even if they come after the peak of the peak, even if they come late, they could have an effect on hastening the decline, and thus providing some relief to the heroes that I think deserve outsized focus. Flattening the curve is the main target – but is it the only target?

                > (especially with supermarkets and other super-spreading locations open and not mandating masks for ~1-3 weeks into the shutdown).

                Behaviors were changed due to the “shutdown” even if supermarkets weren’t closed and everyone wasn’t wearing masks.

                > Also consider that implementing a shutdown *too late* or *not well enough* has a fat-tail risk of incurring massive economic/social damage while not doing much to curb the pandemic.

                So does not implementing one – and it seems to me that there’s a lot of uncertainty related to “not doing much to curb the pandemic.”

                Do you think that the shutdowns, even if they came late, didn’t prevent many more infections from occurring, at least ion some areas, over the short term?

                Even you think that long term the same # are likely to get infected, that argument is so counterintuitive I want to read you actually write that’s what you think is the case.

              • Twain says:

                Yikes! Forgot to include an important point:

                For the NY/NJ/PR region, the shutdowns occurred on March 16 (for NY) and March 20 (for NJ); so they occurred right as oor just after peak infection occurred.

                Maybe there were a few days of wiggle-room for the shutdown to have an affect; but these few days would likely have nominal effects unless Social Distancing was extremely effective — which we don’t know yet based on available data

              • If the shutdowns were effective then we expect peak infection rate to be *right at the time they shutdown* which is exactly what you’re saying. In other words, all of what you’re saying is consistent with the model “shutdowns work” and inconsistent with the model “even if we hadn’t shutdown it still would have peaked”. That is clearly not what would have happened given that our evidence is at most 20% of NY was infected (yes, at most, because of biases in the study).

              • Twain says:

                Daniel,

                Agreed. Thank you for clarifying!

                Something I’ve been wondering: Reports indicate that the first cases in NYC occurred first week in March. So first infections occurred last week in February — that is ~1 month before the shutdown occurred in NYC on March 20 (announced on March 16 but not enforced until March 20).

                So that gave the virus ~1 month to spread unabated. From a virological perspective, it’s surprising that a novel, infectious, and highly transmissible (e.g., airborne) virus only spread to 20% or less of the population in NYC. Why didn’t it spread more? (Granted, we don’t really know until we have reliable randomized serological data…still waiting on that.)

                This makes me wonder: What other driver could be limiting spread? Perhaps there is a lacking zoological reservoir supporting the virus (e.g., like pigs and birds for influenza)? Perhaps there is cross antigenicity with antibodies for common human coronaviruses? Perhaps its airborne transmission isn’t that effective — e.g., it doesn’t deliver a high average viral load to cause infection most frequently?

                Overall, I don’t have a hypothesis. But I’m surprised the virus has not spread more.

              • Zhou Fang says:

                I will point out here that there was likely some voluntary social distancing happening prior to shutdown.

          • I don’t think shutdowns came too late to be before the logistic like curve peak first derivative. That would occur at typically 50% of people infected, yet we have evidence for closer to 20% and probably less (again due to selection bias in the survey in NY and due to false positive rates being above 0% and maybe as high as 13% for that test they used).

            So, basically what I meant by “too late” was “too late to avoid overwhelming the hospitals” not “too late to have any useful effect”

      • Carlos Ungil says:

        Exposure patterns may be quite different in places which are less connected than NYC. The herd immunity threshold is a property of the population and it’s not impossible that, as suggested in the paper, less than 20% could be enough to “deactivate” the “connectors” under certain circumstances.

    • Anoneuoid says:

      I think the average R0 in the US is more like 8, but only for like 3 days. Also, the vast majority of people infect 0-2 others but a few infect hundreds or thousands so the averaged R0 is not very informative.

  12. Yuling says:

    With the spirit that inference should be on quantities that could be observable, what if the prayer only prays for a p=0.04 during the whole procedure?

  13. Paul says:

    Professor Gelman – I just have to say… …I love your blog.

  14. Mendel says:

    Highlights from the Leibovici paper comments:

    — “providence-based medicine” (Arthur Leibovitz, Director of Geriatric Ward – Shmuel Harofe Hospital)

    — “So, applying my “rule of thumb,” to conclude that Leibovici’s prayer
    result is “good” we are also forced to conclude that the “focus of
    prayer,” by implication God, is particularly focused on male urinary tract
    infections. I doubt that this is the case.” (Gary S. Hurd)

    — “I pray now for my own papers to be published” (Luc-Marie Joly)

  15. paul alper says:

    Leibovici’s famous paper on retroactive intercessory prayer can be found at

    https://www.ncbi.nlm.nih.gov/pmc/articles/PMC61047/

    The paper states

    “Remote intercessory prayer said for a group of patients is associated with a shorter hospital stay and shorter duration of fever in patients with a bloodstream infection, even when the intervention is performed 4-10 years after [!!] the infection.”

    “Length of stay in hospital and duration of fever were significantly shorter in the intervention group than in the control group (P=0.01 and P=0.04, respectively).”

    The cost-effective delayed intervention: “A list of the first names of the patients in the intervention group was given to a person who said a short prayer for the well being and full recovery of the group as a whole.”

    Leibovici’s satire was both much appreciated, reviled and misunderstood. An interview with Leibovici can be found at

    https://www.causeweb.org/wiki/chance/index.php/Chance_News_82#Retroactive_intercessory_prayer.E2.80.94II:

    “I was sure that it will be recognized as a satire if only because of the references (one of them to a piece by Borges [dealing with the fallacy of time]). It [i.e., Leibovici’s study] was [however, recognized] by some [as satire]– it is used as an example and exercise in several courses on statistics and epidemiology. And it wasn’t by many. There are some scathing pieces [i.e., responses] on the folly of the trial (I didn’t mind them, although they didn’t speak too well of the intelligence of their writers). I felt embarrassed because some people quoted it as a sign of God’s powers, and I didn’t intend it as a parody on honest belief.”

    “The purpose of the BMJ [British Medical Journal] piece was to ask the reader the following question: Given a ‘study’ that looks methodologically correct, but tests something that is completely out of our frame (or model) of the physical world (e.g., retroactive intervention or badly distilled water for asthma) would you believe in it?”

  16. Eric B Rasmusen says:

    Andrew, I actually wrote a paper on the idea you suggest, that God can and would defeat a randomized study. I need to brush it off and get it published somewhere. Comments welcomed, everyone. The idea has big practical implications, unusual for a philosophic idea. It’s a nice application of game theory to theology.

    “The Concealment Argument: Why No Conclusive Evidence or Proof for God’s Existence Will Be Found.” Logic and Biblical evidence suggest that God wishes that some but not all humans become convinced of His existence and desires. If so, this suggests that attempts to either prove or disprove such things as God’s existence, past miracles, or present supernatural intervention are doomed to failure, because God could and would take care to evade any such efforts. (http://rasmusen.org/papers/conceal-rasmusen.pdf).

    • jim says:

      “God could and would take care to evade any such efforts. “

      So, there’s your proof! All humans are fallible, but through all the Stelons and Eons and Eras and Periods and Epochs in the history of the universe, God hasn’t even messed up once in his effort to evade detection, which proves s/he is not human, and therefore must be supernatural.

    • A.G.McDowell says:

      The argument that God refuses to prove their existence in order to allow free will is well known in society – e.g. https://www.goodreads.com/quotes/35681-now-it-is-such-a-bizarrely-improbable-coincidence-that-anything – but religions will not face up to its consequences, and many of their followers are still further gone, with a more accepting approach to miracles than e.g. St John Chrysostom, who regarded biblical miracles as signs of Christ’s authority, and did not expect or particularly appreciate miracles in his present day. Religions do not seem to have noticed that the power of modern statistical techniques to find correlation, even after the fact, limits the extent to which God could respond to healing prayers without being caught at it, especially when prayer lists are published on the web, and health services record medical records for later analysis. In the UK, attendance at church study sessions will bring you into contact with people who really do believe in the literal power of prayer to heal, even in perfectly ordinary Church of England churches where the Priest does not hold such literal beliefs.

      I do note that although the most traditional versions of the Book Of Common Prayer include prayers for intervention both against plague and against bad weather, I have not heard either used seriously.

      • Eric B Rasmusen says:

        You don’t get out of the house enough, Mr. McDowell. A google search quickly reveals “Prayers for use during the Coronavirus outreach” from an Anglican Birmingham, Alabama cathedral (https://adventbirmingham.org/about/) which includes this prayer:

        A Prayer for a time of any common Plague or Sickness (modernized for diocesan use). O ALMIGHTY God, who in your wrath sent a plague upon your own people in the wilderness, for their obstinate rebellion against Moses and Aaron; and in the time of king David, sent a plague of pestilence that killed seventy thousand, but remembering your mercy spared the rest: Have pity upon us miserable sinners who now are visited with great sickness and mortality; and in the same way that you then accepted an atonement and commanded the destroying Angel to cease from punishing, so may it now please you to withdraw from us this plague and grievous sickness; through Jesus Christ our Lord. Amen.
        https://adventbirmingham.org/wp-content/uploads/2020/03/Prayers_March2020.pdf

  17. Jonathan says:

    1. Great cat pic.
    2. God is on to the remote prayer thing, and inserted a randomizer to hide the intercessions. The randomizing has the effect of generating results that appear non-random when viewed in a certain way because God built humans to extrapolate naturally that results have causal relationships.

    • Andrew says:

      Jonathan:

      It’s tricky. God created humans, but the “God” that we understand is our inference of God based on our own experiences. We don’t know the true God; we only know the inferential God that we created. One way we can assess the reasonableness of our inferential God is to see what sort of humans this God would create, and then compare those hypothetical humans to us. It’s posterior predictive checking.

      The confusing thing is that the generative model is not the same as the inferential model. Speaking generatively, God created humans. But, speaking inferentially, humans create God. Or, to me more precise, humans create inference about God.

      All the above is conditional on there being a God. But that’s just Bayesian, to condition on your model. And Bayes of course was a cleric.

      • Eric B Rasmusen says:

        That’s a great way to put it, and both Bayes and Pascal would apppreciate it, I bet (and Paley).

        Probably philosophers have already figured this out, and maybe this is just restating Hume and Kant, but it seems like “inferential cause” is a good addition to Aristotle’s “4 Causes”. Those are 4 ways to answer the question, “Why does a chair exist?”

        1. The material cause. Because it’s wood, and wood exists.
        2. The formal cause. Because it’s got the shape of a chair.
        3. The efficient cause. (“generative?”) Because the factory made it.
        4. The final cause. Because I need something to sit on, so I told the factory to make it for me.
        5. The inferential cause. Because when you or I look over there, we see something that looks like a chair.

        So I think God would be the efficient and final cause of Man, but Man would be the inferential cause of God (looking at it from Man’s point of view, as we must). Nietzsche, to be sure, said Man, or perhaps The Great Man, was the efficient and final cause of God too. But Nietzsche is dead.

  18. Jeff Walker says:

    Didn’t Andrew’s randomization comments remind anyone of the Terry Schaivo (“Best Friends Forever”) episode of South Park, where the religious/republicans/media want to keep Kenny on life support but god had chosen him to die because he needs Kenny’s mad gaming skills to lead the angel army against satan?

    https://southpark.cc.com/clips/n7qf6s/using-the-republicans

  19. GK says:

    randomized blinded trial of intercessory prayer with nearly 1,000 patients
    https://www.ncbi.nlm.nih.gov/pubmed/22894887

    see what you make of it

  20. Ethan Steinberg says:

    There is a neat argument that prayer and ESP studies serve a useful purpose as a control group for science: https://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/

    Because we know that the effects are impossible, you are basically guaranteed to find that all of the “positive” results are flukes. Studying these “positive” results helps us understand when and how statistics/scientific procedures fail.

  21. Bill Jefferys says:

    To get back to the statistics, it’s dollars to doughnuts that there will be major flaws in the experiment anyway, such as sampling to a foregone conclusion (if they feel it’s OK to add subjects to the study from time to time and then “stop” when p<0.05) and many of the other flaws that Andrew talks about here.

  22. John Mashey says:

    1) Dr Lakkireddy is a cardiologist at the Kansas City Heart Rhythm Institute .. which seems a strange choice for an infectious disease trial, so maybe it’s via HCA Midwest Health(?)
    https://www.acc.org/membership/person?id=ba5513b1-be88-4009-845f-fc1e5b729482
    “Dr. Lakkireddy is affiliated with the HCA Midwest Health network of hospitals located throughout the Kansas City, Mo and Overland Park, Kan. metro areas. “

    2) https://clinicaltrials.gov/ct2/results?cond=&term=prayer&cntry=&state=&city=&dist= is worth checking.
    Right now, it’s the first one, “The COVID-19 ICU Prayer Study”, whose status is Not yet recruiting.
    That’s NCT04361838
    https://clinicaltrials.gov/ct2/show/NCT04361838?term=prayer&draw=2&rank=1
    “This is a multicenter; double blind randomized controlled study investigating the role of remote intercessory multi-denominational prayer on clinical outcomes in COVID-19 + patients in the intensive care unit. All patients enrolled will be randomized to use of prayer vs. no prayer in a 1:1 ratio. Each patient randomized to the prayer arm will receive a “universal” prayer offered by 5 religious denominations (Christianity, Hinduism, Islam, Judaism and Buddhism) in addition to standard of care. Whereas the patients randomized to the control arm will receive standard of care outlined by their medical teams. During ICU stay, patients will have serial assessment of multi-organ function and APACHE-II/SOFA scores serial evaluation performed on a daily basis until discharge. Data assessed include those listed below….”

    • jim says:

      “To think, “Hey, this might work,” is like thinking that that lottery ticket you just bought might win the hundred million dollar jackpot. “

      no…! :) Someone actually wins the lottery eventually.

      Yes this is definitely NPR’s effort to kiss up to a certain demographic that has the “well maybe it *does* work” mentality. Most of NPRs stories are pretty transparently aimed at a specific interest group. It would be interesting to work out the proportion of stories each group gets and track how it changes over time.

Leave a Reply to Ethan Steinberg