The “noble lie” in science reporting

David Weakliem writes:

A few days ago, the New York Times had an opinion piece by Huw Green, a clinical psychologist, which said “A clear causal link between psychiatric illness and gun violence has not been established…” I followed the link, which was an interview with Ragy Girgis, a professsor of psychiatry at Columbia University. That story had a caption saying “Findings from the Columbia database help dispel the myth that having a severe psychiatric illness is predictive of who will perpetrate mass murder.” It also contained a link to an article by Dr Girgis and others using the database (an attempt to compile a comprehensive list of mass murders since 1900), which said “the prevalence of psychotic symptoms among mass murderers is much higher than that in the general population (11% v. approximately 0.3-1%).” That is, people with psychotic symptoms were between 10 and 30 times more likely to commit mass murder than people without psychotic symptoms.

Weakliem continues:

How did we go from 10 to 30 times more likely to “dispel the myth”? The interviewer asked “Are people with mental health disorders more likely to commit mass shootings or mass murder?”

Good question. Here’s what Weakliem suggests:

This could just be a case of miscommunication–anything involving probabilities can be confusing. However, I [Weakliem] think it’s an example of a more general problem: sometimes a focus on making sure that people don’t draw the wrong conclusions comes at the expense of explaining what the research actually found.

I [Weakliem] first noted this when writing a post on a study of coffee consumption, where accounts emphasized a point that wasn’t supported by the data: that benefits only occurred with moderate consumption, not high consumption. I saw another example later in the summer, when a study of diet and exercise was described as showing “that healthy eating and regular workouts do not, in isolation, stave off later health issues. They need to be done together.” In fact, the study suggested exactly the opposite—exercise and diet had additive effects on mortality, and no interactions were found. The reason seemed to be a goal of getting people to think of exercise as a way to improve one’s overall health rather than a way of getting away with a bad diet . . .

I think Weakliem has a point, that health reporting often seems to have two conflicting aims: (1) to report the science, and (2) to encourage healthy behavior, and these goals can interfere. An additional twist comes when bad science is being reported, and journalists put skepticism aside when reporting claims that support their political or scientific ideologies.

P.S. More from Weakliem here.

54 thoughts on “The “noble lie” in science reporting

  1. What I’ve always heard is the link is between psychotropic drugs and “mass murder”, in particular SSRIs.

    And after living with someone whose personality changed within two weeks of being put on some of these drugs (they became delusional and developed a rage problem to the point they could no longer work), I can believe it.

    I’ve never looked into the data, but it probably is some subset of people who are susceptible or misdosed.

    • And this is a classic transposing the conditional fallacy:

      “the prevalence of psychotic symptoms among mass murderers is much higher than that in the general population (11% v. approximately 0.3-1%).” That is, people with psychotic symptoms were between 10 and 30 times more likely to commit mass murder than people without psychotic symptoms.

      So mass murderers are 10-30x more likely (to have been reported) to have psychotic symptoms than usual. That doesn’t tell us the percent of “psychotics” that commit mass murder vs “normals”.

      We would need to know the number of people meeting the psychosis criteria to get that answer, using 1% of the US pop… we get around 0.01% (1 per 10k) vs 0.0001% (1 per million).

      • Thank you, I copied that passage from the original post because it sounded fishy and I was hoping someone might identify the exact smell for me here.

        So, Weakliem’s argument with Green is founded on a logical fallacy, and the majority of the readers have scanned right past it and gone on to consider Gelman’s question about communication priorities or to discuss the relation of mental health and violence.

        It continues to amaze me that we’re able to get anything useful done.

        • Note that the quoted passage is fine.

          > “the prevalence of psychotic symptoms among mass murderers is much higher than that in the general population (11% v. approximately 0.3-1%).”

          Let’s use the following notation: M = mass murderer, P = psychotic symptoms, notP = no psychotic symptoms

          p(P|M) = 0.11

          p(P) = 0.003 to 0.01

          > That is, people with psychotic symptoms were between 10 and 30 times more likely to commit mass murder than people without psychotic symptoms.

          The article says that p(M|P)/p(M|notP) = 10 to 30. Let’s check it.

          Using the identities

          p(M and P) = p(M|P)p(P) = p(P|M)p(M)

          p(M and notP) = p(M|notP)p(notP) = p(notP|M)p(M)

          we can calculate that

          p(M|P)/p(M|notP) = ( p(P|M)p(M)/p(P) ) / ( p(notP|M)p(M)/p(notP) ) = p(P|M)/p(notP|M) p(notP)/p(P)

          The first factor p(P|M)/p(notP|M) = 0.124 because p(P|M) = 0.11 and the complement p(notP|M) = 0.89

          The second factor p(notP)/p(P) is between 0.997/0.003 = 332 and 0.99/0.01 = 99.

          In conclusion, p(M|P)/p(M|notP) = 12 to 41

          Not exactly the 10 to 30 reported but there is a large overlap and the difference could be due to rounding.

    • SSRIs are known to trigger manic episodes in people who are susceptible — that’s most likely what happened to the person you knew.

      If there’s any causal path between SSRIs and violence, I think it’s probably through this. Seems horribly confounded though because they are so widely prescribed, including as adjuncts to other treatments.

      • When you see someone suddenly get these symptoms (that they never had before) soon after this new thing happens, that is far more convincing then looking at averages of two groups.

        • Anoneuoid –

          > When you see someone suddenly get these symptoms (that they never had before) soon after this new thing happens, that is far more convincing then looking at averages of two groups.

          What’s the background rate of people who suddenly become delusional and develop a rage problem to the point they could no longer work?

          Also, obviously there are some preexisting variables that could be important with thst individual, perhaps primarily those that would be related to why they stated taking the SSRI to begin with.

          Given that you’ve beem pushing this for a while and strongly suggesting a significant and prevalent causal phenomenon, surely by now you’ve got some further data-based insight.

        • Old Tom Paine talked about revelation. He said that he granted that the Creator could reveal whatever he wanted to people in an utterly convincing way, but as soon as the recipient of the revelation talks or writes about it it is just words which nobody has a way of knowing to be true or false or in between. The analogy with knowledge from experience, and that experience as typed up by a stranger on the Internet, I leave as an exercise to the reader.

  2. This seems like the general conflict between descriptive claims (traditional science) and normative claims (more modern activist version of science). The former would consider exaggeration or misleading findings (either through garden of forking paths, or in communication) to be akin to fraud, while the latter consider that the ends (some normative goal to make the world better) justify the means (noble lies).

  3. A journalist doing a story on GREs for a physics journal talked with me about a paper I’d written on a much-publicized work that had claimed to show that GREs were worthless predictors. I’d showed that work was filed with major errors, and the data actually implied the opposite of the conclusions. (Andrew has blogged about this. https://statmodeling.stat.columbia.edu/2020/12/14/debate-involving-a-bad-analysis-of-gre-scores/) I kept trying to steer the conversation toward the importance of maintaining scientific honesty and competence, rather than arguing about fuzzy policy issues. The journalist said “But the people I talk with say this is a moral issue.”

  4. People who are mentally well almost never commit mass killings (the only exception might be as actions in wartime). I literally went through the FBI files line by line and transcribed a summary of every mass shooting they listed for 2 decades, and none of them told a story “joe was a regular guy who one day after playing Grand Theft Auto for a while just decided it would be a good idea to kill people.”

    It’s always some kind of anguish or whatever. Often job related, relationship related, or in recent years suicidality from social misfit status https://fivethirtyeight.com/features/suicide-prevention-could-prevent-mass-shootings/ 100% of these perpetrators have some form of mental illness essentially by definition. Of course they aren’t all diagnosed! Just as not all people with colon cancer are diagnosed!

    The problem seems to be a concern that the statement “100% of mass murderers are mentally ill” which should be uncontroversial, will be taken by an innumerate public to mean “100% of people diagnosed as mentally ill are an immediate threat to public safety” which is far from true.

    Round numbers let’s say 20% of the US is diagnosed with mental illness (something like that fraction of the US is on antidepressants for example). Yet there are on the order of 10 mass public “active shooter” incidents per year so 10/(330e6*.2) = 1.5e-7 or around 1/10M people with mental illness are a major threat to public safety. The truth is more like 30-40% of the US is mentally ill, when you include people who believe in clearly false conspiracy theories and alter their lives around those beliefs in maladaptive ways. It’s still around 1/10M because I’m rounding off to order of magnitude but the point is innumeracy is the issue here. People are reporting the wrong stuff out of lack of capacity to correctly communicate to the public. It’s simple to say “while 100% of mass shooters are mentally ill, because only about 10 events occur per year around 1 in 10M people with mental health issues are public shooting risks, making it very hard to figure out who to focus efforts on…” Or something. But no, it’s not the narrative…can’t accurately portray reality if it’s politically unpopular…sigh

    • Since mental illness is, perhaps, part of the human condition, and not really going anywhere, I’d suggest focusing efforts on reducing gun culture: this includes sales, media portrayal, video game portrayal, music portrayal, removal of existing assault weapons/high capacity weapons, and, in the US, dissolving the NRA. Countries with relatively high rates of mass shootings and violent deaths from guns tend to have a culture of gun nuts, including a few that post on this message board.

      • Some level of mental illness is not going to go away, I agree. But the type, amount, and intensity of mental illness in a society are subject to change. Stress, social isolation, drug use, physical abuse… lots of things influence people’s state of mental well-being.

        I certainly don’t object to also making it harder for people to get weapons that allow them to kill large numbers of other people, indeed I strongly support that. I also support trying to keep weapons specifically out of the hands of people with severe mental illness. But I don’t think it’s the case that we just have to accept that we’re going to have about the same number of would-be mass murderers no matter what we do, if that is what you meant by “mental illness is, perhaps…not really going anywhere.”

        • You know what’s the most strongly predictive association in all of social sciences that I’m aware of? Log(homicide rate) is linearly predicted by Gini coefficient.

          Essentially all the developed nations with lower homicide rates than the US have dramatically lower Gini coefficient than the US. It’s even lower than that when you include social services as income.

        • > You know what’s the most strongly predictive association in all of social sciences that I’m aware of? Log(homicide rate) is linearly predicted by Gini coefficient.

          It’s certainly noteworthy how relative poverty in societies, more so than poverty itself, predicts various types of crime, violent crime among them. And it’s not just across countries but across communities in the US as well. It’s also interesting to look at the interaction among gun ownership rates, relative poverty rates, and gun violence rates.

  5. The Times story itself amplified the confusion. Take the sentence, “Findings from the Columbia database help dispel the myth that having a severe psychiatric illness is predictive of who will perpetrate mass murder.” Well, given that so many have severe psychiatric illnesses compared to the number of mass shooters, it’s true that identifying the first is not very predictive of who will do the second. But taking seriously the very notion that such broad categories *could* be predictive of uncommon events and raising it to the level of myth — that’s participating in the innumeracy it supposedly opposes.

    Of course, from a policy point of view, upping diagnosis and treatment of severe mental illness can have a meaningful impact on the frequency of these episodes of violence even if you can’t predict in advance which people suffering from these conditions are the ones who might pull the trigger.

    FWIW, I’ve come to think that countering the widespread inability to think probabilistically is perhaps the most important function of introductory stats courses. A much bigger social payoff than knowing how to test for normality….

    • Peter Dorman said:

      “FWIW, I’ve come to think that countering the widespread inability to think probabilistically is perhaps the most important function of introductory stats courses. A much bigger social payoff than knowing how to test for normality….”

      I agree, at least in part. But as I recall, when I was in high school, probability was a unit in what then was then called “Advanced Algebra” (I think it’s now called “Algebra 2”.) But if I remember correctly, there was also some teaching of probability in grades 6 – 8. And I was also lucky to be able to take a third physics course in high school — that included the Heisenberg Uncertainty Principle. I wish all students had this much early introduction to probability. (Also, I have heard that some parents do not approve of the teaching of probability, because of it’s connection with gambling. Some schools have gotten past this by using “Spinners” instead of dice to introduce probability.)

  6. “A clear causal link between psychiatric illness and gun violence… ”

    “Are people with mental health disorders more likely to commit mass shootings or mass murder?”

    When did “gun violence” become equal to “mass murder”? All these highly qualified PhD scientists and statisticians and no one caught this simple switch? That’s aside from the lack of scientific a definition of either term.

    Funny, an article critical of both scientists and reporting fumbles the fundamentals.

  7. To be as fair as possible, in the interview, Dr. Girgis does not deny that mass murder is statistically associated with mental illness, but rather that mental illness is the primary cause of mass murder

    The data suggest that while it is critical that we continue to identify those individuals with mental illness and substance use disorders at high risk for violence and prevent the perpetration of violence, other risk factors, such as a history of legal problems, challenges coping with severe and acute life stressors, and the epidemic of the combination of nihilism, emptiness, anger, and a desire for notoriety among young men, seem a more useful focus for prevention and policy than an emphasis on serious mental illness, which leads to public fear and stigmatization.

    First, understand that mental illness as the primary cause of any mass murder, especially mass shooting, is uncommon. Half of all mass shootings are associated with no red flags—no diagnosed mental illness, no substance use, no history of criminality, nothing. They’re generally committed by middle-aged men who are responding to a severe and acute stressor, so they’re not planned, which makes them very difficult to prevent. So, we must look much further upstream.

    The points being that:
    1. A large fraction of mass murders are not linked to diagnosed mental illness even if the mentally ill are disproportionately represented
    2. Even the perpetrators with diagnosed mental illness usually have another clearly identifiable acute motivator.

    I imagine the context Girgis is responding to is something like the following.

    When there is a mass shooting reported in the news, there is usually a discussion along the lines of “we need to improve mental health care in this country,” to which Girgis would respond “even if we successfully treated all the diagnosed mental illness, there would still be a bunch of mass murders.”

    Most of the confusion here is created by the editorial context, like the question “Are people with mental health disorders more likely to commit mass shootings or mass murder?” which he doesn’t actually confirm or deny in the following text, and the title of the other article “New Findings from the Columbia Mass Murder Database”. It makes me wonder if those were actually the questions he was responding to.

    As for the empirical question of whether the mentally ill perpetrators’ illness was the “primary cause”, it seems tough to quantify. It seems to be a question of “had this acute stressor been removed, would this mentally ill person still have done mass murder” vs “had this person’s mental illness been treated but the stressor left unaddressed, would they still have done mass murder”, which is really down to a subjective judgement. But it does at least seem like it’s based Girgis’s close reading of many incidents; from the research article:

    For the present study, we focused on personal-cause mass murders, meaning those driven by personal motivations and not occurring within the contexts of war, state- or group-sponsored terrorism, gang activity, or organized crime. Mass murders stemming from family matters and other killings of close associates were not excluded. Felonious mass homicides with automobiles or other vehicles, related to impairment or recklessness, were excluded. Additionally, we employed the Congressional definition of mass murder, involving three or more fatalities, excluding perpetrators. We reviewed 14 785 cases of murder in total for potential cases of mass murder. For a complete description of our methods, sources examined, and a flow diagram of our search results, see the online
    Supplement, Supplementary Table S1 and Fig. S1.

    I feel like if I went through and read all the cases myself, I’d probably concur with his assessment there. I also appreciate the attempt to stick with a colloquial sense of mass murder–if they were to stick strictly to the legal definition like some do, the database would be dominated by gang violence and be completely nonrepresentative of what most people mean by mass murder.

    • I agree with your assessment regarding their choice of sample to analyze – but I am a little confused about the numbers. They say there were 14,785 “potential incidents of mass murder” and 1,315 “final” cases for analysis (as well as 408 “spree” murders). In the supplementary appendix 1, they show the exclusion criteria and numbers, one of which is “Sponsored (1595).” I couldn’t find the definition of sponsored. I agree that gang-related mass murders should be excluded (for their research purpose), but it isn’t clear to me whether that is part of the “sponsored” category or whether those cases were excluded prior to the 14,785 “potential” cases.

      I have no particular expertise on this subject, but I would have expected gang-related mass murders to be more common than the number reported for “sponsored.” If that is not the category, then do we know how many such mass murders occurred? I am asking because when communicating such results to the public, one important aspect is the size of the set they are examining compared with the size of the problem the public hears about. Even if a link between severe mental illness and mass murder were established, what percent of the mass murders are we talking about? (and, of course, the link itself is subject to the proper interpretation of the conditional probabilities, as pointed out by others here)

      • I have to wonder about the classification of arson and vehicular cases as well. It is much easier for that to seem like an accident than someone shooting another person.

        Shooting 3 people on accident basically doesn’t happen, so we can be confident in the classification. I’d guess why people are so much more focused on those vs the others.

      • To shed some more light on the puzzle: the Gun Violence Archive shows 619 mass shootings in 2022 (11 months). The article covers mass murders from 1900 through 2019. At the 2022 rate (which is certainly not the best assumption we could make), that would amount to 73,661 mass shootings. But the article analyzed a total of 14,785 “potential” mass murders. The mismatch between the numbers (in addition, the Gun Violence Archive data is only US, while the article used world-wide data for reports in English) puzzles me – I can’t grasp the extent to which the article captures the phenomenon it is studying.

        • The thing you’re likely missing is that the Gun Violence Archive looks at *mass shootings* which they define as events where more than … It might be 3 or 4 I don’t remember… People are *struck by bullets*

          Now this mostly involves ordinary crime such as liquor store holdups and drug turf wars and carjackings and such. The GVA people intentionally use this “purely statistical” method of classifying “mass shootings” because it hyperinflated the numbers over what people think of as “mass shootings” (ie. A person comes to a place like a mall with the intent to kill as many people as they can before being killed by cops etc)

          By making people think “omg there’s one person a day in the US committing a mall shooting” they massively ramp up support for their cause. In reality the mall shooting type event occurs about 10 times a year order of magnitude, not 350

          Also not everyone struck by bullets dies. In the GVA data it includes anyone struck by fragments of bullets in the big toe or whatever.

        • Anon
          The Gun Violence archive has data broken down by incidents, injuries, and deaths. The latter are around half of the injuries which are around half of the incidents. I still feel like there should be more potential mass murders over the 120 year period than the paper states – and I still don’t understand where gang-related mass murders fall in the numbers. It is possible that the paper is accurate, I’m just struggling to make sense out of the numbers.

          I think your comments about the politics and potential misrepresentation to drum up support are simply not relevant or constructive. If you are claiming that the GVA people – or me – have a motive to misrepresent the facts here, then I think you are off-base (I know you are in my case, but I can’t speak definitively for the GVA). Rather, I think your comment reflects your own political views.

        • The redefinition of “mass shooting” to “n people struck by bullets” was actually initially done by an anonymous political activist on Reddit who was very explicit about his purpose. I unfortunately can’t remember which one, this new definition first hit the media somewhere around 2014 or something.

          MOTHER JONES of all organizations wrote this article decrying this approach in response: https://www.motherjones.com/politics/2015/12/no-there-were-not-355-mass-shootings-this-year/

          Mass shootings of the “people show up trying to kill a lot of people” are a terrible and important event. Ordinary liquor store robberies are also a terrible and important event. Conglomerating them together for political ends by some activist is a bad idea, because the solutions for the two problems are very very different and unless you keep track of the criminological issues separately, you will not make progress against either kind of problem.

          Andrew I can forgive you for misunderstanding, but I assure you I’m not making light of people being struck by bullets, only saying that we gain nothing real by doing a bad job of studying the problem.

        • Fortunately we don’t need to focus on just a single metric. We can look at total gun deaths, total shooting incidents, total number with more than 1 intended victim, total number with more than 1 people struck by bullets, total with more than n people struck by bullets, etc. The relevant metric(s) depend on what policy you are trying to evaluate or what point you are trying to make.

          I do think there are some discussions in which one would want to distinguish between the guy (and it’s always a guy) who shows up at a mall or a church or a festival and tries to kill as many people as possible; the classic disgruntled co-worker who shows up at his workplace or former workplace and tries to kill specific people or maybe just as many as he can; the group of gang members who gets in a firefight with another group of gang members; and other types of criminal activity. If you want to reduce these types of violence then the policies that would help in some of these categories might not help in others. For example, if you’re a big company you might want to know the stats on ‘disgruntled co-worker’ shootings so you can decide whether to have some sort of psychological screening or counseling for mentally troubled workers, but that’s not going to help with gang shootings at all.

          On the other hand, there are other discussions in which the distinction is pretty much irrelevant. If you’re arguing for a ban on weapons that can fire more than fifty rounds in a minute when used by a person of average skill, the distinction probably doesn’t matter much and it makes sense to cite the total number of people hit by bullets, or killed by them, etc., using the now-standard definition of ‘mass shooting’ that Anonymous doesn’t like.

          Basically I think Andrew is wrong to mock the idea that the distinctions are irrelevant, and Anonymous is wrong to dismiss the idea that the distinctions sometimes don’t matter much.

        • Andrew,
          “Mock” isn’t quite the right word. When you say ” . . I can’t even” I think you are indicating disdain for the position you are responding to. So I should have said “dismiss with disdain” rather than “mock”.

        • Phil:

          I was just stunned by the statement, “In reality the mall shooting type event occurs about 10 times a year order of magnitude,” as if 10 times a year is ok. And then there’s school shootings and all those other things that keep happening. My “I can’t even” was a reaction to what seemed to me to be the commenter’s attitude that this was no big deal. I guess I was misinterpreting what that commenter was saying. But I wasn’t mocking or dismissing or expressing disdain, I was just kinda stunned. As I said, I guess this was a misinterpretation on my part.

        • Not that mall + school + workplace shootings, which happen typically 10-20 times a year are no big deal, just that those events are what people think of when they think “mass public shooting”, and redefining a “technical” definition of mass public shooting as “any event where 3 or more people are struck by bullets” which results in a count like 300-500 per year is specifically intended to mislead the public into believing that the horrific events of things like Uvalde are currently happening more than once per day.

          In reality there are order of magnitude about 10k gun homicides per year, and about 20k gun suicides per year (maybe a bit more of both in recent years but that order of magnitude). The number of deaths per year associated with what people actually think of as mass-public-shootings (Uvalde, etc) is on the order of 50-100 people. That’s bad, but it’s nowhere near as bad as the implicature of this intentional obfuscation which is that maybe 4000 people die in Uvalde type events per year (ie. 400 events avg ~10 people per event).

          In reality, almost all the violent gun deaths are still caused ultimately by the war on drugs, or by robbery, carjackings, and etc. The things that stop these deaths will be improvements in social conditions, income, housing opportunities, reduced Gini coefficient, eliminating the black market for drugs, decriminalization etc.

          On the other hand, the public shootings are well understood by researchers to be a form of suicide, and a particular form that is pretty well studied, and would best be addressed through suicide prevention, improved opportunities for typically young men, and taking the warning signs seriously as almost every single one of them has raving flashing light warning signs ahead of time.

          GVA does nothing to reduce societal risk by adopting this disingenuous tactic invented by an anonymous activist on Reddit. But I’m guessing they got plenty of money from Michael Bloomberg by doing it.

        • I think that claiming they “intentionally use this “purely statistical” method of classifying “mass shootings” because it hyperinflated the numbers” is (a) something you have no way of proving, and (b) kinda colours your response.

          There are many reasons to use purely statistical methods. Often the intent of the shooter isn’t definitively known, or isn’t reported. Sometimes it’s vague – like if a robber goes into a shop and takes hostages, and then gets mad and starts killing everyone, is that a mass shooting? And an intent based framework runs into issues like “suppose a guy goes into a school with the intent to kill everyone, but misses his shots and gets arrested”, does that count as a mass shooting? Logistically it gets a lot harder and fuzzier to calculate as well.

          Differences between the public image of the things and the academic/legal definition of things is hardly unique to this matter. Paedophilic crimes are rarely about shady people in alleys grabbing kids.

          Anyway, for Dale, the GVA has a “mass murder” number as well as a “mass shootings” one, with mass murder being lower.

        • Anon:

          You write, “the public shootings are well understood by researchers to be a form of suicide, and a particular form that is pretty well studied, and would best be addressed through suicide prevention, improved opportunities for typically young men, and taking the warning signs seriously as almost every single one of them has raving flashing light warning signs ahead of time.”

          Suicide is horrible, whether by men or by women, and I agree about taking warning signs seriously etc. Also it does not seem unreasonable to imagine a world in which, when young men do commit suicide, they don’t do it by shooting up a school.

    • Yes, I agree. If you go through to the article abstract, it goes:

      Conclusions. These results suggest that policies aimed at preventing mass shootings by focusing on serious mental illness, characterized by psychotic symptoms, may have limited impact. Policies such as those targeting firearm access, recreational drug use and alcohol misuse, legal history, and non-psychotic psychopathology might yield more substantial results.

      I think this is mostly a case of miscommunication.

  8. This quote sounds like a classic transposition of what we are assessing the probability of and what we are conditioning on.

    ‘“the prevalence of psychotic symptoms among mass murderers is much higher than that in the general population (11% v. approximately 0.3-1%).” That is, people with psychotic symptoms were between 10 and 30 times more likely to commit mass murder than people without psychotic symptoms.’

    • Leaving aside sampling variability – and the accuracy of those prevalence numbers – is there really a problem there?

      Let’s imagine that people with psychotic symptoms were as likely to commit mass murder as people without psychotic symptoms.

      Wouldn’t the prevalence of psychotic symptoms among mass murderers match that in the general population?

      • Yes, in the very special case that those with symptoms were exactly as likely as those without to commit mass murders, the prevalence of symptoms amongst mass murderers would equal that of the general population.

        HOWEVER, by Bayes’ law, it’s generally NOT the case that if symptoms are 10-30 as likely amongst murderers than amongst the general population, then those with symptoms are 10-30 times as likely to be murderers. They might be only 0.00001% more likely or something like that to be murderers. That’s because the vast majority of people are not murderers. Formally, it’s because the prior enters Bayes’ law.

        • As I just wrote in another comment

          p(M|P)/p(M|notP) = p(P|M)/p(notP|M) p(notP)/p(P) = p(P|M)/p(P) (1-p(P))/(1-p(P|M))

          The ratio depends on the prevalences among mass murderers and in the general population only. When the prevalences are low the second factor is close to one and the ratio is just the ratio of the prevalences. When the prevalences are not low – and the prevalence among mass murderers is higher than in the general population – the second factor will be higher than one.

          I’m not sure what do you mean by “they might be only 0.00001% more likely or something like that to be murderers” because you seem to say that in opposition to “10-30 times as likely”.

          Say the probability is 0.000011% for those with symptoms and 0.000001% for those without symptoms. Both of the following statements would be true:

          – those with symptoms are 11 times as likely to be mass murderers

          – those with symptoms are 0.00001% more likely to be mass murderers

      • If only there were some sort of mathematical or statistical relationship that allowed us to relate the probability of (A, given B) to the probability of (B, given A). That would sure be an important mathematical tool that could be applied to this sort of question!

  9. If journalists consciously (or unconsciously) allow their prior beliefs to colour their interpretation of scientific studies, how much more does this apply to domestic and international politics?

    In fact, we need to be much more vigilant when it comes to claims like China’s human rights abuses. These false narratives create conditions for conflict down the road (“manufacturing consent”).

    Yet many academics seem to just take Western reporting on Ukraine, Russia at face value.

  10. Even if the relative risk of violence presented by those with severe mental illness is high compared with those who don’t have such diagnoses, targeting severe mental illness may not make a large difference in mass shooting or violence rates because severe mental illness is relatively rare. Thus, what epidemiologists call the population attributable risk of severe mental illness may be fairly low. Here is what the RAND description of the link between mental illness and violence says:

    “There is also evidence that people with mental illnesses commit certain types of violent offenses at higher rates than those without mental health conditions do, at least for specific diagnoses. For example, Fazel et al. (2009) performed a meta-analysis and found that the risk of homicide among those with schizophrenia was 0.3 percent relative to 0.02 percent among those without a diagnosed mental illness. However, the risk of homicide among those with substance abuse was also 0.3 percent, implying that individuals with either diagnosis (schizophrenia or substance abuse) had a higher risk of committing homicide than did those without a mental illness diagnosis. Although the risk of committing homicide is elevated among those with schizophrenia, studies that have examined the percentage of violent crimes committed by people with evidence of schizophrenia or related psychoses found that these individuals account for less than 10 percent of violent crimes (Fazel et al., 2009), a finding largely explained by the relatively low prevalence (less than 1 percent) of these conditions among the general population (Kessler et al., 2005; Wu et al., 2006).”

    From: https://www.rand.org/research/gun-policy/analysis/essays/mental-illness-risk-factor-for-gun-violence.html

  11. Whether individual psychotic people are more likely to commit mass murder than individual non-psychotic people seems a distraction to the most pertinent descriptive statistic: 89% of mass murders are committed by people who are not psychotic, so focusing public policy on the small fraction who are psychotic—even though they do appear to be much more likely as individuals to commit mass murder—would accomplish little and would avoid dealing with the largest part of our mass-murder problem.

    And to the point in the original post: This is where communicating more clearly would be helpful. Instead of introducing the conditional probability language into this at all, the focus should be on the fact that the vast majority of mass murders are committed by people who don’t have severe mental illness, so effectively addressing the problem requires addressing those mass murders.

    • I’m still looking for some more clarity on what percent of mass murders are committed by people who are psychotic – it isn’t clear to me how many gang-related mass murders there are. Are they part of the 1595 “sponsored” murders which were excluded, or are they an unknown number of murders that were not included in the total 14,785 “potential” mass murders? If the former, then I still find the total numbers somewhat puzzling as they seem low to me.

    • When you say “people who are not psychotic”, doesn’t that operationalize as “people with no recorded diagnosis of psychosis”?

      If so, that’s a very important distinction.

      • I would hope that if they used a recorded diagnosis of psychosis they would use “prior diagnosis of psychosis” or “history of psychosis” rather than “psychotic symptoms,” which to me implied symptoms (regardless of diagnosis) present around the time of the shooting. (Psychotic symptoms = delusions, which are fixed, false beliefs that are culturally “inappropriate” — so QAnon doesn’t count — and/or hallucinations, which are perceptions of something that is not present.) Many people are psychotic at just one time in their life or for brief periods, which could be from mental illness or other medical illness, whereas people with schizophrenia may have psychosis that is more difficult to control. So using “psychotic symptoms” to mean a recorded diagnosis of psychosis would be misleading, in my opinion.

  12. This is a lot like discussion about domestic abusers being more likely to be from families where there was domestic abuse. Well, maybe they have high risk, but that doesn’t mean they are most or even a large proportion of domestic abusers. People often misinterpret that to mean that people who come from families with domestic violence are highly likely to be violent themselves. It would be such a nice and neat story if it were true, but the misinterpretation of relative risk leads to the stigmatization of victims.

Leave a Reply

Your email address will not be published. Required fields are marked *