The seventy two percent solution (to police violence)

And now it is your turn,
We are tired of praying, and marching, and thinking, and learning —  Gil Scott-Heron

So. It turns out that Gil Scott-Heron was right and he was wrong. We once again, during a time of serious social inequality and political upheaval, sent whiteys to the moon (ish). On the other hand, the revolution is definitely being televised. Live.

And as the protests and discussions continue there are a lot of reform ideas being mooted. Some ideas are transformative, like defunding and demilitarizing the police (this does not mean “get rid of police”! see here and here for what that might mean) and redistributing parts budget to housing, community projects, health, employment, education and other vital social programs. (This is predicated on the idea that the police are not the right solution to every problem they’ve been hurled at.)

At the other end of the spectrum we have 8 Simple Rules for Dating My Teenage Daughter to reduce police killings by 72%.

These rules are:

  • Require de-escalation
  • Have a use-of-force continuum
  • Ban chokeholds and strangleholds
  • Warn before shooting
  • Don’t shoot at moving vehicles
  • Exhaust all other means before shooting
  • Other officers have a duty to intervene to stop another office from using excessive force
  • Require comprehensive reporting of uses and threats of force.

And really, all of those are perfectly reasonable things. The question is do they do anything? (NB: Eric Garner died from a chokehold in 2014. They were banned in NY in 1993.)

Well the reason why I’m talking about this at all is that these rules were heavily being promoted as being “data driven”. (I am not even going to pretend that I’m well versed in police violence or structural racism in the US. I have a very well-defined lane to stay in.)

In fact, the interventions were sold with the tag-line “Data proves that together these eight policies can decrease police violence by 72 percent”. (See, e.g. here). Very recently, project has walked this back, somewhat, to “Research shows more restrictive use of force policies can reduce killings by police and save lives”.

If you look into their docs, you get this

Moreover, our research found that having all eight of these use of force restrictions in place was associated with 72% fewer police-involved killings compared to departments with none of these policies in place and a 54% reduction for the average police department.

This research was part of the Police Use of Force project, which looked at police killings linked to 100-ish large police forces and compared which of these 8 policies had been implemented. (Why these 8? No idea.) The data is not open, but on Twitter, Emma Glennon, who is an epi researcher and PhD student at Cambridge has compiled some similar data and released it on github. She wrote a really great twitter thread that dives into the data and the analysis and that I’m copping  a lot from here. (Eve Ewing also has a really great discussion about this.)

So where did that 72% come from? Well, if you read the report (which doesn’t appear to be peer reviewed, but who even knows) you see the following:

  • 100 departments were approached for their use of force policies, leading to 91 departments that could be included.
  • Deaths were taken from The Guardian’s The Counted database, which covers 2015-2016. (A more up-to-date database is available from the Fatal Encounters project.)
  • The policies were included as a total number of policies enacted
  • The model was fit using a negative binomial regression that also included (with regression coefficients and standard errors)
    • Percent minority, 1.7 (0.7)
    • Arrests, 0.9 (0.2)
    • Income, 0.8 (0.5)
    • Inequality (Gini coefficient from the census), 0.1 (1.7)
    • Assault on officers (~0 effect) and number of officers (~0 effect)
  • The regression coefficient for Number of Policies was -0.16 (0.07).

Now, there is no way that this analysis supports causal conclusions. It’s not built for it and it’s not analyzed that way. And that’s ok. Exploratory data analysis is great. We all love it.

(Let’s not be glib about this though. That policy data was hard got and has value beyond this particular analysis. As always the hardest part of doing a data-driven analysis is getting the damn data.)

But the problem comes when making the 72% claim. That is a claim about the effect of a hypothetical intervention. Which is to say that it’s a causal claim

And it’s not a good one.

The argument is that if $latex k$ policies are implemented, then the reduction in police killings compared to similar city with no policies enacted is

$latex \left[\exp(-0.16k) -\exp(-0.16*0)\right]\times 100\%$.

This gives the 72% reduction. (Yes Andrew, that extra precision annoys me too.)

An average city has 3 of these 8 policies enacted, so the change would be<

$latex \left[\exp(-0.16k) -\exp(-0.16*3)\right]\times 100\%$,

which is a 34% change.

Note: The 54% quoted above is wrong. You get the 54% quoted above if a department that previously had zero policies enacted (in the data Irving, Kansas City, Reno, and Stockton) enacts 3 policies.

So. Are these numbers in any way real? Well, cracking open my copy of Regression and Other Stories, I know that this will be ok if the model holds (there is no model checking in the report) and under some unlikely assumptions that basically mean that observations with the same covariates are randomized into each of the 9 treatment levels (from 0 to 8 policies).

The numbers are also assuming that if you enact 4 policies, it doesn’t matter which 4 policies you enact. In the data there are 49 different policy combinations. And it is not unreasonable to expect that some combinations will be more effective than others.

All of this is to say that it is difficult to use this data and this modelling to predict the effect of enacting these policies. It’s definitely wrong to say they will lead to a 72% or 34% (or 54%!) reduction.

The correct interpretation is that (modulo fit issues), of the departments studied, those that enacted more of these policies recorded fewer killings (even after accounting for income, inequality, racial makeup of the population, and the number of arrests).

Does this mean the policy proposal should be scrapped? Well. That’s a hard question. It’s similar to the question of if COPSS should rename the Fisher Lecture after someone who wasn’t a notorious racist. (There is a petition) Sure. Do it. But beware of opportunity cost, because there is no evidence that this intervention will even touch the sides of the actual problem. So if enacting these policies is the end of the story, we likely won’t get very far.

And, as always, we should probably be aware that simple interventions rarely solve complex problems. And we should always be fairly skeptical when a relatively small intervention is advertised as leading to a large change. Even (or especially) what that intervention is advertised as “data driven”.

99 thoughts on “The seventy two percent solution (to police violence)

    • Straight from wiki:
      “In 1950, Fisher opposed UNESCO’s The Race Question, believing that evidence and everyday experience showed that human groups differ profoundly “in their innate capacity for intellectual and emotional development” and concluded that the “practical international problem is that of learning to share the resources of this planet amicably with persons of materially different nature”, and that “this problem is being obscured by entirely well-intentioned efforts to minimize the real differences that exist”. “

      • Harvard professor David Reich in the New York Times:

        But as a geneticist I also know that it is simply no longer possible to ignore average genetic differences among “races.”

        Recent genetic studies have demonstrated differences across populations not just in the genetic determinants of simple traits such as skin color, but also in more complex traits like bodily dimensions and susceptibility to diseases.

        I am worried that well-meaning people who deny the possibility of substantial biological differences among human populations are digging themselves into an indefensible position, one that will not survive the onslaught of science.

        These are the sorts of “well-meaning people” behind the COPSS petition.

        While most people will agree that finding a genetic explanation for an elevated rate of disease is important, they often draw the line there. Finding genetic influences on a propensity for disease is one thing, they argue, but looking for such influences on behavior and cognition is another.

        And since all traits influenced by genetics are expected to differ across populations (because the frequencies of genetic variations are rarely exactly the same across populations), the genetic influences on behavior and cognition will differ across populations, too.

        So how should we prepare for the likelihood that in the coming years, genetic studies will show that many traits are influenced by genetic variations, and that these traits will differ on average across human populations? It will be impossible — indeed, anti-scientific, foolish and absurd — to deny those differences.

        Reich’s opinion is indistinguishable from Fisher’s. Is Reich a “notorious racist” also?

        • Dan: You seem to be dodging the question. Reich is a leading, if not the leading population geneticist of his generation. His views are, on this topic, indistinguishable from Fisher’s, as I have demonstrated above.

          Are those of us, like me, who agree with Reich, also notorious racists?

        • I do genetics research and this is the first I have heard of Reich as _the_ single leading population geneticist of his generation. Nothing against his distinguished track record in genetics but there’s zero evidence for this claim.

          David Reich’s New York Times op-ed is widely regarded as committing serious errors by geneticists. Here’s a good summary:

          https://www.theatlantic.com/science/archive/2018/04/reich-genetics-racism/558818/

          I imagine David Reich would be happy to distinguish his views from Fisher for you and would not at all appreciate being put into the same category as a notorious racist such as Fisher. After all he’s been on record at other times as opposing other interpretations of genetics research that could support racism:

          https://cehg.stanford.edu/letter-from-population-geneticists

          As for you, it is impossible for you to be a “notorious” racist when almost no one knows who you are. If you realize that you have the same views as a notorious racist, I would see that as an opportunity for reflection on your own views. That you believe something certainly doesn’t provide any evidence, outside the cognitive dissonance in your own mind, that someone else who believes the same is not a racist.

        • That link doesn’t seem to be pointing to “serious errors” as much as semantical sloppiness in relating the nebulous notion of “race” with genetics:

          “But his op-ed starts losing clarity when, thanks to some unfortunate language, the distinct concepts of “races” and “populations” seem to become admixed themselves. (…) He’s apparently trying to defend the use of both, but in the process somewhat blurs his earlier distinctions between race and ancestry. (…) Readers can easily miss all this, especially if Reich’s words are excerpted or twisted to another author’s own ends. (…) Reich’s op-ed includes not just vague words, but vague rhetorical logic. (…) This argument, fleshed out with examples in Reich’s book, is that truculent and overly PC anthropologists, unobstructed by timid geneticists, are suppressing discussion of genetic variation. (…) In the days after the op-ed appeared, there were several rebuttals from fields outside genetics. In some cases, the corrective reactions of geneticists to these (admittedly sometimes flawed) rebuttals seemed swifter, noisier, and more vigorous than the corrective reactions of geneticists to Reich’s op-ed itself. Reich, too, published a follow-up in the Times, in which he clarified some of the language, but reiterated the argument against timid geneticists.”

        • The usual dictionary definitions seem fine to me:

          https://en.wiktionary.org/wiki/racism: Belief that there are distinct human races with inherent differences which determine their abilities, and generally that some are superior and others inferior.

          https://www.merriam-webster.com/dictionary/racism: A belief that race is the primary determinant of human traits and capacities and that racial differences produce an inherent superiority of a particular race.

          https://www.dictionary.com/browse/racism: A belief or doctrine that inherent differences among the various human racial groups determine cultural or individual achievement, usually involving the idea that one’s own race is superior and has the right to dominate others or that a particular racial group is inferior to the others.

          I see no evidence that Fisher believed that the white race, or the English race, was “superior” to all others, much less that it had “the right to dominate others.” Indeed, like most Darwinians, he almost certainly believed that general notions of superiority, outside a specific environment, were meaningless.

        • D Kane –

          > I see no evidence that Fisher believed that the white race, or the English race, was “superior” to all others, much less that it had “the right to dominate others.” Indeed, like most Darwinians, he almost certainly believed that general notions of superiority, outside a specific environment, were meaningless.

          I’m actually more curious about your views. You seem to believe that the white race is superior to certain other races by virtue of a genetic predisposition to be more intelligent. If that’s true, I would assume that if you met two random people on the street, one black and one white, you would say that on average the white person is superior in intelligence, in ways that can be clearly measured.

          Is that a fair characterization of your view?

        • >I would assume that if you met two random people on the street, one black and one white, you would say that on average the white person is superior in intelligence, in ways that can be clearly measured.

          >Is that a fair characterization of your view?

          That would fairly characterize the view of any honest man. But the Western world is experiencing a crisis of integrity, and to speak such truth openly is a career ending event.

        • Nick –

          > That would fairly characterize the view of any honest man. But the Western world is experiencing a crisis of integrity, and to speak such truth openly is a career ending event.

          Well, I guess I’m not honest.

          But briefly, here are the reasons why that wouldn’t be my view.

          The first is that I don’t have any confidence that we can measure intelligence effectively. The second is because people who believe that intelligence – as a genetic attribute – can be measured effectively, also believe it has a causal relationship with societal outcomes. I don’t think that causality has been established.

          But regardless, can I assume, then, that you proudly accept the label of racist – as your open racism proves your honesty?

        • reich is a dude at harvard with a lot of funding who has attracted a big lab of smart people to implement stuff that he wrote grant proposals for, while he flies around giving talks. he’s used that effectively to do some splashy work that will get his name in textbooks. he’s good at that, and science probably needs people like that.

          he’s also done an elegant (hopefully unintentional) job of getting positive-but-deniable buzz from racists and crypto-racists by making incoherent public statements like the one you quote from. if it is indeed unintentional, it’s a really beautiful instance of how scientific caution combined with poor communications skills can be damaging. i’ve run across multiple creepy people on the internet making strange racist appeals to authority because “david reich said [something with a bit of a smell to it]”.

          to the extent that his statements reflect fisher’s on human variation, they are racist and well outside the mainstream of human population genetics. to the extent that he’s just bad at writing words for public consumption, well, he should use some of that NIH funding to hire an editor. i think what michael hoffman has already brought forward illustrates the facts of that particular article well enough.

          i don’t know which of those is the case, and it’s not really worth parsing a poorly-written 2 year old op-ed too hard IMO.

        • Reich’s view is completely different from Fisher’s.

          Reich’s view is that there inevitably will be genetic differences on average between any subpopulations you circumscribe and that he’s worried people will deny them.

          Fisher’s view is that different races of human as defined in his era certainly have large intrinsic differences in intelligence, emotional development, and pro social behavior, that the races are of a fundamentally different nature, and that part of the solution to the problem of coexistence between what are according him almost distinct subspecies is eugenics.

          If your view is functionally identical to Fisher’s, then you are a racist, just not particularly notorious.

        • > Fisher’s view is that different races of human as defined in his era certainly have large intrinsic differences in intelligence, emotional development, and pro social behavior,

          Correct. And that is Reich’s view. Or at least Reich’s view is that we need to be prepared for large differences.

          > the races are of a fundamentally different nature

          Fisher does not write that.

          > Part of the solution to the problem of coexistence between what are according him almost distinct subspecies is eugenics

          Fisher does not write that. (Fisher did have opinions on eugenics, but these are nowhere mentioned in work we are discussing here.)

        • > Fisher did have opinions on eugenics, but these are nowhere mentioned in work we are discussing here

          But these things are relevant to the start of this thread which was:

          > Ronald Fisher was not a “notorious racist.” To suggest otherwise, without evidence, is quite sleazy.

          A wall of text from the Fisher Wikipedia entry:

          “The last third of The Genetical Theory of Natural Selection [a book written by Fisher] focused on eugenics, attributing the fall of civilizations to the fertility of their upper classes being diminished, and used British 1911 census data to show an inverse relationship between fertility and social class.”

          That’s a bad look.

        • > That’s a bad look.

          Is “look” how you judge truth? I don’t know this literature. Are you claiming that Fisher is lying about the relationship between fertility and social class in Britain in 1911? Or that me miscalculated the relationship?

        • The issue is what is being causally inferred and projected into the future — not whether the statistics are accurate. The implication is the social class is genetic and I would bet it’s difficult to find a discussion of the most plausible and obvious alternative, that of a new group of people at the top of the economic and social ladders. Imagine that.

        • This is the issue we deal with in all sorts of fields where people take pride in and desire to be “data driven”. Where there is a belief that the data speak for themselves. Where the notion of a model and causal reasoning are rarely acknowledged as important and quickly dismissed and justified by “prediction is what matters, anyway”. It isn’t what matters. This belief that prediction is what matters when talking about human beings is fundamental problem in applied social science and one that helps to reify and justify racially biased algorithms.

          It is no different than accepting the argument that ESP is real because a statistically significant study was published in a peer review journal.

        • > Is “look” how you judge truth?

          Why do you keep moving the bar on these arguments? I don’t see evidence that you want to listen to what other people are saying.

          > I don’t know this literature.

          Me either! That’s why I went to Wikipedia to read about Fisher to evaluate the notorious racist thing. Turns out Wikipedia says he wrote a 1/3rd of a book on eugenics which seems like a pretty bad sign. If you want to investigate that, go investigate that. Or if you want to contest the characterization of that, go argue with the historians. Wikipedia is free to edit, etc. I’m just doing the Googling for you.

        • > And that is Reich’s view.

          No, it is not. Did you even read the article you linked to?

          Reich says

          > One of Mr. Wade’s key sources, for example, is the anthropologist Henry Harpending, who has asserted that people of sub-Saharan African ancestry have no propensity to work when they don’t have to because, he claims, they did not go through the type of natural selection for hard work in the last thousands of years that some Eurasians did. There is simply no scientific evidence to support this statement. Indeed, as 139 geneticists (including myself) pointed out in a letter to The New York Times about Mr. Wade’s book, there is no genetic evidence to back up any of the racist stereotypes he promotes.

          and

          > What makes Dr. Watson’s and Mr. Wade’s statements so insidious is that they start with the accurate observation that many academics are implausibly denying the possibility of average genetic differences among human populations, and then end with a claim — backed by no evidence — that they know what those differences are and that they correspond to racist stereotypes.

          > If scientists can be confident of anything, it is that whatever we currently believe about the genetic nature of differences among populations is most likely wrong. For example, my laboratory discovered in 2016, based on our sequencing of ancient human genomes, that “whites” are not derived from a population that existed from time immemorial, as some people believe. Instead, “whites” represent a mixture of four ancient populations that lived 10,000 years ago and were each as different from one another as Europeans and East Asians are today.

          This is nearly the opposite of

          > different races of human as defined in his era certainly have large intrinsic differences in intelligence, emotional development, and pro social behavior,

          You claim on two counts

          > Fisher does not write that.

          Yes, he writes both of those things. He writes on the necessity of a “ruling race” for the founding and maintenance of a prosperous civilization, of a “paucity of the necessary types of ability from the indigenous population”, and the necessity of establishing a deliberate eugenics policyto prevent the collapse of civilization due to negative selection pressures. He phrases this in no uncertain terms with explicit reference to “inferior races.” He was no Hitler making explicit claims about the Jewish people being an inferior race, or any specific racial hierarchies which are supposed to be stable over all time. But at the time he wrote it and made eugenical proposals, the ruling races were the white northern europeans and the lower races were colored peoples, and were his policies adopted the effect would have been to promote the fertility of white people and limit that of colored people to maintain “the type of a ruling race.”

          > Fisher did have opinions on eugenics

          What an incredible understatement! Fisher was FANATICAL about eugenics, and was unambiguous about his belief that it was necessary to prevent the collapse of civilization.

          > but these are nowhere mentioned in work we are discussing here

          Irrelevant. The point under contention is not, “is this particular statement by Ronald Fisher racist.” It’s, in your own words

          > Ronald Fisher was not a “notorious racist.”

          It seems, based on your frankly dazzling understatement of Fisher merely having “had opinions on eugenics”, that really what this boils down to is that you’d prefer to ignore this part of him. Well, here it is anyways.

          > In the problem of the decay of ruling classes it is shown that neither race-mixture, nor the selective action of climate and disease, would suffice to explain their failure under favorable selection. The causes to which we have traced the inversion of fertility must have been operative in the most ancient civilizations, as in our own, and serve to explain the historical importance of ruling races, through the absence of the proper attributes in the native populations. The same causes ensure an adverse selection acting upon each conquering people in turn. The decline of barbarian peoples, which have received the civilization, and shared the decay, of some more advanced state, without suffering from a foreign climate or from intermixture with an inferior race, is more intelligible by the social promotion and extinction of their more capable members.

          > The fact of the decline of past civilizations is the most patent in history, and since brilliant periods have frequently been inaugurated, in the great centres of civilization, by the invasion of alien rulers, it must be recognized that the immediate cause of decay must be the degeneration or depletion of the ruling class.”

          > The composition of existing populations, graded in both social ability and in effective infertility, presents special, and much graver difficulties, which only a people capable of deliberate intentional policy could hope to overcome.

        • The relevant citation is here. Thanks for pointing it out! I need to read more Fisher.

          Andrew: To the extent that you think this is a +1 comment, we probably need a new thread, as we are now so nested that I don’t think further replies are possible. I would certainly be happy to go through the above complaints item by item.

        • FWIW, I would say that if Reich’s opinion was truly indistinguishable from Fishers then Reich would be very strongly racist as well. But in, mildly, Reich’s defense, Reich does not go remotely as far as Fisher. The claim of racial difference “in their innate capacity for intellectual and emotional development” adopts a linear racial hierarchy of “development”. Reich speaks much more neutrally in terms of varying influences.

          Further while Reich talks about “substantial differences in averages”, Fisher goes for “profound” differences, to the extent that he argues that it makes some people be of “materially different nature”, to the extent that these differences should have an effect on international relations and the allocation of resources. That’s very close to nazi talk.

        • > The claim of racial difference “in their innate capacity for intellectual and emotional development” adopts a linear racial hierarchy of “development”.

          No, it doesn’t. If Fisher believed that there was a linear racial hierarchy, then he would have written that. If you think he believes it, then provide some evidence.

          > Reich talks about “substantial differences in averages”, Fisher goes for “profound” differences

          True. Is that the distinction that you (and Dan Simpson) want to rely on? That using the word “profound” means that you are a racist while writing “substantial differences in averages” is A-OK?

        • > No, it doesn’t. If Fisher believed that there was a linear racial hierarchy, then he would have written that. If you think he believes it, then provide some evidence.

          You don’t think there’s an implicit value judgement in saying different people have different “capacities for development”?

          > True. Is that the distinction that you (and Dan Simpson) want to rely on? That using the word “profound” means that you are a racist while writing “substantial differences in averages” is A-OK?

          Well more than that, Reich warns against denying the *possibility* for substantial differences that might be uncovered by future research. Meanwhile Fisher claims those profound differences exist using the prior evidence of 1950, no more research required. But even without that distinction, talking about possible substantial differences in averages is much weaker, because Fisher’s profound difference is a statement about policy-relevant effect sizes. Reich limits himself merely to the idea that such differences might be detectable at some point in the future.

        • > because Fisher’s profound difference is a statement about policy-relevant effect sizes

          Do you really want to hang your argument on Fisher’s use of the word profound? I will just quote Reich again:

          So how should we prepare for the likelihood that in the coming years, genetic studies will show that many traits are influenced by genetic variations, and that these traits will differ on average across human populations? It will be impossible — indeed, anti-scientific, foolish and absurd — to deny those differences.

          For me, a natural response to the challenge is to learn from the example of the biological differences that exist between males and females. The differences between the sexes are far more profound than those that exist among human populations, reflecting more than 100 million years of evolution and adaptation. Males and females differ by huge tracts of genetic material — a Y chromosome that males have and that females don’t, and a second X chromosome that females have and males don’t.

          Most everyone accepts that the biological differences between males and females are profound. In addition to anatomical differences, men and women exhibit average differences in size and physical strength. (There are also average differences in temperament and behavior, though there are important unresolved questions about the extent to which these differences are influenced by social expectations and upbringing.)

          How do we accommodate the biological differences between men and women? I think the answer is obvious: We should both recognize that genetic differences between males and females exist and we should accord each sex the same freedoms and opportunities regardless of those differences.

          It is clear from the inequities that persist between women and men in our society that fulfilling these aspirations in practice is a challenge. Yet conceptually it is straightforward. And if this is the case with men and women, then it is surely the case with whatever differences we may find among human populations, the great majority of which will be far less profound.

          Fisher would certainly agree that “the great majority” of genetic differences between human populations will be less “profound” than those between men and women. He and Reich are on the same page. But Reich is warning you that some of them won’t be. Is Reich a racist too?

        • I don’t hang my argument on the “use of the word profound”, I hang my argument on the overall thrust of the argument. If you are arguing that Reich believes like Fisher that there are differences between racial groups in terms of their intellectual capacity that means they cannot be treated equally, that they are persons of materially different nature, and deserve different amounts of resources, then yes, he is racist.

          Reich used different words from Fisher and I took that to mean that he meant something different.

        • There is the possibility of substantial truth in your statements, but what you are saying seems profoundly misguided. You are attempting, at best, an indirect argument from authority.

        • D Kane: Perhaps I am missing something. In what way is a genetic disease comparable to a social construct measured crudely?

        • Yes and some. “Intelligence”, “behaviour” are social constructs with implicit biased metrics. Read Steven Jay Gould (even) about history of IQ tests (Mismeasure of Man) ++

        • Respectfully you are you are equivocating. First statement by Fisher says races differ in “…innate capacity for intellectual and emotional development”.
          Is this the same as, “genetic differences exist across race”?

          Nope.

          Please think about it.

      • Sorry textbook def of racism. We can talk about sep the man from the science but changing the award is trivial and sends an important message. Awards should be about celebration and progress! Sign it

    • > What is the evidence that opportunity costs actually exist in this context though?

      Yah. I had copied the same aspect of the article and I was about to write essentially the exact same comment.

      Not to differ with the analysis of the analysis, but this may be a situation where you don’t want to wait for perfect evidence and you certainly shouldn’t *assume* opportunity cost absent a specific proposal for what would work with less cost.

      Not to say that the potential for opportunity cost should just be dismissed, but it seems reasonable to me that requiring de-escalation, having a use-of-force continuum, and having transparent access to histories of complaints lodged should be rather no-brainers in terms of benefit absent clearly demonstrated evidence of realizable opportunity costs.

      So beware opportunity cost? Sure. No doubt. But at some point you may want to make some change even if you can’t prove that none will manifest.

      • This is a good point. I was thinking along the lines of enacting ancillary reforms (which are good but aren’t touching the core of the problem) and, having enacted them, doing nothing else.

        • Dan –

          > This is a good point. I was thinking along the lines of enacting ancillary reforms (which are good but aren’t touching the core of the problem) and, having enacted them, doing nothing else.

          Sure. But on the other hand any reforms may have to be viewed in the context of what is realistically feasible. If these reforms are politically achievable, then they might net better results than going after reforms that might meet stronger political opposition like decriminalizing drugs.

          Of course, that also is a difficult analysis to make. Many advocates for more far-reaching reforms make a solid point that historically sometimes you get more in the end by pursuing reach goals than by going after less radical measures.

        • “Sure. But on the other hand any reforms may have to be viewed in the context of what is realistically feasible.”

          Two weeks ago I would have had opinions about what is realistically feasible; times have changed, and I don’t pretend to understand how much.

        • The most significant opportunity cost, I would conjecture, is actually in the time and energy and political cohesion spent in the debate itself – and thus not implementing these milder reforms will keep these issues in future debates as a counterargument to any more radical change. “Why do X when we haven’t tried Y and Z” is a good way to gum things up so that none of X, Y or Z happen.

          I think the failure of milder reforms will in themselves create the impetus for further action.

        • Dan Simpson

          “I was thinking along the lines of enacting ancillary reforms (which are good but aren’t touching the core of the problem) and, having enacted them, doing nothing else.”

          My interpretation is that the eight rules treat the symptom, not the disease, and if we implement them and police killings of african americans is no longer an issue (let’s face it, it’s not currently an issue for white americans), we’ve not addressed the underlying racism.

          Therefore, as you’re saying, we may have lost the opportunity to tackle the underlying problem: racism.

        • The problem is that any attempt to tackle underlying racism is necessarily going to be very slow to bear fruit.

      • The opportunity cost is in other “not-passed” policies.

        For instance, much reporting seems to indicate that abuses are “predictable”, in the sense that cops who use force aggressively, have also done so a lot in the past. This indicates room for a policy which is “fire the worst 1% each year”, or “do hotspot policing — of the police”. By forcing turnover — these policies induce change *in worlds where it is possible* by getting rid of bad actors. In worlds where change isn’t possible i.e. context not individuals is driving action, nothing has an effect. But the policies listed in 8can’twait, don’t guarantee success in *any* worlds, because they are likely driven by strong selection effects.

  1. Require de-escalation
    Have a use-of-force continuum
    Ban chokeholds and strangleholds
    Warn before shooting
    Don’t shoot at moving vehicles
    Exhaust all other means before shooting
    Other officers have a duty to intervene to stop another office from using excessive force
    Require comprehensive reporting of uses and threats of force.

    How about get rid of the reason for the cast majority of non-violent crime: ineffective (even anti-productive) prohibition of drugs

    There is much more to be done but I would start there. Give law enforcement less reason to interact with those behaving non-aggressively to begin with. Unfortunately, I suspect the end result of all this is going to be in the opposite direction though. Apparently it is defund the police then there will be some “federal police force” that comes in to save the day later?

    • I tend to think that decriminalizing drugs, demilitarizing the police, and redirecting the money that’s saved into community programs, health, and education would do a lot. (That is some version of the “Defund” platform)

      • Or cut that stuff then send everyone a check for the savings. If community programs, health, and education is what they want, then thats what theyll spend it on.

        • These programs only really work if they’re paid for at some sort of municipal or local level. Individuals would struggle to do that. Same reason all the boring stuff (roads, schools, hospitals, post office) need to be government run or else they lose quality, service, and/or accessibility.

        • Handing the savings back for individuals to spend how they please utterly neglects the purchasing power of the state when it comes to building things like hospitals, schools and community programmes. It’s a libertarian furphy that individuals can just buy a walkable street, or spend a few hundred dollars to buy into a new school for their community. This mantra of “cut spending and let tax payers choose what to fund” commodities basic services and basically tells poor people that they clearly don’t value their kid’s education enough if they’re not choosing to send them to a private school after receiving a small tax cut.

        • I used to think similar until I worked at a federal government institution and saw incentivized waste and inefficiency I wouldnt have believed otherwise. Im sure any large enough organization will be similar, which is why I now favor decentralization.

          Maybe a more local government could do it though since the people getting the service have more control.

        • Anoneuoid said,
          “since the people getting the service have more control”
          This is dubious — local governments typically “represent” a widely diverse population, so in many cases (e.g., if they are a minority of voters), the people needing the services don’t have more control.

        • This is dubious — local governments typically “represent” a widely diverse population, so in many cases (e.g., if they are a minority of voters), the people needing the services don’t have more control.

          I don’t see how that would be better than having governments that represent an even larger and more diverse population run it. Also, then it sounds like the government isn’t local enough. A large city like NYC or LA is also probably too big for the citizens to control.

        • Even an inefficient government is better placed to build infrastructure than a loose collection of individuals. There are some good examples of community managed health and education out there but a community organising to build and staff a school takes a long time.

          Keep in mind that the US federal government doesn’t have the responsibility to provide health, education or community safety services; those are the responsibilities of states or cities. And a lot of the USA’s political problems are tied to the election system being absolute garbage; how do you hold politicians to account under a voting system as broken as FPTP with electoral boundaries not drawn by an independent body?

  2. Dan –

    > And really, all of those are perfectly reasonable things. The question is do they do anything? (NB: Eric Garner died from a chokehold in 2014. They were banned in NY in 1993.)

    There have been a few articles recently about the really prevalent use of choke holds Minny over the past decade? or so.

    It seems that rather than being banned, they are an accepted part of policing practice there. As such, I think that your point about Eric Gardner dying from a choke hold despite the ban in New York needs some further consideration. The existence of one case where a choke hold killed someone in police hands does not imply that a ban has zero benefit.

    • A policy can be effective even when it isn’t always followed. But the aftermath, including that it took 5 years for him to be fired, speaks to serious problems that are not addressed even when the policy is in place.

      • Dan –

        > But the aftermath, including that it took 5 years for him to be fired, speaks to serious problems that are not addressed even when the policy is in place.

        No doubt. There’s a huge problem there, given that “the left” is pushing for reforms and police unions are a huge obstacle to many of those reforms being enacted, or even to the enforcement of regulations already in place.

        Looks to me like a political quagmire (and so unlike simple issues like gun control and healthcare).

        I wonder if Jonathan Haidt and Tucker Carlson will start campaigning for greater viewpoint diversity in the police unions?

    • I guess I’m making a distinction between policies that should be in place and policies that will fix the problem. I suspect all 8 are more the former than the latter.

  3. This may be slightly off the immediate topic, but similar in the sense that it is about using data to think about social issues and possible reforms:

    Just before coming to this blog today, I looked at my email — which included discussions from the ASA Section on Statistics and Data Science Education on the topic “Incorporating race, social justice, inclusion in your classes”. I suspect that some readers of this blog may be interested in this topic. Some items in today’s discussion:

    “Stand Your Ground court cases (with link). Good example of unexpected results and Simpson’s paradox. As a bonus, nothing is statistically significant, so how to interpret these results?”

    “Flint lead data. Simple descriptive statistics and sampling design. Based heavily on the Significance article on the same topic.”

    Using student level from the institution where one teaches to study questions such as class pass rates by race and gender; then adding Pell Eligibility to the analysis; then bringing in Simpson’s paradox — resulting in a good class discussion of probability and conditional probability.

    Using a recent article “Racial Health Disparities and Covid-19 – Caution and Context” in the New England Journal of Medicine, discussing how well-intentioned statistical analysis can perpetuate harm, and approaches for more deliberate/careful analysis that align with the goal of reducing inequality.

    I recommend that anyone reading this who is interested in these and other similar discussions join the ASA Section on Statistics and Data Science Education, which will allow you to access the discussions above and more.

  4. “Policies” are the mystical incantations of social engineers and grandiose politicians.

    Just how do you get cops to obey those 8 Simple Rules ?
    (we can’t even get them to keep their body-cameras on)

    Core problem is that cops generally operate above-the-law compared to normal citizens — and thus feel much less constrained in their daily interactions with the public. Threats, intimidation, menacing, assault, etc are routine police activities … but would prompt criminal charges if practiced by a civilian.
    ‘Qualified Immunity’ formally puts cops above the law, and government prosecutors are generally extremely reluctant to bring any charges against cops.
    Everybody is Equal Under the Law, theoretically, but cops know that is untrue in the real world.

    Fine tuning department level police policies is naive fluff IMHO.

    • From https://reason.com/2018/01/22/nyc-police-union-to-limit-get-out-of-jai/

      New York City’s largest police union, the Patrolmen’s Benevolent Association, is limiting the number of “courtesy cards” it distributes to its membership, according to the New York Post…

      The courtesy cards—or “get out of jail free cards,” as the Post calls them—can be pretty valuable. While some Patrolmen’s Benevolent Associations insist the cards are just public relations tools, the New York union “encourages officers to avoid ticketing cardholders,”

    • Guarino1

      “‘Qualified Immunity’ formally puts cops above the law, and government prosecutors are generally extremely reluctant to bring any charges against cops.”

      Qualified immunity only applies to civil suits brought against individual government employees, and yes, it has been extended by court rulings to the point where it no longer resembles the modest immunity embodied in the law passed by Congress over a century ago.

      This is an important reason as to why you see civil suits brought against cities, etc rather than police officers themselves.

      The difficulty in getting criminal convictions against police officers is another issue, a very important one, but different.

  5. The question of whether anything actually ‘works’ runs deep here. For example, the underlying idea in enacting some number of positive policies is that defines a culture. The military has been famous for handing out cards with slogans on them: get behind the mission in attitude and action. But does that work? It may be motivational for some and annoying to others: who hasnt sneered at some company mandated training program?

    What if the police lag larger cultural change? And then if they lag, maybe they lag because if the police were radical toward an edge, one way or the other, you couldnt trust them politically.

    Cultural lag. You deal with that a lot with attitudes. One expression interests me now, almost as an irony: the areas of the largest unrest have been run by the Democratic Party for decades, meaning they’ve been in charge of fundings and allocations, and have voted for an enacted the police militarization which has occurred in all big cities. So, while it appears there’s a lag in attitudes within the Democratic Party by race, the politics of policing have clearly not been simple.

  6. Dan,
    These policy debates are difficult but I think there is unnecessary tension being built between #8cantwait and #abolition.

    The other side of the spectrum is “Law and Order” and tear gas, not policies that are more politically viable and have a chance at success.

    Furthermore, these policies could easily enter into a more coherent strategy that builds incremental progress and moves the overton window.

    • I don’t think they need to be in opposition to each other either. As I said, sure. Do these things. I take issue the campaign using wrong and misleading numbers to massively overstate the size of the effect these changes will have. I think that’s what pushes this from an “and” to an “or’. Because if you can get a 72% reduction that people will take that and go home. Every death is a tragedy, but a 72% reduction is nothing to sneeze at. In the centre of the poster was “Together, these 8 policies can decrease police violence by 72%”. Why would you do more?

  7. The rules you mention may be obvious, but they are not actually always appropriate. The one that is most obviously wrong is the idea of the continuum of force. There are many places where slow escalation of conflict is much *more* likely to result in harm. There have been multiple studies that have shown that in some circumstances a rapid deployment of overwhelming force will resort in less injury than slow escalation. The classic example is subjugation of a prisoner in a cell. Folk go in there with mulitple people, bring the inmate down quickly, and use one person to secure each limb. In doing so, the prisoner does not have the opportunity to actually engage the correction officers. A similar issue involves the use of TASER or similar devices. While TASER deployment early on is dramatic, it routinely results in lower rates of injury. The reason is that you don’t go through the stage of person to person fighting, with the use of batons, etc. which are *much* more likely to produce severe, and often life-lasting injury.

    The same thing is true with de-escalation. There are often times when the threat of injury to a victim is very likely unless action is taken immediately. This can be true, for instance, in certain hostage situations.

    • Prisons generally avoid cell extractions if they can as they are incredibly dangerous to the officers involved as injuries are common.

      Extractions are only used now if an inmate is engaging in self-harm or poses some other immediate threat. Otherwise, they’ll just keep talking to the inmate until they get tired & hungry and then they willingly comply with orders. If there’s one thing prisons have, that’s plenty of time. Plus they can cycle in officers to keep talking to the guy without officers getting tired.

      As for your TASER claim, lower rates of injury for whom? The officer? Sure, as they don’t have to get their hands dirty. The citizen? No, as there’s always an injury to someone being tased. I don’t understand why people think there’s no injury in tasing. The whole point of the weapon is to have metal prongs break the skin to deliver a incapacitating surge of electricity that disrupts muscle functioning.

      As for escalation/deescalation, there’s plenty of evidence that deescalation tactics works better. Does that mean that it works in every situation? Of course not, I want the police to drop the guy who’s a mass shooter. We can figure out what went wrong in his head later. But the guy talking to himself on the street corner? Yes, deescalate and get him to the necessary care provider.

  8. I would think the main thing missing from the list is actual accountability for actions which go against the other policies. There is no point imposing rules and regulations if the result of ignoring them is, essentially, nothing, unless enough people go out and protest on the street.

  9. Dan, Thank you for your blog post on statistical aspects of policing reform and racism in policing. It’s good to see this blog engage with specific, statistics-focused aspects of the issue, which (I hope) could be a way for the blog to be contributory while minimizing the occurrence of racist statements in the comments about less technical and more general topics.

    Another specific, statistics-focused topic that could be of interest is how to best communicate risks associated with the US criminal justice system.

    In public health and medical research, there is a large focus on how to best communicate the risks associated with diseases and medical treatments, which are often un-intuitive and therefore require careful explanation. I’d be interested in seeing a similar examination of how to best communicate risks and evidence about arrest rates, incarceration rates, sentencing, and so on. If anyone has good references to suggest on that, please let me know.

  10. Their outcome makes no sense to me.

    First, just take the number one city on their graph, Orlando, with (going by their axis) roughly 25 killings per a million people (KPM). Ignoring the fact that they’ve got just over 1.5 years of data from 1/1/2015 to 7/14/2016, the data from the Guardian lists 8 police killings in this time frame in Orlando including Omar Mateen, who killed 49 people in a mass shooting in June 2016.

    Using their 25 KPM and the 2016 Census of 278,130 for Orlando to back-calculate the frequency gives 25*278130/10^6 = 6.95 or 7 killings in that time frame.

    Maybe they dropped Mateen from the analysis? His killing, after all, was justified under any use of force policy given that he killed 49 people and wounded another 53 before the police shot him.

    Also, 25 KPM seems awfully high. If you know something about Orlando geography, it’s that there are a lot of unincorporated Orange County addresses which list Orlando as its city when they’re actually outside the city boundaries. Which is why Orange County Sheriff patrols the area instead of Orlando PD.

    Hence, they’re dividing by the wrong Census number to get their rate for Orlando. Also, there are two police agencies listed that have different use of force policies, one with city-wide jurisdiction and one with county-wide jurisdiction. So there is likely more than one agency per city. In fact, there could be federal, state, local and specialty police (e.g., campus cops) operating in a given city. How can they account for those use of force policy differences?

    They’re taking a complex problem and oversimplifying to the point that what they’re claiming is misleading.

    • digithead, You are referring to figure 2 in the report: Title: “Examining the Role of Use of Force Policies in Ending Police Violence”, Author: “Samuel Sinyangwe”, Date: “September 20, 2016”. It clearly states on the graph that it uses US Census 2010 population data, which is 238,300 for “Orlando City, Florida”. That computes to 6 killings.
      Putting the Guardian’s “The Counted” database into map view, I confirm that there are 8 killings in the time frame stated on the graph, from 1/1/2015 to 7/15/2016. However, the graph is about “police departments”, and 2 of these killings only involved the Orange County Sheriff’s Office; the Guardian lists 6 killings invlving the Orlando PD for that time frame, which tallies with the number on the graph.

      What they are doing is looking at ONE agency and its performance, which seems like a sound approach.

      • So they used a 2010 number for 2015/2016 data. I mean, it’s not like there was an updated number they could use.

        And I disagree that it’s a sound approach, the geography of Orlando, even restricted to one agency, still complicates the matter. They are not comparing apples to apples across cities even within one state. For instance, Jacksonville (Duval) and Miami (Dade) are both city/county entities where Orlando is just a city. Those entities get the benefit of a bigger population to water down the KPM even when limiting it to their respective city police agencies.

        For instance, the City of Jacksonville is policed by the Jacksonville Sheriff’s Office rather than having two policing entities like Orlando. How do you compare KPM rates between Orlando PD and Jacksonville Sheriff Office to determine the effects of their respective use of force policies to see who’s better? Do you see why KPM is a misleading statistic?

        Moreover, it’s incredibly misleading to look at only killing without controlling for the context of an encounter. It’s classic confounding by indication. What happened to George Floyd was a crime. What happened to Omar Mateen was necessary.

  11. “…defunding and demilitarizing the police (this does not mean “get rid of police”! see here and here for what that might mean)”
    The second link says: “Groups have a range of demands, with some seeking modest reductions and others viewing full defunding as a step toward abolishing contemporary police services. ” So, yeah, some people clearly do mean ‘get rid of the police.’ One should be clear as to whether one means partially or fully defund police departments. Some people do mean the former.

    Also, has anyone considered that, and I know this idea is just absolutely bonkers, if we’re going to defund police departments, that instead of redirecting the money to some other project, I don’t know, just not spend it? given the fiscal conditions of many state governments?

  12. I might be interpreting this wrong, however, isn’t a binomial regression in terms of probabilities of Y happening rather than actual Y outcome itself? Meaning that the 72% figure is a 72% reduction in the probability of police violence happening rather than an actual 72% reduction in police violence itself (this is assuming as you said, if a police department implements 8 of the policies vs a department which implements 0 of the policies).

  13. For those who want to read Fisher for themselves, The Genetical Theory of Natural Selection (“perhaps the most important book on evolutionary
    genetics ever written”) is online here: https://archive.org/details/geneticaltheoryo031631mbp/page/n197/mode/2up?q=race. The eugenics part starts on page 170; it’s a very substantial section of the book. I think it’s especially important to note that this section appeared in the 1930 edition and was kept almost unchanged for the 1958 edition: Fisher had 30 years to revise his views and didn’t.

    The most charitable thing one can say about this section is that Fisher is mostly concerned with what we might call within-society or within-race genetic variation: he thinks the “ruling classes”, of any society, are genetically superior, because they’ve gotten there by selection. If there’s a claim that some races are on the whole inherently superior to others I didn’t see it in the quick re-scan I just did. But, obviously, it’s not a great leap from the one view to the other.

    • > Fisher is mostly concerned with what we might call within-society or within-race genetic variation

      Which is an active area of research today.

      > If there’s a claim that some races are on the whole inherently superior to others I didn’t see it in the quick re-scan I just did

      In other words, there is nothing in The Genetical Theory of Natural Selection to suggest that Fisher was a racist. But the mob must be appeased!

      • > In other words, there is nothing in The Genetical Theory of Natural Selection to suggest that Fisher was a racist.

        Fisher clearly takes racist views in other writing:

        ‘Sir Ronald Fisher has one fundamental objection to the Statement [1951 Unesco Statement on the Nature of Race and Race Differences], which, as he himself says, destroys the very spirit of the whole document. He believes that human groups differ profoundly “in their innate capacity for intellectual and emotional development” and concludes from this that the “practical international problem is that of learning to share the resources of this planet amicably with persons of materially different nature, and that this problem is being obscured by entirely well intentioned efforts to minimize the real differences that exist”. ‘

        That is crystal clear.

        available here: https://unesdoc.unesco.org/ark:/48223/pf0000073351 (quote is on the original’s page 27, and there’s more of the same from Fisher on page 56)

      • > In other words, there is nothing in The Genetical Theory of Natural Selection to suggest that Fisher was a racist. But the mob must be appeased!

        I don’t know about that Kane. As I pointed out, Fisher made repeated reference to the “virtues” of “ruling”, “higher” races, in opposition to the “lower”, “inferior”, “indigenous” races. Yes, he actually used the phrase “inferior races.” He never explicitly named the contemporary analogs to his historical examples, but I trust you can infer which subpopulations would be restricted by his eugenical proposals were they enacted when he proposed them.

        Inferring from his policy proposals, it’s clear that he believed that contemporaneously with himself that ruling race to be supported by a deliberate birth rate control was white Europeans. What does make his view, charitably speaking, more sophisticated than your garden variety racist was that he viewed racial superiority, as defined by a race’s genetic capacity for civilization, as very unstable over time. He believed that the superiority of a ruling race was a feature which frequently (on an evolutionary timescale) emerged under “barbarian” conditions, helped to build a sophisticated society, and then was bred out of existence by the very conditions of that society. In other words, he believed that the British aristocracy was genetically superior at the present moment, but that superiority was a relatively recent phenomenon which was at risk of declining to be replaced by the emergence of another class of superior rulers from “barbarian peoples.” Still pretty racist.

        Also the eugenics chapters aren’t really aren’t worth reading. It’s completely bare of serious scientific content, just the kind of useless pseudoscientific historical theory which is built to explain everything but can predict nothing. Its chief historical appeal is that it seems to dovetail nicely with the arc of civilizations repeatedly rising and falling through history, but it does so because it makes no falsifiable statements about the timescale or manner of collapse. The only relationship to empirical facts arises in the fact that there have been civilizations that rise and fall. It also purports to extract universal behaviors of human populations from exclusively European, Mediterranean, and Levantine examples. Complete ignorance of China, India, the Mongols, the Maya, the Incans, and the Sahelian kingdoms is unbecoming of a grand theory of human civilization.

    • Ben said,
      “Fisher is mostly concerned with what we might call within-society or within-race genetic variation: he thinks the “ruling classes”, of any society, are genetically superior, because they’ve gotten there by selection. ”

      This is an example of what bothers me about describing evolution as “survival of the fittest” — and why I think a better way to describe evolution is “survival of the fit enough to have survived under the circumstances in which they have so far found themselves”. Kind of like “Count no man happy while yet he lives”.

  14. Enough with the gaslighting. Maybe we should simply listen to what is being said, rather then come up with our rationalizations about what they “really” mean.

    https://www.forbes.com/sites/tommybeer/2020/06/12/minneapolis-city-council-unanimously-votes-to-replace-police-with-community-led-model/#16e5d88f71a5

    https://www.nytimes.com/2020/06/12/opinion/sunday/floyd-abolish-defund-police.html

    “No more cops” and “ACAB”, are spray-painted all over University buildings and public and private property in my small to moderately sized Midwest town. Let’s stop pretending there isn’t a meaningful and powerful faction that wants to completely remove the police and law enforcement.

      • But there is a substantial faction that want no police or law enforcement whatsoever.

        First it was “defund the police doesn’t mean what you think it means”. Now it’s “nobody knows what they really mean”. Please, enough with the gaslighting.

        Of course there are people who are advocating for police and law enforcement reform, restructuring, etc. They didn’t “abolish” the Camden police department and suddenly the crime rate went down as you suggested. The City of Camden Police Department was “disbanded in favor of a new county police department, the Camden County Police Department”. An exact quote from the Wikipedia article.

        Enough with the gaslighting, obfuscation, and bad faith arguments.

        • Nathalie –

          Your categories are vague, and your quantifications of who fits into which category are non existent (which they would have to be since your categories are vague).

          So it’s not possible for me to assess who you think is gaslighring about what, who is obfuscating, who is arguing in bad faith etc.

          There is a substantial faction that is talking about Camden (and Compton) as an example for the kinds of reforms they want. Most of what I see is people arguing for creating less of a need for armed police by increasing social services to address the non-criminal and non-violent calls that cops currently spend the bulk of their time dealing with now. And also by addressing the kinds of social problems that lead to violent crime.

          A great deal of what I see are calls for better training in deescalation techniques, less militarization, and better transparency and oversight of cops with a history of use of force problems.

          In fact, the articles you linked above suggested exactly those types of reforms (despite the headline of the NYTimes arricle, the article mentioned reducing police funding and workforce by something like 1/2, not eliminating cops altogether, and reducing the need for armed police).

          I would say that you’re not likely to further a goal of good faith discussion by outrage-mining through focusing on a *relatively* smaller faction of extremists, and wrongly suggesting that many people are ignoring that faction. And that’s what it appears to me that you’re going.

          Here’s a prominant spokesperson for many folks towards the left end of the spectrum.

          https://www.google.com/amp/s/thehill.com/homenews/sunday-talk-shows/502643-stacey-abrams-calls-defunding-police-movement-a-false-choice-idea%3famp

          The thing is thst the way forward will require stakeholder dialog among factions with many views.

          What would you suggest how to go forward given that there is a faction who want to elongate any police all-together? How do you qanirdybthe size of thsr faction? What would you suggest doing with them?

Leave a Reply to Mendel Cancel reply

Your email address will not be published. Required fields are marked *