Skip to content
 

Retraction of racial essentialist article that appeared in Psychological Science

Scene 1:

It all started for me on 2 Jan when I received this email from Keith Donohue in Fargo, North Dakota:

I am a longtime reader, and I am curious about your reaction to an (in press) journal article that I recently came across. . . .

The paper is “Declines in Religiosity Predicted Increases in Violent Crime—But Not Among Countries with Relatively High Average IQ”, by Clark and colleagues, which is available on research gate and is in press at Psychological Science. Some of the authors have also written about this work in the Boston Globe. . . .

Psychology has a long and ignoble history of using dubious measures for intellectual ability to make general claims about differences in average intelligence between groups of people – racial/ethnic groups, immigrant groups, national groups, etc. Often, these claims have aligned with prevailing prejudices or supported frankly racist social policies. This history disturbs me, and seeing echoes of it a flagship journal for my field disturbs me more. I guess what I am trying to say is that my concerns about this paper go beyond its methodological issues, and I need to be honest about that.

With that in mind, I have tried to organize and highlight the methodological issues that might interest you or might be worth commenting on. . . .

I posted something on it that very day:

Someone pointed me to an over-the-top social science paper that is scheduled to be published soon. I then wasted 2 hours writing some combination of statistical commentary and rant.

I expect that, once the paper is published, there will be major controversy, as its empirical findings, such as they are, are yoked to political opinions which seem pretty much targeted to offend lots of people in its academic field.

Fortunately, by the time my post appears, the furor should have quieted. . . .

So, what happened? It wasn’t a matter of the furor quieting. There was no furor at all! Psychological Science just published the paper, and everyone was like, OK, sure, no prob. Just another This Week in Psychological Science.

Scene 2:

Then on 31 May my scheduled post appeared:

The Association for Psychological Science (a “highly educated and intelligent group,” for sure) has decided that they should be humble about promoting the unique and relatively novel values that thrive among them. And these unique and relatively novel values include . . . “criminal justice reform and unrestricted sociosexuality.” If members of the Association for Psychological Science want criminal justice reform and unrestricted sociosexuality for themselves, that’s fine. Members of the APS should be able to steal a loaf of bread without fear of their hands getting cut off, and they should be able to fool around without fear of any other body parts getting cut off . . . but for groups with lower self-control and cognitive ability—I don’t know which groups these might be, you’ll just have to imagine—anyway, for those lesser breeds, no criminal justice reform or unrestricted sociosexuality for you. Gotta bring down the hammer—it’s for your own good! . . .

I also discussed the methodological problems of the paper:

Set aside for a moment problems with the data and statistical analysis, and suppose that the data show exactly what the authors claimed, that time trends in religious attendance correlate with time trends in homicide rates in low-IQ countries but not in high-IQ countries. Suppose that’s all as they say. How can you, from that pattern, draw the conclusion that “The prescriptive values of highly educated groups (such as secularism, but also libertarianism, criminal justice reform, and unrestricted sociosexuality, among others) may work for groups that are highly cognitively sophisticated and self-controlled, but they may be injurious to groups with lower self-control and cognitive ability”? You can’t. To make such a claim is not a gap in logic, it’s a chasm. Aristotle is spinning in his goddam grave, and Lewis Carroll, Georg Cantor, and Kurt Godel ain’t so happy either. This is story time run amok.

All this is beyond the serious data problems with the article.

I also wrote that I was surprised that Psychological Science would publish this paper, given its political content and given that academic psychology is pretty left-wing (Roy Baumeister aside) and consciously anti-racist.

My guess:

It’s some combination of: (a) for the APS editors, support of the in-group is more important than political ideology, and Baumeister’s in the in-group, (b) nobody from the journal ever went to the trouble of reading the article from beginning to end (I know I didn’t enjoy the task!), (c) if they did read the paper, they’re too clueless to have understood its political implications.

Scene 3:

A few days later, the cognitive dissonance of the Association for Psychological Science stepped up a notch when they issued this statement:

From its inception, the field of psychological science has studied the causes and harmful impacts of stereotypes, prejudice, and disparity. . . . our profession now has a sizable body of research that can be brought to bear to understand the persistent racism and subsequent confrontations we are witnessing in cities and communities across our nation and throughout the world. . . .

We stand ready to add our voice and expertise to bring about positive change and to stand against injustice and racism in all forms. . . .

As a diverse, international organization of more than 30,000 scientists and researchers around the world working to understand all aspects of mind and behavior, APS profoundly respects the essential worth of all people and cultures, regardless of race, gender, and ethnicity.

This seemed kinda wack, given that they’d just published a paper with a racial-essentialist perspective that slammed “criminal justice reform and unrestricted sociosexuality.” So much for standing against injustice and respecting the essential worth of all people!

Here was my take on how the Association for Psychological Science could help to bring about positive change:

As the statistics indicate, there are serious disparities in American society. But I’m not convinced that the profession of psychological science “has a sizable body of research that can be brought to bear to understand the persistent racism and subsequent confrontations we are witnessing in cities and communities across our nation and throughout the world.” I’m not convinced that the implicit association test and things like it have anything useful to add, and I’m not convinced that traditional racial explanations of social and economic disparities have anything to add.

I understand that, as individuals and as an organization, the members and leaders of the Association for Psychological Science would like to be helpful in these difficult times. As individuals, they can help in various ways: they can peacefully protest, they can join neighborhood watch groups to protect their communities, they can be good parents, friends, and neighbors, they can vote and write their congressmembers etc. They can be productive members of society and coach Little League and support local businesses, they can minimize their ecological footprint and bring a smile to work every day. They can give Ted talks, they can blog, they can even go on twitter. Lots of ways to make yourself useful. But, to paraphrase John Tukey, the combination of an archive of Psychological Science articles and an aching desire to be helpful does not ensure that reasonable help can be extracted from a given archive of articles.

I get that we all want to feel useful, but maybe promoting old journal articles isn’t actually useful at all.

Scene 4:

I still haven’t heard from the Association for Psychological Science on this—maybe they feel bad for publishing lies about me a few years ago—but in the meantime the authors of the disputed article retracted it, on 17 June.

Here’s the story, and here’s what the authors wrote:

Over the past weeks, we have heard considerable concern and scrutiny over our recent article on religion, violence, and IQ in Psychological Science. In particular, papers by Karis et al. (2015; pointed out by Fearon and Eisner) and by Dickins et al. (2017; pointed out by Sear) have prompted us to spend much of the last week digging deeper into the research behind our measures—a level of vetting we should have done before submitting the paper. As a result, we no longer have confidence in our findings. Because of imputation for many countries, the homicide data have limitations that call our conclusions into question. The IQ data, however, have much more serious issues. The persistence of these highly questionable data sources in the psychological literature has convinced us that research with certain kinds of flaws should be pulled from the record as its existence steers us further from the truth rather than closer to it. We now believe our paper falls into that category and are retracting it immediately.

It’s good when people can admit their mistakes. In this case there was no need for them to refer to Karis, Fearon, Eisner, Dickins, and Sear, as these problems were pointed out by Keith Donohue in his email:

This data seems to be collections of IQ estimates made from proxies of intelligence, such as school achievement or scores on other tests of mental abilities. Based on my training and experience with intelligence testing, this research decision seems problematic. Also problematic is the decision to input missing values for some nations based on data from neighboring nations. . . .

My point here is not that Donohue is a better reference than these other sources but rather that the problems were well known to seemingly everyone who’d looked at these data—except for the four authors of this paper and the editorial board of Psychological Science.

I guess it’s tough if you’re editing a journal that’s published so much junk science. It kinda lowers your standards, and you’ll publish pretty much anything that’s headline-worthy and has a bunch of p-values in it.

The other message is that open post-publication review has once again outperformed secret pre-publication review. Remember, the problem with peer review is . . . the peers.

I wouldn’t ever want a paper of mine to be reviewed by a peer of Roy Baumeister, Robert Sternberg, or various other dukes and earls of the academic psychology establishment.

Scene 5:

I don’t know what will happen next. The above-linked Retraction Watch article quotes the former editor of the journal as saying that the current editor “is working on how best to respond to criticisms” of that paper. I’m not sure why they need to respond, now that the paper has been retracted.

It’s pretty clear that all this fuss is about the article’s political content, not its data problems.

Here’s an interesting question: what if the journal article had the same inflammatory political content and the same breathtaking leaps of logic, but no data problems? What would Psychological Science do then? I’m not sure.

For example, suppose the four authors of that paper had put together some statistics on racial differences in standardized tests in the U.S. and then run some state-level analysis interacting with religiosity and ethnic composition? With U.S. data there aren’t such data problems as with the cross-national IQ data.

They could’ve supplemented this with some experiments on undergraduates and on Mechanical Turk participants, finding some differences—you can always find some differences—and then spun some story about how religion is needed to keep the masses under control. They could’ve still taken that bold stand against criminal justice reform and unrestricted sociosexuality.

I guess that such an article could’ve been accepted by the journal, for the same reasons they accepted the original paper. If so, then any retraction would not have been able to use the low quality of the international data as an excuse: they would’ve had to face up to the problems of story time. And that would be tough, given that the APS has gone all-in on story time (“We stand ready to add our voice and expertise to bring about positive change”) when coming from the left, if not the right.

One more thing

I’m glad that the authors stand by the principle of data quality, but there’s something about their retraction that bothered me.

In their note, they didn’t say anything about “criminal justice reform and unrestricted sociosexuality.” But, as I wrote in my earlier post, their conclusions about society do not follow from their statistical analysis, even setting aside any data problems. So, sure, their data were a mess, but even if their data were perfect, I don’t think they could draw all those conclusions in their paper.

But if Psychological Science were to retract all the papers they’ve published where the substantive conclusions don’t follow from the data analysis, they’d have to retract a lot of papers.

What to do?

I guess that, when see a published article, we have to judge it on its own merits rather than relying on information from the journal.

I mean, sure, sometimes you’ll get a cue such as “written by X and appearing in Psychological Science” or “published in PNAS and edited by Y” and you’ll know not to take it seriously, because it’s just pushing some political or professional agenda, but in general that’s not enough information, and you’ll have to actually take a look at the article or borrow the judgment of a trusted colleague or arbiter, or follow the hivemind of open post-publication review.

Even papers from Harvard published in Lancet get things right sometimes.

P.S. Zad sends the above image, titled “How to properly clean your data.” If only Team Baumeister were as competent as this cat, think of how much trouble would’ve been avoided!

19 Comments

  1. Decisions about acceptance/rejection/revision are made by the *editor* and not the reviewers. If there is a failure of peer review here, the responsibility lies primarily with the action editor (and the policy set by the senior editor/association), who selects reviewers, interprets their advice, makes the decision, oversees revisions, and accepts the paper. I have heard from one reviewer of this paper, a very statistically sophisticated person, who tried to get the editor to reject the paper based on the very low quality of the data and the over-interpretation of it.

    It’s important to focus on where the real responsibility lies–we’re often unsure about the nature of what the reviewers did, but the ultimately responsibility lies in the hands of the person who made the decision, and not her/his advisors.

  2. Luis Usier says:

    This is an extremely well-written blog post. Very clever.

    And yes, story time has run amok. Too many researchers use statistics as a crutch, or as a shield, for the claims they want to make. But it should be the claims that support the results of the research, not the other way around.

  3. Wonks Anonymous says:

    Before you acknowledged that they didn’t even use the word “race” in their paper, but instead was merely “consistent with racist attitudes”. So to call the paper “racial essentialist” is simply not true. If you want to know what actual racial essentialism looks like, see this.

    • Andrew says:

      Wonks:

      As I wrote in my linked comment, they do not use the word “race” in their paper. But I think that’s what they’re talking about when they refer to country-average IQ and “groups of different mean cognitive ability.” Regarding your other link: there are different degrees of racial essentialism, that’s for sure.

      • Wonks Anonymous says:

        A country is not a race, which is why social scientists make international comparisons all the time without the concept of race being necessary at all. The concept of “race” isn’t necessary for their paper, so it’s simply not racially essentialist (not even a “degree” of it). You might “think that’s what they’re talking about”, but it’s not what they actually wrote. I could claim that your obviously false characterization of their paper is an attempt to make a Straussian argument in favor of it and thereby distract from the actual deficiencies of it, but that would be foolish since everybody else can see what you actually wrote rather than what someone imagines is in your head. It’s impossible to prove what’s in someone’s head, whereas what people actually say is verifiable and makes communication possible.

        • Andrew says:

          Wonks:

          I have no idea what is in the heads of the authors. Their paper does not just talk about countries, it also talks about “groups.” The authors have a problem with “criminal justice reform and unrestricted sociosexuality,” at least among “cognitively disadvantaged groups.” It all seems pretty political to me, the kind of thing that might be popular at the Edge Foundation or other venues where people use evolution talk to explain why inequality is a good thing.

          But, sure, we can restrict ourselves to what is verifiable. In that case, I’ll just say that the paper is a standard social-science example of “story time,” big conclusions not supported by data. The unusual thing about this paper is not its reliance on questionable data—we see this in social science all the time—but that it was put in the wrong place, from a political perspective. In an econ journal, a paper with this political angle would fit right in. But in a psychology journal, it stands out.

          • Wonks Anonymous says:

            The existence of people (perhaps that would have been a better term than “groups”, although one could still group them together analytically) with a cognitive disadvantage doesn’t depend on evolution, or race, and their argument doesn’t appear to depend on those things either. It is a political argument, which explicitly touches on political topics.

            Your description of the paper as engaging in unfounded “story time” seems fair enough, and I would expect you to critique it on that ground. Going on about things nobody actually said seems more typical of those complaining about “methodological terrorism”.

            • Phil says:

              Wonks,
              The article smacks of racism and is the kind of thing a racist would write if they wanted to promote their views without using language that is outright unacceptable in today’s society, but — to agree with you a bit — just because it is consistent with a racist view of the world doesn’t prove the authors are racist.
              But: if you publish a paper that says that entire countries full of people are “cognitively disadvantaged” to the extent that social influences that are fine and dandy for smarter “groups” just won’t work for them, you’ve got to know that you are giving aid and comfort to racists even if you aren’t one yourself. If you are right, maybe that’s OK: if something is true you should be allowed to say it, indeed encouraged to say it. But you’d better make really sure you’re right. If you’re using junk science that is easily recognized as such to give aid and comfort to racists, don’t be surprised if people think you’re a racist yourself. Yeah, maybe it’s wrong to jump to that conclusion, maybe there’s some less morally objectionable reason someone would publish this sort of thing without doing any sort of due diligence…but that’s not the way to bet.

    • prognostication says:

      Oh cool, a link to the Unz Review. You know, the website run by the guy who thinks that David Irving is Actually Good and that Ford’s tract “The International Jew” had some good points. High caliber stuff.

      (/s, obviously)

      • Andrew says:

        Hey, if it’s good enough for David Brooks . . .

      • Wonks Anonymous says:

        Your suspicion that the contents there are low-caliber is correct. If you are a racial essentialist who thinks that I’ve “weak-manned” that position by linking to an unusually bad argument for it (my assumption is that all arguments for it would be bad), I can’t claim to have come across that many for it but it is indisputably AN example regardless of its quality.

        • prognostication says:

          To be honest, I couldn’t figure out what you were getting at. I thought you might be posting the Unz piece approvingly, and it was so terribly written I couldn’t get through more than a couple sentences so I still wasn’t sure. With this follow-up I now understand that you were posting it as an example of what hardcore “race realists” (vomit) believe, to draw a contrast to this article’s dog whistle-y version, which you don’t consider to be clear racial essentialism. I don’t agree, I think when you start talking about national IQ you are necessarily talking about racial and ethnic IQ differences, as Andrew says, but I think yours is a defensible position. My apologies.

          • Wonks Anonymous says:

            Since you understandably didn’t read much of the actually essentialist argument, then you didn’t get to how essentialism is distinguished from other perspectives. Essentialism dates back to ancient Greek philosophers who thought there was an “essence” to various types of things, sometimes referred to as a “Platonic essence” nowadays. Darwin argued that all species share a common descent and gradually differentiated over generations and are thus not different in “essence”. Modern biologists sometimes use Ernst Mayr’s species concept to distinguish between them, but even that can only possibly apply to sexually reproducing species and when they find fossils they don’t actually use his interbreeding criteria before declaring them to be different “species”. “Species” is thus a concept used pragmatically because it is useful for some purposes rather than because it is a fundamental scientific fact. The linked argument is against Darwinism because Darwinism is incompatible with special essentialism, much less racial essentialism. Species essentialism is nowadays mostly found among people who don’t know much science and just think intuitively. But if an argument doesn’t involve species at all (let’s say something along the lines of Asimov’s “Not as We Know It” which only uses the term “species” at the molecular level), it can’t very well be species essentialist even if such a person is one of those with such beliefs.

            I don’t agree, I think when you start talking about national IQ you are necessarily talking about racial and ethnic IQ differences

            That sounds more like racial essentialism to me! A particularly weird variety of it which thinks that racial and ethnic differences are the only possible cause of variation in IQ across countries, even though demographically similar countries can have significant differences in cognitive tests.

  4. Joshua says:

    I have a sneaky suspicion we’ll soon hear about politically correct censorship.

  5. One has to be careful in this area. If one comes up with essentially the same numbers, but it’s someone one doing it, no one bats an eye. It’s only the unfortunate Richard Lynn who shall not be mentioned (or worse, cited). Cordon sanitaire. World Bank has produced similar numbers, economists too e.g. Hanushek http://hanushek.stanford.edu/sites/default/files/publications/Hanushek%2BKimko%202000%20AER%2090%285%29.pdf, and even some Korean plagiarizers managed to copy the work and publish it in The Lancet of all places!

    Small compilation of related near-duplicate research here:
    https://emilkirkegaard.dk/en/?p=7746

    Review of the 2019 Becker book that redid all the numbers from scratch:
    https://emilkirkegaard.dk/en/?p=7980

Leave a Reply