Dale Lehman points to a research article, “Is discrimination widespread? Testing assumptions about bias on a university campus,” linked to by Alex Tabarrok. Here’s the abstract:
Discrimination has persisted in our society despite steady improvements in explicit attitudes toward marginalized social groups. The most common explanation for this apparent paradox is that due to implicit biases, most individuals behave in slightly discriminatory ways outside of their own awareness (the dispersed discrimination account). Another explanation holds that a numerical minority of individuals who are moderately or highly biased are responsible for most observed discriminatory behaviors (the concentrated discrimination account). We tested these 2 accounts against each other in a series of studies at a large, public university (total N = 16,600). In 4 large-scale surveys, students from marginalized groups reported that they generally felt welcome and respected on campus (albeit less so than nonmarginalized students) and that a numerical minority of their peers (around 20%) engage in subtle or explicit forms of discrimination. In 5 field experiments with 8 different samples, we manipulated the social group membership of trained confederates and measured the behaviors of naïve bystanders. The results showed that between 5% and 20% of the participants treated the confederates belonging to marginalized groups more negatively than nonmarginalized confederates. Our findings are inconsistent with the dispersed discrimination account but support the concentrated discrimination account. The Pareto principle states that, for many events, roughly 80% of the effects come from 20% of the causes. Our results suggest that the Pareto principle also applies to discrimination, at least at the large, public university where the studies were conducted. We discuss implications for prodiversity initiatives.
Lehman writes:
I [Lehman] don’t have access to the published study, but I did find what appears to be an earlier draft. Admittedly, I only read through the first half – at 80+ pages, it was a bit too much for my patience. In many ways, it isn’t a bad exploratory analysis of an important subject. But I had a few repeated reactions as I read through (the first half). First, the actual numerical results and their interpretation did not seem to match up to me. While I say some big differences in the campus climate survey results for minority groups compared with majority groups, the authors appear to cherry pick ways to interpret the differences as being small (like relying on whether or not a majority of the group feel welcome on campus – rather than the differences in how much they perceive other groups to treat them differently).
There is the usual reliance on statistical significance and some odd practices (such as using a logistic regression for one Likert scale question but then linearalizing the scale in another study) and the usual forking paths. But my larger questions fall into two categories:
1. Can studies like this really shed light on the basic issue: systemic/dispersed discrimination vs the “bad apples” (concentrated) discrimination? They are all indirect measures (by necessity) for samples of a particular population – university students. Some measures, such as willingness to hold a door for a stranger or help pick up dropped cards don’t seem to me to say anything about the discrimination issue. In fact, I suspect that many forms of discrimination manifest themselves by an increased willingness to help rather than a decreased willingness. While we can’t measure discrimination directly ( at least not easily), it isn’t clear to me that it has been well defined in these studies. I would view any differential treatment of groups as potential implicit discrimination. They do cite a considerable body of research by psychologists regarding discrimination (which I have no expertise in, but feel should be respected), but I kept having the feeling that it was unclear whether they were measuring the right things.
2. Do their studies really support the concentrated discrimination model and not the systemic discrimination model? It almost seemed to me that you could have interpreted the same results in the opposite direction than the authors did – although since I only read half of the paper, I might be wrong. But the responses from marginalized groups do seem different than the majority responses to me. It almost seems like they overlayed their preferred interpretation on results that could have supported either model.
I think research in this area is important – society has devoted large resources (monetary, time, and emotional) on the basis of believing the systemic model. And, while I don’t like surveys generally, I’m not sure they can be avoided on this topic. Nor do I think it wise to make blanket judgements on an entire field of research. As I said, as exploratory research I think it is useful and potentially important. But I have some discomfort whether the measures are up to the task they have been designed for.
My reply: I think the paper is flawed both in conception and in analysis because of binary thinking. In real life, everyone has some bias but some people have a lot more bias than others. So I think both their hypotheses are true. I also think it does not make sense to try to estimate the percentage of people who are biased by looking at the percentage of people who show a statistically significant difference. If you do this, the percent biased you will see becomes a function of your sample size. As you say, the questions they are asking are potentially important. Unfortunately I don’t see this study as being useful.
Credit to the commenters on Tabarrok’s post for expressing skepticism on this study.
Not familiar with the study in question but the two papers below by Patrick Kline and Chris Walters are both very well done and estimates the distribution of discriminatory hiring practices among a group of large US employers, and finds support for the concentration hypothesis in that context. most firms don’t seem to discriminate (or discriminate very little) while a minority discriminate a lot. the inference issues you raise (share discriminatory is a function of sample size) are also carefully dealt with in these studies
https://eml.berkeley.edu//~crwalters/papers/randres.pdf
https://eml.berkeley.edu/~pkline/papers/Kline%20Walters%20RD%20EMA21%20plus%20supp.pdf
The “black sounding name” thing is a rough measure at best. What percentage of black people have “black-sounding” names? Russel Wilson? Not that I doubt names have an effect. Some immigrant groups give their kids “American” sounding names to help them fit it. Just the same what name is and isn’t “black sounding” is highly variable.
Whatever the case, it’s not quite clear why this type of discrimination would result in a society-wide effect. If the “talent” factor has any significant effect – and I’m not sure it does, but if it does – non-discriminatory firms should on average have the edge, and that should mean that their employees are better off. So while discriminatory firms may protect jobs for certain groups, their employees should be less well off on average (lower bonuses, for example, fewer higher ranking jobs because fewer of these firms are strongly competitive). So in this respect discrimination should be self-correcting.
Also, this paper seems to claim this as fact, but although I can’t find it right now Tabarrok on Marginal Revolution recently referenced a paper that concluded that there was **no** racial discrimination in the US, at least within the resolution of the available measures. So I’m not quite clear – it seems there are two kinds of social scientists on this issue?
“it’s not quite clear why this type of discrimination would result in a society-wide effect”
the employers in the first study i linked to employ about 15 million workers, and represent all major industries. if Kline and Walters’ results generalize, which the results of the second study (which sample a larger set of employers but samples each employer a smaller number of times) suggests they do, this would likely have “society-wide” effects
“Tabarrok on Marginal Revolution recently referenced a paper that concluded that there was **no** racial discrimination in the US”
ah, well, if a blog post by Alex Tabarrok on Marginal Revolution says there is no racial discrimination in the United States, then i suppose the question is settled and all research on racial discrimination can be put to an end. that’ll save us a lot of time and trouble ;)
“if a blog post by Alex Tabarrok on Marginal Revolution”
It was a blog post referencing a published paper. The published paper claimed that apparent “discrimination” results from differences in education among many other factors. Whatever the case, there very large differences in important characteristics between ethnic groups in the US – teen pregnancy rates, for example, which vary by a whopping order of magnitude between ethnic groups – that could conceivably account for purported “discrimination”.
“the employers in the first study i linked to employ about 15 million workers, and represent all major industries”
So what? The point is that if a discrimination is concentrated in a few firms and the “talent effect” is real, discriminatory firms and their employees would be less successful, while the non-discriminatory firms and their employees would be more successful, therefore at least partially offsetting any aggregate wealth effect from the “discrimination” on a society-wide basis. It doesn’t matter how many people or employees you’re talking about.
Just for example, what would happen if the Boston Celtics decided to employ only Irish basketball players? Hilarious, right? They’d be permanent basement dwellers. The group that “benefits” from discrimination would be far worse off NBA-wide than the groups that the “Irish Only Celtics” discriminated against. If there were a great Irish player, the last team they’d want to play for is the “Irish Only Celtics”, since they’d make far less money playing there.
So the question is this: do the people that are purportedly “discriminated” against offer any talent or skill that matters in the outcome of companies or not? If not, then what discrimination has occurred? Or does this paper implicitly claim that there’s no real talent factor anyway, therefore companies should have some racial balance specified by society wide proportions?
I can think of of other potential flaws in the “sending resumes” model of testing discrimination that would doom it as any kind of reliable measure of discrimination, but I’m not going to read the paper so I guess there’s no point in outlining them if I don’t know they’re relevant to this research. Maybe they’ll come up in discussion.
“I think the paper is flawed both in conception and in analysis because of binary thinking. In real life, everyone has some bias but some people have a lot more bias than others. So I think both their hypotheses are true.”
I’m not sure that this makes contact with the hypotheses, which are about the relationship between implicit bias and explicit discriminatory behaviors. What you say about implicit bias could be true, but it’s not inconsistent with their framing or results. They seem to be showing evidence that rather than diffuse, low-grade implicit bias leading to slight but widespread discriminatory behavior, a minority carry implicit bias that is strong enough to manifest in explicitly discriminatory behaviors that are fairly concentrated.
Adam:
I’m not claiming that the general substantive claims of the article are true (or false). I’m saying that it’s not going to work to try to estimate the percentage of people who are biased by looking at the percentage of people who show a statistically significant difference. I think the statistical approach used in the paper is fundamentally flawed, and I connect that to a conceptual flaw of binary thinking. Even without thinking about the conceptual flaw, the statistical method is inappropriate.
thought this might be of interest, new preprint server with public peer review:
https://twitter.com/namalhotra/status/1564255571959615488