43 is the loneliest number that you’ll ever do

Peter Hickman writes:

I am a research assistant for an economist, and I just came across something that seems to me to be poor research that is getting media coverage.

The Cigna study is here. It’s measuring the degree of loneliness in America. You can click “read the full report” for more details. If you google *cigna study loneliness* you’ll see there’s lots of media coverage.

What first seemed fishy to me is that they say “The evaluation of loneliness was measured by a score of 43 or higher on the UCLA Loneliness Scale.” The number stood out. Why is 43 being used? That seems arbitrary. I don’t know anything about this literature but my quick google of “ucla loneliness 43” and “ucla loneliness cutoff” didn’t suggest that this was a previously accepted cutoff.

Then I looked at what the survey actually asks and how it is scored. The UCLA loneliness survey (first version I found is here) has 20 questions that all basically ask “do you feel lonely” in different ways. You can say “always”, “sometimes”, “rarely”, or “never.” The link I found suggests scoring it using a 0-3 scale with “always” being a 3, but it seems that Cigna used a 1-4 scale. So that means that if you say “rarely” on all the questions, you’ll get a score of 40 (20*2). To get a 43, you could say “sometimes” for 3 questions and “rarely” for the other 17 (3*3 + 17*2). Seems to me that you could be considered as “rarely feeling lonely,” but as Cigna spins it you’d be “lonely”. Then we get media outlets like this with the headline that “loneliness is an epidemic.” [I followed the link, and this is not a news outlet at all! It’s a public relations outlet. — ed.]

Loneliness is bad and I believe other work has been done that has suggested loneliness is on the rise or is at least affecting a significant number of people. I’m not challenging that, but the way that the Cigna results are being read just seems wrong.

My reply:

1. The report says, “Possible loneliness scores range from 20 to 80, with the total average national loneliness score in America reaching 44.” So:
(a) This is indeed consistent with adding the responses to 20 questions, each of which is on a 1-4 scale, and
(b) If the average score was 44, I wonder if they picked 43 as a cutoff so that they’d be able to say that more than half of Americans suffer from loneliness.
Maybe they should’ve set 20 as their threshold: then they could say that 100% of Americans are lonely!

2. The news coverage indeed seems credulous. For example, here’s NPR—they seem to fall for every one of these wacky studies:

Now a nationwide survey by the health insurer Cigna underscores that. It finds that loneliness is widespread in America, with nearly 50 percent of respondents reporting that they feel alone or left out always or sometimes.

“Always or sometimes”! That pretty much covers it, huh?

Also this bit:

“Half of Americans view themselves as lonely,” said David Cordani, president and CEO of Cigna Corp. “I can’t help but be surprised [by that].” . . . But the results are consistent with other previous research, says Julianne Holt-Lunstad, a psychologist at Brigham Young University . . .

This is the familiar two-step, where every claim has to be a big surprise (hence newsworthy) but also no surprised at all (hence consistent with earlier research).

And here’s the USA Today report, which pretty much doubles as an ad for the study’s sponsor:

Cigna is testing the integration of “mental wellness” into medical care at doctors’ offices in Denver, New Jersey and Virginia and plans to expand the pilot to ten office by the end of the year. The program will be offered to patients with chronic conditions such as type 2 diabetes, heart and gastrointestinal disease, depression, eating disorders and substance use disorders.

3. From a statistical standpoint, it is typically not a good idea to take a numerical scale and summarize as high/low: you’re throwing away a lot of information. If you feel that for reasons of communication you really must break the scale into categories, you should use three divisions—high, medium, low—and then compare high to low; see this paper for discussion of this general point.

4. I took a look at the report. The data are correlational but the report is full of causal language. For example, the subtitle is “Survey of 20,000 Americans Examining Behaviors Driving [italics added] Loneliness in the United States.” On the next page, it says “Loneliness has the same impact [italics added] on mortality as smoking 15 cigarettes a day.” And also, “The research also highlighted some bright spots that will be instrumental in driving change. It showed that individuals who are less lonely are more likely to have regular in-person interactions, are in good overall physical and mental health, have found a balance in their daily activities, and are employed.”

Hey, just give people more regular in-person interactions, and they’ll be less lonely. Who’d a thunk it??

Yeah, yeah . . .

I agree with Hickman that loneliness is real and worth studying, and I think that this sort of national survey can be a good way of getting a picture of the problem. I also have no problem with an insurance company doing this study: sure, some conflicts of interest arise, especially when it comes time to distribute and promote the study, but conflicts of interests are unavoidable, and that’s no reason not to trust data that come from a reputable source.

No, my problem is pretty much all in the presentation. But the presentation, and the uncritical press, is what people see.

It may be that the net effect of all that publicity is positive for society: sure, maybe 43 is an arbitrary number; sure, you can get categorized as a “lonely” person even if the vast majority of your responses are that you’re rarely feeling lonely; sure, the causal inferences are shaky at best; but the big picture is that people are talking about loneliness being a problem, and that’s important, even if it took some shaky data analysis to get there.

The reasoning here is similar to the reasoning of a guy who criticized me on twitter for writing about an error in that mortality-trends report a couple years ago. He didn’t question my statistics; he just thought the larger goal of alerting people to scary mortality trends was so important, that it would counterproductive from a political standpoint to call into question any aspect of that well-publicized study.

And, indeed, from a policy outcomes standpoint, maybe the person who criticized me was correct. Maybe, if a message is important, it’s best to give it undiluted support, to pretend it has no problems in order to keep public focus on the larger problem. I don’t know.

Long run, though, I do feel, or hope, that a better understanding of what we can learn from the data will allow us to better assess and ameliorate our social problems. In any case, I don’t want this post to be taken as a criticism of the people who did this study: I have every reason to suspect that they’re doing their best, and I hope that in future studies they’re more careful about how they summarize numerical measurements.

10 thoughts on “43 is the loneliest number that you’ll ever do

  1. Good point about the unjustified causal language: the word “impact” appears 16 times in the report! When they discuss these “impacts” (correlations), though, they do compare the average (numerical) loneliness scores among people in three categories (e.g., too much exercise, the right amount of exercise, too little exercise), which is what you suggest in your paper, right? Except the Cigna report often finds that the middle (“just right”) category has the lowest loneliness scores…

  2. “Maybe, if a message is important, it’s best to give it undiluted support, to pretend it has no problems in order to keep public focus on the larger problem.”

    A media headline after a reporter read this, “Statistician says it’s okay to publish garbage as long as the message is important.”

Leave a Reply to Sameera Daniels Cancel reply

Your email address will not be published. Required fields are marked *