Night Hawk

Sam Harper writes:

Not sure whether you saw the NYT story a couple of days ago about the declining prospects for democracy in rich countries (based on a recently published paper by Roberto Foa (University of Melbourne) and Yascha Mounk (Harvard). This graph, showing differences in the fraction of individuals reporting that it is “essential” to live in a democracy (i.e., 10 out of 10 on an importance scale) across birth cohorts was used (by the Times reporter) as evidence that “the percentage of people who say it is “essential” to live in a democracy has plummeted”.

Of course there are many potential issues here (e.g., the arbitrary choice of 10/10 as a definition of “essential”), but I primarily object to the use of birth cohort differences from a cross-sectional survey (World Values Survey) as evidence of “plummeting” rates, which I think typically would mean changes over time rather than over birth cohorts. I used the same data to replicate this graph but also showed that there has been little change over time, and, moreover, (at least for the US) the declines in saying it is “essential” to live in a democracy were primarily coming from older birth cohorts. I wrote a little blog about it here, but if you graph *changes* across survey waves by birth cohort, I think it is hard to back up the story that the NYT was selling.

6 thoughts on “Night Hawk

  1. I’ve never liked this “10 / 10 is the only acceptable rating” attitude. It reminds me of working for a Sears service center in my early 20s, and customers would be given satisfaction questionnaires, and all management cared about was how often we weren’t rated a perfect 10 on anything. A 9 was as good as a 4 to them.

    Or, similarly, when you fill out a survey after purchasing some product or service, and any time you don’t give the highest possible rating for a category, they ask you why not. “What can we do to get you to give us the highest rating possible?” I don’t know… complete and utter perfection is a silly thing to expect as a routine outcome. I don’t want to claim that I can imagine no better service than that which I just received. BUT – if you don’t give the highest possible rating, then some manager somewhere is going to be using against some lower level employee.

    If someone asked me how important it was to live in a democracy, on a scale of ten, I’d probably say 9. Just cos I don’t like sounding too certain about these things.

    • This can also lead to serious problems with cultural differences. I remember a report from a few years ago (I believe about the Kaiser system) where certain hospitals that served Asian immigrant communities struggled with their ratings because their patients tended, quite reasonably, to reserve 5s for exceptional service.

      • My university’s English Department said that perhaps Shakespeare would get an 85%, no one else got over 80% and that was “exceptional”.

      • I hope the hospitals identified the sources of the discrepancies quickly. Americans should not assume that the rest of the world share our expectation that mediocrity is unacceptable and perfection is the norm. But then I guess we aren’t giant fans of subtlety.

        To go back to the study in question, I can see why changes in the “How essential is democracy, on a scale of 1 to 10?” rating over time would be interesting. So report it in terms of means, or find a better way to plot changes over time. Why give so much importance to 10 / 10? Is the implied proposition that only those who say “10 / 10” really believe in democracy?

  2. Lots of issues with that NYT piece:

    “Venezuela, for instance, enjoyed the highest possible scores on Freedom House’s measures of political rights and democracy in the 1980s. But those democratic practices were not deeply rooted. During that apparent period of stability, Venezuela already scored as deconsolidating on the Mounk-Foa test.

    “Since then, Venezuelan democracy has declined significantly. In 1992, a faction of the Venezuelan military loyal to Hugo Chávez attempted a coup against the democratically elected government. Mr. Chávez was elected president in 1998 on a wave of populist support, and he immediately passed a new constitution that consolidated his power. His government cracked down on dissent, imprisoned political opponents and shredded the country’s economy with a series of ill-planned economic overhauls.”

    Argument by outlier meets spike-based trends. Venezuela is highly unusual along any number of dimensions with an extremely volatile and sometimes violent political history (including that “apparent period of stability” the 1980s – has the NYT blocked Wikipedia?). Ass an example it is less than worthless.

Leave a Reply to Ben Prytherch Cancel reply

Your email address will not be published. Required fields are marked *