Skip to content
 

All maps of parameter estimates are (still) misleading

I was looking at this map of coronavirus cases, pondering the large swaths with seemingly no cases. I moused over a few of the gray areas. The shading is not based on counties, as I assumed, but on some other spatial unit, perhaps zip codes or census blocks or something. (I’m sure the answer is available if I click around enough).  Thing is, I doubt that all of the cases in the relatively low-population areas in the western half of the country are concentrated in those little shaded areas. I suspect those are where the tests are performed, or similar, not the locations of the homes of the infected people. [Added later: Carlos Ungil points out that there was indeed a link, just below the map, that says “For per capita: Parts of a county with a population density lower than 10 people per square mile are not shaded.”] 

I’m well aware that all maps of parameter estimates are misleading (one of my favorite papers), but I think the way in which this map is misleading may be worse than some of the alternatives, such as coloring the entire county. Yes, coloring the whole county would give a false impression of spatial uniformity for some of those large counties, but I think that’s better than the current false impression of zero infection rates in a large swath of the country. In terms of cases per 100,000 Nevada is much worse than Ohio but it sure doesn’t look like that on the map. [Note: I originally said ‘Illinois’ but either that was a mistake, pointed out by Carlos Ungil, or it changed when the map was updated in the past hour].  

 

Many western states appear to have low case rates but actual rates are not low

27 Comments

  1. Carlos Ungil says:

    > (I’m sure the answer is available if I click around enough).

    The answer may have been just one click away, under the “About this data” arrow right below the map and the source.

    “For per capita: Parts of a county with a population density lower than 10 people per square mile are not shaded.”

    > In terms of cases per 100,000 Nevada is worse than Illinois

    No, it isn’t. At least of you mean in the last week to be consistent with the map you show.

    • Greg Jensen says:

      > > In terms of cases per 100,000 Nevada is worse than Illinois

      > No, it isn’t. At least of you mean in the last week to be consistent with the map you show.

      According to the table that goes along with this map (i.e. the last 7 days as of October 21st), Nevada has 2,976 cases per 100k, as compared to Illinois’ 2,810 cases per 100k. This seems to be getting driven largely by Clark County, which at 3,348 cases per 100k is doing worse than all but 6 of Illinois’ counties. However, those worse counties (such as Union County IL, which only had 10 cases a day but has a relatively small population) are *dense* enough that nearly all of them are getting colored on the map.

      • Bob76 says:

        Hmm. That’s not what I see. I read

        Total Per 100KPOP Last 7 days Last7/100K
        Illinois 359K 2,840 26,991 213
        Nevada 92K 2,989 4,709 153

        The numbers you quoted appear to be for the total cases, not the last 7 days.

        Bob76

      • John Williams says:

        I can see an argument for putting the shading where the people are, even though privacy considerations generally keep health departments from being very specific about the location of the cases. For example, where I live in Humboldt County, CA, the cases were reported only by north, central, and south county for a long time. Recently, the data are given by zip code, but only if there are six or more. In my zip code, the health dept says only 0-5.

        All of this just underscores Andrew’s headline about maps of parameter values being misleading.

    • What’s the link to the original map?

    • Phil says:

      Thanks Carlos, I’ve corrected the post. Perhaps I was looking at the wrong column or something.

  2. david P says:

    I can’t speak for other areas, but I grew up in Tucson, and a lot of that gray area east is Tucson really is very sparsely populated. A big chunk of it is a bombing range, another chunk has the Cabeza Prieta and Organ Pipe National Monuments (or National Parks?), and another large chunk is the Tohono O’odham reservation, which is inhabited, but sparsely. Remember, it’s DESERT.

    I think something like 75 percent of Nevada is federal land.

    • Phil says:

      I know there are huge areas of those states that are very sparsely populated; I’ve driven across several of those states several times and flown across them many times. But this is not a map of population, it is a map of COVID case rate.

      Maps are used in more than one way. Obviously they are intended to show the spatial variation of a parameter, that’s why people use a map instead of a table. But they are often also used as lookup tables of a sort. In present circumstances, states are important spatial units because state governments set rules for dealing with the pandemic. Looking at this map, one sees at a glance that Wisconsin has a very bad outbreak, Missouri and Illinois and Tennessee are pretty bad, and a lot of other states are more or less the same. A savvy viewer may also recognize that Iowa, South Dakota, North Dakota etc. are very bad too, maybe worse than Wisconsin…or maybe they are having bad outbreaks in their towns but the rural areas are completely unscathed? It is literally impossible to tell, because a rural area with 0 case rate is indistinguishable from one with the highest case rate in the country.

      I think that if you show this map to randomly selected people — not readers of this blog — very few will realize that, say, New Mexico has a worse outbreak (in cases per 100K) than Tennessee, or that Nevada is worse than Ohio.

      • Dale Lehman says:

        If the interpretation is going to focus on comparing states, then why not shade the entire states? Sure, that won’t work for many purposes, but one map is not likely to work for all intended uses. Decide first what comparison you are trying to make, and make the map address that – and include the caveats warning against (mis)using it for other purposes.

        • Phil says:

          Coloring entire stated would give a false impression of spatial uniformity, especially in large states with big spatial differences.

          All maps of parameter estimates are misleading. That’s just the way it is. But they are not all equally misleading. I think this one makes some bad choices by using “0 or low” to mean both low rate and low population density at the same time. I’m not saying any fix would be perfect.

          • Carlos Ungil says:

            Assigning colors to states is the only way to get a map that is not misleading if you want to show that New Mexico has a worse outbreak than Tennessee, or that Nevada is worse than Ohio.

            Coloring entire counties rather than the populated sub-areas you would still get misleading maps because the information in the map would be the same and it just wouldn’t be enough to calculate state-level numbers.

          • Phil, what would you think of a map that used a grey color for regions with less than a certain population density, and a very light yellow color for above a certain population density but below say 8/100k or whatever the current low is exactly?

      • jim says:

        I guess that argument just doesn’t hold for me.

        One thing that’s really galling me in the modern “public discourse” environment is the concept that the writer or creator of an article or graphic is somehow responsible for the misinterpretations of people who don’t bother to read it. I’m sorry but if people don’t read an accurate graphic correctly it’s not my problem. Well it is my problem but it’s not a graphic production problem. It’s a too many lazy people problem.

        It reminds me of:

        Many years ago (2000?), my dad had gout in his ankle. The doc gave him an ankle support to wear. But did he get paper instructions on how to put it on (it was plainly obvious with no instructions)? No. He got a video. His insurance was billed $800 for the video, because some people are too lazy to read.

        FF to 2020. Recently I read an article by an “Oncological Physical Therapist”. Apparently people recover from cancer better when they exercise. Who would have guessed? But today, a $0.02 printed page of how to exercise is not sufficient to motivate the poor cancer victim. Even the $800 video isn’t enough. Not even a series of $800 videos is enough. No: today the poor cancer victim is so challenged to do anything they need an individual PhD therapist to help them touch their toes for an hour a week.

        Where does it stop? When do people start taking responsibility for their own behavior again?

        • Phil says:

          “You don’t want a map to show 0 rate in areas where the rate is actually high? You must want people not to be responsible for their own behavior.”

          OK, man. You be you.

          • jim says:

            My point is that people should just read the map rather than complain that it “made” them misunderstand or tricked them into some misunderstanding. There’s nothing wrong with this map at all. Maps have legends. People are supposed to read them.

        • Martha (Smith) says:

          Jim said,
          “Recently I read an article by an “Oncological Physical Therapist”. Apparently people recover from cancer better when they exercise. Who would have guessed? But today, a $0.02 printed page of how to exercise is not sufficient to motivate the poor cancer victim. Even the $800 video isn’t enough. Not even a series of $800 videos is enough. No: today the poor cancer victim is so challenged to do anything they need an individual PhD therapist to help them touch their toes for an hour a week.”

          It sounds like something is left out of this, starting at “Apparently”. Did the article explicitly say “people recover from cancer better when they exercise” or “the poor cancer victim is so challenged to do anything they need an individual PhD therapist to help them touch their toes for an hour a week” Did it really say the Oncological Physical Therapist is a Ph.D.? The answers to these are not included in your comment.

          My understanding is that many programs for training physical therapists are now Doctor of Physical Therapy programs. My understanding is also that physical therapy (perhaps called a rehab program) is not just motivating the patient to exercise, but involves exercises aimed at the type of cancer and the particular treatment for the cancer — especially if it involved surgery — and the particular physical side effects of the treatment that the cancer patient is experiencing.

          • Anonymous says:

            “My understanding is also that physical therapy (perhaps called a rehab program) is not just motivating the patient to exercise, but involves exercises aimed at the type of cancer and the particular treatment for the cancer — especially if it involved surgery — and the particular physical side effects of the treatment that the cancer patient is experiencing.”

            Those can’t be explained in a book?? WTH are we sending everyone to college for?

            • Martha (Smith) says:

              Anonymous said, “Those can’t be explained in a book?? WTH are we sending everyone to college for?”

              Many people who need rehab exercises do not have the detailed knowledge of human anatomy and physiology to understand the needed explanations. The explanations would require a lot of 2-dimensional figures of things that are three-dimensional. This is a situation where in-person training is much, much more helpful for most people than reading a book. The physical therapist can do things like “show” positions and movements in three dimensions, and physically correct the patient’s first awkward attempts to assume the positions. A physical therapist can also make alterations for different bodies — for example, adjust the weights used to be appropriate for the individual; adjust the angles to take into account the patient’s individual anatomy (e.g., some patients are more flexible than others; some have different ratios of body parts.) A physical therapist can observe how a patient does everyday activities that involve movement, and give individual corrective advice if the patient’s habits are leading to misuse of the body. Physical therapy really needs an individualized and in-person approach, because different people’s bodies can be so different. Also, some exercises might require special equipment.

              What Anonymous proposes would be analogous to trying to teach someone to be a ballet dancer or opera singer by reading a book. It just doesn’t work that way.

  3. jim says:

    “but I think that’s better than the current false impression of zero infection rates in a large swath of the country.”

    The current coloring seems fine to me. There’s nothing but a few ranches between Burns OR and Winnemucca, NV over several hundred miles. Really it doesn’t seem deceiving to me at all. To me it’s obvious that the uncolored areas have no data for one reason or another. I guess it helps that I’ve driven across the western US several times, so I didn’t have to guess what the blank areas were all about.

  4. Yuling says:

    Presumably an alternative visualization is to map only the spatially-smoothed results in which there is some estimate even for low population density regions. Also since the general audience tends to directly interpret “hot-spot” as enhanced personal level risks, it would be even better to directly estimate that quantity from raw data.

  5. I often use maps as a metaphor for theories and modeling in my courses. All maps, not just data maps, are also wrong. Even Google Maps, which is getting really good, is not going to have everything you need alone to predict your trip. Sudden driving events, potholes, bathroom and food breaks, and other things may all delay you, but it’s pretty damn good regardless.

    The metaphor was way better prior to google maps, but it still holds.

  6. Tom says:

    Here’s one reason why the results of this kind of mapping are suspect: https://medium.com/@uwdata/surprise-maps-showing-the-unexpected-e92b67398865

Leave a Reply