In criticism of criticism of criticism

I do a lot of criticism. I’m sure you can think of lots of things that I like to criticize, but to keep things simple, let’s focus on graphics criticism, for example this post where I criticized a graph for false parallelism.

At this point some people would say that graphics criticism is mean, and not just mean but counterproductive, that we should be more constructive and less critical, you catch more flies with honey than with vinegar, etc. Let me call this criticism criticism.

If I’m in a quippy mood, I’ll reply that maybe you can catch more flies with honey than with vinegar, but you can capture even more flies with poop!

More seriously though, here’s an advantage of criticism that we don’t always hear about, which is the positive way in which criticism can shift the discourse.

The audience for criticism is not just the direct objects of the criticism, it’s also the community more generally.

Take the above-linked example. It may be that Dan Kahan, the person who pointed made this graph, will be motivated to think harder about his goals when plotting data. Or maybe my negativity will just send Dan into a defensive crouch. Or, perhaps most likely, he’ll find my criticism to be pretty much irrelevant to his larger goals. That’s fine. But I hope that other readers of this post will be made aware of the Gricean messages being sent implicitly by various choices in a graphical display.

In political science, I proceed on two tracks: I criticize graphs I don’t like, and I make graphs following my own principles. The criticism I do, makes me more aware of my goals, of how to do better. And, in the field of political science, sure, there are some people who think it’s funny that I’m always there to criticize a graph. But, in the past 10 years, I’ve been seeing more and more intense, informative graphs coming from researchers in that field. It’s taken awhile, but I’m seeing forward movement. I think criticism is part of this. Criticism makes us aware of our goals and what we need to do to get there.

So I’m a critic of criticism of criticism. And this post is an exercise in criticism criticism criticism.

P.S. In graphics, as in all fields, our criticism should come with an effort to understand the context of the behavior we’re criticizing. For example, when Antony Unwin and I wrote about infovis and statistical graphics. Typically the behavior we don’t like is done in service of some goals, and it’s a good idea to try to understand these goals. Criticism is a valuable part of this process.

13 thoughts on “In criticism of criticism of criticism

  1. It is my understanding that one catches flies a lot more effectively with vinegar than with honey, despite what the saying suggests. I am not sure what this means for statistical criticism.

  2. That post on my graphics was *mean*? I missed that completely. The time I gave a workshop & you grabbed your throat & tipped back in your chair & made that loud retching sound when I showed 58-bar bar chart–now that was mean! But very constructive

    • When does mean mean?

      Is this criticism of criticism mean? or just a criticism of criticism of criticism?

      Some of my past experiences.
      At one hospital I worked at, I told someone in human resources that an important part of my job as a statistician working in clinical research was to be critical, they contacted my boss and offered to help him re-mediate me. He actually told them to F!!! Off.

      In clinical epidemiologist rounds, I pointed out to a surgeon that step wise regression did not resolve the need to worry about confounding. He replied that it was to date the most advanced analysis of the data and I had no right to comment and loudly concluded with “I will never work with you again”. When my boss heard about this, he told the surgeon that he had to write a paper with me on analyzing observational studies. We started, but I left before we finished, but it was going well.

    • That’s a nice piece, basically … if you are going to criticize it is totally fine for people to criticize your criticism.

      The sad thing to me about all of this is that you hardly ever see data on how readers actually read these visualizations. I mean I know what misleads me at first glance, and I also know something about what is confusing to my students. But I’m not really sure we really know much about the rest or about all the different dimensions on which there can be problems (e.g. the circle lines with different radii, the issue Andrew raised about false parallelism, the problem of using two different y axes).

      • Actually, psychologists into perception do know some stuff about this, that has never made it to the stats world except in Isabel Meirelles’s excellent book Design For Information (which I should really get commission on, or maybe stockpile them and sell them online)

Leave a Reply

Your email address will not be published. Required fields are marked *