Katherine Rundell quotes Samuel Taylor Coleridge writing “his central Principle of Criticism”:

never to lose an opportunity of reasoning against the head-dimming, heart-damping principle of judging a work by its defects, not its beauties. Every work must have the former – we know it a priori – but every work has not the latter, and he, therefore, who discovers them, tells you something that you could not with certainty, or even with probability, have anticipated.

This sounds exactly right!

But then it makes me wonder what I’m doing all the time here, criticizing science papers, popular science writing, etc.

For example, recently Nick Brown and I wrote a paper, How statistical challenges and misreadings of the literature combine to produce unreplicable science: An example from psychology, that was centered on the criticism of an article published in a scientific journal. The negative was that this published article made several strong claims that were not supported by the data that were presented, and they also made several other strong claims that were not supported by the literature they were citing. That was bad! But, as Coleridge says, every bit of writing has defects, so what value did Nick and I add by pointing these out? Wouldn’t we have been more effective critics by writing about all the good things in that article? Or, if we couldn’t find anything nice to say, saying nothing at all and instead devoting our effort to praise and exploration of the good?

For another example, I recently slammed pop-science impresarios Steven Levitt and Sean Carroll for hyping the aforementioned preposterous claims. Was that a good idea for me to do this? As Coleridge might have said had he lived in our time, all podcasts have their defects. The real gain is from pointing out their defects, no? And I do imagine there are good things to say about Levitt’s and Carroll’s podcasts . . . someone who wants to go to the trouble of actually listening to them can perhaps inform us of their virtues.

But . . . I do think we’re contributing something by our critiques.

Why is that? How can I agree with the above Coleridge quote and then go back to my usual routine of sarcasm, criticism, and indignation?

My quick answer: literary criticism and scientific criticism are different. Literary criticism is about the work being criticized; scientific criticism is about the aspects of the real world addressed by the science.

To put it another way, in our criticism of that published article, Brown and I were not saying that the article, as an artifact in and of itself, was poorly constructed, nor were we pointing out flaws in the article in order to shoot the article down. Rather, we were expressing skepticism–and giving our reasons for that skepticism–about the article’s claims of evidence regarding a real-world hypothesis (in this case, something about mind-body healing). Similarly, my criticism of Levitt’s and Caroll’s podcasts was not that they were poorly-made podcasts. Rather, I was annoyed that the podcasts were lending their credulity to unsupported claims about the real world. Indeed, my problem with the podcasts was that they were insufficiently critical: they were approaching the scientific work in question with a Coleridgian sense of wonder . . . which led them to give some inappropriate partial endorsement to some unsupported and highly speculative claims about the real world.

It strikes me that this general issue causes a lot of confusion, with people such as Nick Brown and me treating science as a series of statements about evidence and reality, in contrast to some insiders such as Levitt, Carroll, and various tenured members of the science establishment treating science as a series of beautiful careers to be nurtured, not criticized.

I think this distinction is important! Not that I would expect Levitt, Carroll, or the Nudgelords to agree.