Skip to content
 

You may not be interested in peer review, but peer review is interested in you

Here’s an ironic juxtaposition from Tyler Cowen’s blog.

On 28 Apr he discusses a paper with a market system for improving peer review and concludes, “Interesting, but the main problem with the idea is simply that no one cares.”

The day before, his assorted links featured “Frequency of Sex Shapes Automatic, but Not Explicit, Partner Evaluations,” which pointed to a Psychological Science paper that was just full of forking paths, arbitrary data selection and analysis decisions, comparisons of statistically significant to non-significant, characterization of “p less than .1” as “marginally significant” . . . the works. The full Psych Sci monty.

The point is that, as long as generally savvy opinion leaders such as Tyler Cowen trust peer-reviewed publications on sight, we can’t afford not to care about peer review.

5 Comments

  1. ZHD says:

    Is the title of this post a throwback to a certain anonymous cartoonist?

    • numeric says:

      It is of course an allusion to Trotsky’s comment (substitute dialectic for peer review). There’s a lot of similarity between dialectical materialism and peer review, IMHO. And academia in general is very Soviet in its operation, with small collegiums acting as gatekeepers and deciders for not only membership (tenure!) or “relaxation to the secular organs” (adjunct status or even worse, industry).

  2. Moreno Klaus says:

    I don’t think “Tyler Cowen trusts peer-reviewed publications on sight”, I think he posts it more as a curiosity. His blog is more meant to be entertaining (i.e., cocktail conversation generator) than rigorous, while once in awhile promoting his right-wing views. It is also virtually impossible to be rigorous when you post so many things every day.

  3. Joseph says:

    It’s not a small problem. Peer review is often laborious, so it feels like there is a lot of work involved in making reviewers happy. I have had easy reviews, but not many. That tends to make us trust people who have gone through the effort of shepherding a manuscript through the process. But if the process focuses on the wrong points then it’s not helping anywhere near as much as we think.

    • Andrew says:

      Joseph:

      Yes, there are big issues. I was just frustrated with Cowen casually dismissing interest in peer review. He may be right that (almost) “no one cares”—but, if that’s so, it’s a problem. Cowen himself should care, a lot, given that he’s relying on peer review to decide what articles are worth forwarding to his audience.

Leave a Reply