Mike Spagat writes:
You’ll definitely want to see this interesting paper by Kristian Gleditsch.
Research and Politics, a journal for which Kristian Gleditsch is one of the editors, has hosted several valuable rounds of post-publication peer review.
One instance starts with a paper of mine and Stijn van Weezel which replicated, critiqued and improved earlier work on excess deaths in Iraq. This was followed by a critique or our critique by the authors of the original work and a rejoinder from us.
Another is a paper by Silvio Rendon that critiqued war-death estimates for Peru made by Peru’s Truth and Reconciliation Commission and proposed alternative estimates that substantially change our understanding of the war. This was followed by a response from two authors of the Commission’s statistical work and a rejoinder from Rendon. I’ve blogged rather extensively on this discussion.
An interesting aspect of the Peru debate is that the rejoinder is probably the most interesting and penetrating piece in the series. It was only with this piece that the really low quality of the Commission’s work was properly exposed. They were, for example, using perfectly fitting models to extrapolate beyond their data. It took the whole back and forth before we got to this.
I have not followed all the links, but I appreciate the general point of having thorough discussions with multiple exchanges.
Also, the bit about the Truth and Reconciliation Commission reminds me that we shouldn’t automatically accept conclusions just because they come from what sounds like an official source. Remember the problems with that Organization of American States report on the Bolivian election? The OAS sounds so respectable. But, no, they’re just people like any of us.
I guess we have to pay for access to Kristian’s paper.
Hi. Here’s an ungated version of Kristian’s paper:
http://repository.essex.ac.uk/27923/
Thank you so much. I’m interested in this topic b/c we discussed it in various international relations/security fora, including the United States Institute for Peace. I believe then the Lancet study of the deaths in Iraq was much in the fore. So I’ll pay closer attention to the articles.
Why do they always expect people in Houston to fix things?
Remember the Alamo!
That’s in San Antonio!
I forgot.
According to history, Houston did fix the problem caused by the Alamo. First, the decisive battle that won Texas its independence was fought within the present-day Houston metropolitan area. Second, the Texans were led into battle by a gentleman named Sam Houston.
Maybe Houston IS the answer to all our problems.
Andrew raises a good point.
We’re accustomed to fallacious appeals to authority based on publications in peer reviewed journals with their concomitant dismissal of criticism as “not peer reviewed” or “just blog posts”. But what happens when a Truth Commission which implicitly, if not explicitly, claims to reveal the Truth gets things wrong?
Yeah, pre-publication peer review can be an useful filter to avoid the most fringe/crackpot/etc stuff, but shouldn’t IMO be treated as nearly as much of a Guarantee Of Sound Science (or critical “sine qua non” of science…) as it often is.