Errata

Steven Levitt’s blog is great, but . . . shouldn’t it be Monica Das Gupta who deserves the hero treatment? Here are Das Gupta’s graphs:

fig3_missing_women.gif

fig2_missing_women.gif

This isn’t news at all–Das Gupta’s graphs came out at least a year ago. Shouldn’t the scientist who was correct all along–and published the data to show it–get more of the credit?

P.S. I published a false theorem once myself (and an erratum note a few years later, when the error was pointed out to me), but I’d hate to think this is “incredibly rare” behavior.

P.P.S. And many other errors get caught before publication.

9 thoughts on “Errata

  1. Arguing someone else is wrong is very common-place. As Levitt, states, what's more unusual is the author who was wrong to do another study, verify they are wrong, and then publish those results publically.

    Obviously the cost of doing a social science study is significantly larger than the cost of a retraction letter. That's what makes it rare.

  2. Jor,

    I don't buy it. First, Oster has been aware of Das Gupta's work for years and has many opportunities to retract but has not done so until recently. Second, I'm concerned about the moral hazard here. Oster makes a surprising and counterintuitive claim, and gets lots of credit for it, then, years later, retracts, and gets more credit. Meanwhile Das Gupta was right all along and had the data to show it. This just sets up incentives for people to be flamboyantly in error. To say that Das Gupta deserves less credit because all she did is get it right the first time . . . well, that doesn't make sense to me. That seems like celebrating the captain who wrecks the ship and apologizes, rather than the captain who steered around the iceberg and never crashed in the first place.

    Just to be clear, I'm not faulting Oster here: as I noted above, I've published mistakes too (including, embarrassingly, a mathematical error that had a simple counterexample). And I understand that, once she was committed to her claim, it took her awhile to step away from it (hence, not writing a note a year ago recognizing the strength of Das Gupta's evidence). As far as I can tell, she's been an honorable scientist, and I applaud her efforts. I just don't know that she should be celebrated more than the person who got things right (and with a snappy graph) a year ago.

  3. I think Andrew is right is his "flamboyantly in error" comment.

    We live in an era with an insatiable press appetite. There's a lot of bad science on the evening news [social science and health science in particular]. Obviously, it's easier to come up with a counterintuitive finding if you are either wrong or sloppy.[1] Do you deserve double credit?

    That said, Oster deserves some credit for being honorable.

    [1] Not saying Oster was sloppy; I don't know the work and can't comment. I'm commenting on the temptation to go to press before peer review.

  4. I think the other issue is the disciplinary thing. Oster is a Chicago economist who tried to overturn conventional public health wisdom by doing some econometrics and publishing in the Journal of Political Economy and was feted for it. That's not the normal way this reseach is done. Das Gupta is a demographer/anthropologist who looked at the problem by doing some old-school demography and anthropology.

    There's a moral here. Conventional wisdom is normally what it is for a reason, and you should probably get a thorough grounding in the field to understand why, and do a bit more that run some regressions, before you going around saying people have got it all wrong.

  5. Andrew – Oster did interact with Das Gupta's work back in December 2005 (see here). In essence, Oster says she recognizes son preference plays a role (Das Gupta's main point), and acknowledged interestingly that her original paper couldn't explain the increases in sex ratios following the availability of sex selective abortion technology. Her work originally stopped short, in the time series, as to when those technologies became available (FWIW, though, I think Sen originally claimed the missing women wasn't due to birth imbalances, but to systemic female maltreatment that increased female mortality up to adolescence and beyond. Hepatitis B would be focused just on birth imbalances, though), so it was always an earlier period of time in which the abortion technologies would have been unavailable. I think Oster originally dismissed Das Gupta's findings as being more likely due Das Gupta focusing on periods when those technologies would've been more present, and Oster thought her work – by focusing on an earlier period – had sufficiently controlled for that problem. (Whether she did is another matter, and I've not read Das Gupta's work in over 1-2 years, so I cannot remember).

    Oster then has a curious way of dealing with Das Gupta's approach. Das Gupta says the patterns observed in the data leave little room for a biological explanation, since the sex ratio imbalances are so correlated with the sex compositions of the family. Basically, the graphs you posted. To this, Oster wrote, "Put simply, this confuses marginals and averages. … Two countries may have different levels in the sex ratio at birth, but income constraints or parental preferences could still move them around within a country. It is worth noting, of course, that if I had claimed that hepatitis B explained al l of the sex ratio imbalance then the evidence on incentives would be more
    problematic. Since this is not the claim made in Oster (2005a,b), there is no reason that both factors could not be acting simultaneously. … The key to thinking about the relative potential of culture and biology to explain the over-representation of men is understanding that marginal effects may be seen to operate and still tell us relatively little about the average. In the end, it seems perhaps better to think of these two explanations as complementary."

    In essence, my reading of Oster's reply to Das Gupta's work back in 2005 was that Oster took seriously Das Gupta's criticisms, or at least seemed to. She acknowledges the ground that Das Gupta puts forward on son preference in explaining the variation, but believes at the margin, the biological factors still play a role.

    BUT, at the same time, I think what works against Oster here is that in her original article, the magnitudes are very large, making the marginal/average distinction a little more problematic. She claims she can explain as much as 75% of the variation in the sex ratio in some regions (India, I think) and as low as 25% in other regions (China, I think [I may have these backwards]) by HepB. Not all the variation, but also not a small amount either.

    All that said, I think one thing that this illustrates to me is that the media does a lot of hyping around academic research, and necessarily truncates or simplify to the point of caricature what a researcher has done. For instance, the "reality checks" argument – that economists can't add anything to this if they don't first get permission that their hypotheses are sensible from medical professionals – is one I'm having trouble swallowing. Some of that seems like turf war mentalities. Also, it eclipses the real differences between disciplines. Much of what the economist brings to the table is methodological tools that help establish causality with observational data, whereas medical professionals work exclusively with randomized trials. There are many questions we have in which the only data we have is observational, and thus the social sciences may have a comparative advantage.

    Ultimately, science has progressed through this Oster debate. How was her original paper "flawed" exactly? No coding errors, no fraud that we know of. Just a hypothesis and an empirical test of that hypothesis. More empirical tests and we ultimately rejected the hypothesis. Personally, I think there's value in rejecting hypotheses – even more than finding evidence for the hypothesis themselves. The more things we know are not at play, the more confidence we have about the things left on the table.

  6. Scott,

    Yes, I'm aware of Oster's earlier discussion of Das Gupta's work–follow the links in my blog entry above. Beyond this, I'm not saying that Oster should be punished for making a mistake, I just think that Das Gupta deserves more credit for getting things correct (and showing it in clear graphs). Oster was an exemplary researcher, and Das Gupta, in retrospect, appears to me to be more exemplary.

    I agree with commenter Alex above. Oster made a big deal of saying that Das Gupta was wrong, then Oster comes back and says, no, Das Gupta was right all along. So why does Levitt write only about Oster? Sure, she's an economist and so is Levitt, but, still . . .

  7. I didn't see your links to Oster's interaction with Das Gupta, so apologies I wrote such a long comment. Did Oster make a "big deal," though, in saying that Das Gupta was wrong? Was that in the original JPE hepb paper, or in the response to Das Gupta? Not saying I disagree, but her response seemed to me to be trying to say that the Hepb virus hypothesis and the son preference hypothesis were not mutually exclusive, and that both she and Das Gupta could both be right. I didn't take her as making a big deal, as much as trying to smooth out the differences, for whatever that's worth.

  8. The irony, of course, is that Steven Levitt himself became famous to the public in 1999 for his slapdash abortion-cut-crime theory, which has since taken a quite a beating at the hands of observers. Levitt, however, has merely dug in, and even defamed one of critics (as he was forced to admit to settle economist John Lott's lawsuit).

Comments are closed.