Reprint of “Observational Studies” by William Cochran followed by comments by current researchers in observational studies

Dylan Small organized this discussion in the new journal, Observational Studies. Cochran’s 1972 article is followed by comments from:
Norman Breslow
Thomas Cook
David Cox & Nanny Wermuth
Stephen Fienberg
Joseph Gastwirth & Barry Graubard
Andrew Gelman
Ben Hansen & Adam Sales
Miguel Hernan
Jennifer Hill
Judea Pearl
Paul Rosenbaum
Donald Rubin
Herbert Smith
Mark van der Laan
Tyler VanderWeele
Stephen West.

My discussion is called “The state of the art in causal inference: Some changes Since 1972.”

Cochran’s article and all the discussions are downloadable in a convenient pdf here, at the journal’s website. Lots to chew on.

6 thoughts on “Reprint of “Observational Studies” by William Cochran followed by comments by current researchers in observational studies

  1. I followed Norman Breslow’s comment to the 1964 Surgeon General’s report[1] and looked at figure 1 of chapter 8 (page 95 of the linked pdf) as shown as attached Fig1 here. I believe there is an error that may substantially change the conclusions. There is a point plotted for non-smokers in between the 72.5 and 77.5 age groups. It is highly unlikely that was on purpose. The age values are the means of 40-45, 45-50, etc which is the usual method of aggregation. The data for 42.5 years old is also not plotted, therefore I believe the data should be shifted left for the non smokers (Fig2) so as to use all the same age groups. The report describes the figure:

    “If the lines for cigarette smokers and non-smokers were parallel, this would imply that the mortality ratio of the smokers to the non-smokers was constant at all ages, because the vertical distance between the two lines at any age is the log of the mortality ratio for that age. In Figure 1, however, the slope is slightly less steep for the cigarette smokers than for the nonsmokers. This indicates that the mortality ratio is declining with increased age.”

    In Fig2 we see that the curves do appear parallel, or that. Whether or not that is the correct fix, I am nearly certain there is an error in that figure and possibly the corresponding fit.

    Fig1: http://postimg.org/image/k1klvxp7d/
    Fig2: http://postimg.org/image/k98ta7pwv/

    [1] United States Surgeon General’s Advisory Committee Report (1964). Smoking and Health.
    U.S. Department of Health, Education and Welfare, Washington D.C.
    http://profiles.nlm.nih.gov/ps/access/NNBBMQ.pdf

    • Typo: “In Fig2 we see that the curves do appear parallel, or that.” This sentence should say that the difference is even increasing, although the ratio remains about the same.

      Also, we can look at overall mortality data from 1964 (See Table 1-3. Death Rates by Age, Color, and Sex: United States, 1960-69 on page 14 of the VSUS 1969 report). This data is consistent with my “fixed” version of the chart, but using the data as shown in the Surgeon General’s report the non-smoking population has much lower mortality rates for all ages up to ~75 years old (see Fig3). There are some issues with different age groups and effects of aggregation over different years, but I am now extremely confident that chart is in error.

      Fig3: http://postimg.org/image/uc1izist5/

      Vital Statistics of the United States, 1969. Volume II, Mortality, Part A. 1974. 489 pp.
      http://www.cdc.gov/nchs/products/vsus/vsus_1965_1979.htm

  2. I’ve only scanned the comments, but it’s interesting that Cox and Wermuth interpret Cochrane as emphasising the importance of data generating mechanisms, whereas you and Pearl seem to disasgree.

    Meanwhile, any thoughts on Pearl’s claim to have solved the problem of external validity? (p202)

  3. brian,
    My reading of these comments is different. All three commentators emphasize the importance of data
    generating mechanism but in different ways. For Cox and Wermuth the need to reason about mechanisms
    is an excuse to give up on causal inference altogether. see http://www.mii.ucla.edu/causality/?p=1545
    For Andrew Gelman, Rubin’s conceptualization of ignorability and Angrist and Pischke’s quest for instrumental
    variables are examples of reasoning about mechanism. For Pearl, reasoning about mechanism means representing
    mechanisms mathematically and transparently and learning how to draw causal conclusions from such representations.
    For him, “ignorability” is a cognitively formidable construct, while “instrumental variables” are just a tiny tip of
    the iceberg of modern methods of causal inference.

    As to Pearl’s claim to have solved the external validity,I doubt you could convince any statistician to comment on such
    questions, because that would require explicit reasoning about mechanisms; “ignorability” is simply insufficient.
    (see http://ftp.cs.ucla.edu/pub/stat_ser/r452.pdf )

Leave a Reply to brian Cancel reply

Your email address will not be published. Required fields are marked *