Skip to content

Chris Schmid on Evidence Based Medicine

Chris Schmid is a statistician at New England Medical Center who is an expert on evidence-based medicine. I invited him to present an introductory overview lecture on the topic at last year’s Joint Statistical Meetings, and here are his slides. All 123 of them. I don’t know how he expected to go though all of these in an hour. You could teach a semester-long course based on this material.

Good stuff, I recommend you all read it.


  1. numeric says:

    Biologists always cram a huge number of slides into short presentations. This is how they think, also. A deep theory relating number of slides to modes of inquiry awaits elucidation.

    • C Ryan King says:

      I think one-figure one-point is slightly more common in biological science, where cleanly designed experiments to get at one factor are possible and desirable. The statistician will complain about wasted white space, and the biologist overloading a figure with too much data. Putting lots of text in slides (which translates into many short slides) is more a function of how busy you are (it takes less practice time) and the use of your slides: with lots of text they turn directly into handouts / emails, with little text they’re cryptic and useless as a stand-alone.

  2. K? O'Rourke says:

    Yes, and what wide and clear coverage!
    (In just 123 slides)

    Some minor quibbles for Chris:

    1. Slide 7 – for pathophysiological rationale I would rather say it should strongly encourage clarification and “testing” of this as that is required for any sensible and sensitive statistical (meta-) analysis.

    2. Slide 35 – historically there was a “gruesome” change in meaning, especially in the US. Before about 2000, “Meta-analysis” meant systematic review in North America and then the Brits lead the change to distinguish “Meta-analysis” as just the statistical part of a “Systematic review”. This makes reading the earlier literature troublesome and open to mis-understanding as when I was said meta-analysis of qualitative data would be fine, and every thought I was nuts and would not say anything.

    3. Sides 103-104 – it is important for people to realize that in these indirect comparisons there is/are curves or function of parameters that do NOT come from data models for something somewhere in the study data sets. As such these arguably need to called priors and peer reviewed as such 9and informative!). Thinking it has all come somehow from the studies and the only assumptions required need to be assessed in those studies has lead to mis-understandings, especially when non-inferiority analyses are based on such methods. Mengerson and Wolpert have written about this in a paper with the (unfortunate?) term “adjusted likelihoods” in it.

    But again very well put together “one hour”.

    Also (yes its getting long, and I am past my 15 minutes) it has been 25 years since my first kick at the can here with the L’Abbe and Detsky paper and a few thoughts.

    In discussing systematic reviews/meta-analysis with some statisticians recently we seemed to agree that it is a very important area but not very suitable for statistical research – mainly because the data model is just too weak.

    This also helped me (recently) better understand Ingram Olkin’s (and other approaches) which focus primarily on thoughtfully combining p_values. This could be understood as realizing sensible likelihoods (data models) with something common (the same fixed or random parameter) are really out of reach in many applications and one should only suggest what studies had in common was the null was (approximately) true or the alternative was (sometimes) true and always in the same direction (one-sided) or in different directions (two-sided).

    But until the data model is enriched by addressing the selective reporting of studies (i.e. not publishing non-significant studies) its large much about understanding we learn little from the published literature.

  3. C Ryan King says:

    cd Documents/classes/EBM/courseNotes2010/
    pdftk *.pdf cat output ~/Documents/Presentations/JSM_slides2011.pdf

    (I kid)

  4. Alan says:

    Some background information:

    Even though EBM and its goals are often presented in a vague manner by its founders the movement has in fact developed a very specific methodology for assessing the evidence presented by conventional medical statistics. It doesn’t replace these tools, it builds on them, in a meta analytic fashion. but it is different from conventional statistical meta analysis. The need for this additional layer arises because there may be a very large body of literature on any given medical topic, far too much for an individual practitioner to review.

    Instead of an individual expert reviewing this information and writing a review article which summarizes his or her opinion on the matter, the EBM methodology is intended to be applied by a whole team of experts who classify and rate all the different studies in the literature which relate to a problem, and assign levels of evidence according to the type of study. For example, in one scheme Level 1 is randomized clinical trials or meta- analyses of multiple clinical trials with substantial treatment effects. The lowest Level 8 is rational conjecture (common sense); common practices accepted before evidence-based guidelines.

    It is possible to be rather specific in assessing the level of evidence generated by any given study because there is now a large body of experience which confirms how often the conclusions of a given type of study are later independently verified. In other worlds, a track record for a given study type. EBM was initially developed at McMaster University, Canada, building on the earlier work of the Cochrane Collaborations around the world:

    There are now quite a few EBM centers around the world, with a major one at Oxford.

    A couple of unusual findings which bear upon EBM and its levels of evidence: the low level attributed to basic biological conjecture, and the fact that observational studies are often not confirmed by follow on interventional studies, implying that the true causal factor has not been identified.

    Though much of EBM is very specific to medicine, there are many conclusions which can be drawn which impinge on the whole of statistical practice and the philosophy of science. The molecular biology revolution of the 1950s has also finally made it possible, at least in principle, to develop fully reductionist scientific explanations in biomedicine ranging from the quantum mechanics of hydrogen bonding all the way up to cultural anthropology.

    To give an example of how medical thinking itself has been changed by EBM, I present the JAMA book of basic medical practice:

    The great emphasis on randomized clinical trials in EBM is a testament to the original work of Sir Ronald Aylmer Fisher on randomized experiments. In fact, one may say that EBM is Fisher on steroids :-)

    • K? O'Rourke says:


      I am sure there are some true things and some new things here but I am less unsure about the over lap.

      Now as I’ll indicate later, I can be accused of the same, but your post _seems_ overly certain and mockingly definitive.

      It is too soon for anyone to do serious history and it should not be done by those involved (at least unless they won a real war) but Ian Chalmers talked me into doing such here

      I did have some overlap with Dave Sackette at McMaster and Ian Chalmers at Oxford.

      p.s. the link I gave to Fisher is very likely wrong, its unimaginable that as an Cambridge scholar he would not have read Airy’s book, but most of us think there is no records of that. I did bring that to the attention of those (actually in a History Department) who made the claim and they assured me they would address my concerns. Last time I checked, about a month ago, they had not changed anything. What do people think I should do?

  5. […] that recently you shared some thoughts of yours and others on meta-analysis. I especially liked the slides by Chris Schmid that you linked from your blog. A large portion of my work deals with meta-analysis and I am also […]