What to teach in a Bayesian data analysis course

Gal Elidan writes:

I am starting as a faculty next year in the statistics department at the Hebrew University, Israel. As it may be interesting to both the computer science and statistical community, I plan to give a course a course next year on Bayesian data analysis. My (still in its early stages) plan is to give a course based on your book along with some relevant topics/applications that have seen light in the computer science community in recent years (e.g. the Chinese restaurant process). I would greatly appreciate it greatly if you could share with me any material that you have used in the past in teaching this course. Since I have little experience estimating work load, I could use help in knowing how many problems you assigned each time.

My reply:

I can’t actually remember how many homeworks I assign; probably about 5 per week, since most of them are pretty short. The key, as always, in such a course, is not to get stuck forever in the first few chapters. In the early chapters, I recommend the following material:

– Chapter 1: All of this is ok; you can pick and choose what you want to cover.
– Chapter 2: My favorite is the kidney cancer rate example. The example earlier in the chapter on sex ratios is sort of old fashioned.
– Chapter 3: My favorite is the bioassay example. Some of the stuff earlier in the chapter on various conjugate and semi-conjugate priors is ok, but you don’t want the students to waste too much time on the algebra. It’s there for them later when they need it.
– Chapter 4: This is a short chapter, you can cover all of it.
– Chapter 5: This chapter has 2 main examples. They’re both good, but the second example–the 8 schools–is better and also generalizes better to the later material.
– Chapter 6: I like all of this stuff a lot.
– Chapter 7: This is important stuff but you can pick and choose what might be relevant to the students. It’s difficult material so you might want to skip it.
– Chapter 8: This stuff is important, I think.
– Chapter 9: I like this stuff too, even though I think a lot of people skip it…

The key thing in the early chapters is to not obsess on the question of “where do the priors come from.” They’re just models, they come from the same place that likelihoods come from. In the first edition of the book, I probably wasted too much time trying to set up reasonable noninformative priors, and a lot of that stuff remains in the book, probably it’s a bit of a distraction from the main challenges.

Finally, if you have students of varied backgounds, you could have them go to my new book with Jennifer Hill and read chapter 18 (I think that’s the one, it’s the chapter on likelihood and Bayes), since that’s a quick non-mathematical overview that goes over the basics of Bayes.