Bayesian Computation, Fall 2011, Andrew Gelman (Statistics G8325, section 002)

Class meets Wed 9:30-12, Mudd Hall 1106B

Research topics. We are working on the following Bayesian computation projects:
– Automated adaptive Hamiltonian Monte Carlo
– Penalized maximum likelihood for multilevel models
– Weakly informative prior distributions
– Multilevel regression and poststratification
– Missing-data imputation using iterative regressions
– Automating graphical displays for checking and understanding models

Skills. In the homeworks, you will work in R to:
– Program the following Markov chain samplers: Gibbs, Metropolis, Metropolis-Hastings, slice, Langevin, Hamiltonian, also variational Bayes
– Make statistical graphs and perform posterior predictive checks
– Check Bayesian inference programs using fake-data simulation

All readings, class notes, and homeworks will be posted as pdfs at http://www.stat.columbia.edu/~gelman/bayecomputation

As a start, you should download and read course.outline.pdf, GelmanBoisJiang1996.pdf, introbookchapters2and3.pdf, bdachapters10and11.pdf, bdaAppendixC.pdf, class1handout.pdf, and, most important, hwk1.pdf and hwk2.pdf

Here is our current plan:

Week 1:
Reading: Gelman, Bois, and Jiang (1996) paper on hierarchical Bayesian toxicokinetics
Homework due 9am Friday of week 1: Set up R and Jags
In class: Review of Bayesian data analysis using toxicokinetics example; plan of course

Week 2:
Reading: Chapters 2-3 on R from my intro book, Chapters 10-11 and Appendix C of Bayesian Data Analysis
Homework due 5pm Tues before class: Gibbs sampler
In class: Review of homework; presentation of Gibbs sampler and Metropolis algorithm

Week 3:
Reading: Gelman and Shirley (2011), Kass et al. (1998), Lunn et al. (2009)
Homework due 5pm Tues before class: Metropolis algorithm
In class: review of homework; presentation of methods for implanting Gibbs and Metropolis and improving their efficiency

Week 4:
Reading: Abayomi et al. (2009) and Su et al. (2011) papers on multiple imputation, chapter 25 of Gelman and Hill
Homework due 5pm Tues before class: Optimize Metropolis
In class: review of homework; presentation of mi

Week 5:
Reading: Gelman (2006) and Gelman et al. (2008) on weakly informative priors
Homework due 5pm Tues before class: Metropolis-Hastings
In class: review of homework; presentation of weakly informative priors

Week 6:
Reading: Neal (2003) on slice sampling, Lax and Phillips (2009) on multilevel regression and poststratification
Homework due 5pm Tues before class: Slice sampling
In class: review of homework; presentation of slice sampling; presentation of mrp

Week 7:
Reading: Description of Langevin algorithm in MacKay (2004), Chung et al. (2011) on avoiding boundary estimates in hierarchical models, Dorie (2011) on sim
Homework due 5pm Tues before class: Langevin
In class: review of homework; presentation of Langevin algorithm, presentation of maximum likelihood and Bayes inference for hierarchical models

Week 8:
Reading: Neal (2011) on Hamiltonian Monte Carlo, Hoffman and Gelman (2011) on the no-U-turn sampler, Carpenter et al. (2011) on Stan
Homework due 5pm Tues before class: Hamiltonian Monte Carlo
In class: review of homework; presentation of HMC, the no-U-turn sampler, and stan

Week 9:
Reading: Description of variational Bayes in MacKay (2004)
Homework due 5pm Tues before class: Variational Bayes
In class: review of homework; presentation of variational Bayes

Week 10:
Reading: Gelman (2004) on Bayesian exploratory data analysis, Gelman and Unwin (2011) on information visualization
Homework due 5pm Tues before class: Exploratory data analysis
In class: review of homework; presentation of Bayesian exploratory data analysis; presentation of lattice graphics in R

Week 11:
Reading: Schofield et al. (2011) on hierarchical spline models for tree rings
Homework due 5pm Tues before class: Fit and graph a model
In class: review of homework; presentation of spline models; presentation of tree-ring analysis

Week 12:
Reading: Chapter 6 of Bayesian Data Analysis, Zheng, Salganik, and Gelman (2006) on “How many X’s do you know?” surveys
Homework due 5pm Tues before class: Posterior predictive check
In class: review of homework; presentation of Bayesian model checking, presentation of social network models

Week 13:
Reading: Cook, Gelman, and Rubin (2006) on Bayesian debugging
Homework due 5pm Tues before class: Fake-data debugging
In class: review of homework; presentation of Bayesian debugging; general discussion