I led a 4-hour workshop on teaching statistics at the Association for Psychological Science meeting yesterday. Here’s the powerpoint–I didn’t actually get through all of it, because we spent nearly half the time in group discussions.
Here’s the book, and here are other thoughts on teaching from the blog.
Very interesting. I remember the confidence interval demo (estimate 25% and 75%) from when I was an undergrad at Caltech. It made a huge impression. I can't remember anything else from the class, not even the topic. The teacher didn't tell us a "foolproof method" of being properly calibrated. I suppose that would be when asked for a 50% interval give a 90% interval.
Seth,
I took the demo of the uncertainty intervals (in deference to statistical terminology, I wouldn't call them "confidence intervals") from the classic Alpert and Raiffa (1968) article that was reproduced in the Kahneman, Slovic, and Tversky book. (All references are given in my book with Deb.) Unfortunately, this was one of the demos I didn't have time to do in the presentation.
The "foolproof method" is really a joke: the idea is to flip a coin: if heads, give the interval (0,0), if tails, give the interval (-infinity, infinity). Perfect 50% coverage (in expectation) but not at all informative.
The particular comment on sampling distribution strikes a chord with me. If we want to teach frequentist confidence intervals, it's crucial to understand the concept and yet it is so hard to convey it well.
I have had some success with the "recount" scenario. Imagine a close election in which candidates are allowed to request recounts of random samples of the cast ballots. What happens if each candidate successively asked for recounts over and over again? It's quite likely that different samples will yield different winners. This allows for a discussion of a sampling distribution. Maybe this can be made into a group activity too.
I have soured on Monte Hall as an example. I find that a lot of time is wasted to convince the inevitable few who just can't or won't understand why one should switch.