Beyond the teaching question, this is an interesting topic on its own: thinking about classical statistical ideas of point estimation, hypothesis testing, and uncertainty quantification, but taking Bayesian methods as a starting point.

The idea is that, instead of structuring this based on the methods of maximum likelihood estimation, null hypothesis significance testing, and confidence intervals, you start with the goals of estimating parameters, making predictions, comparing and evaluating models (i.e., alternative explanations of the world), and summarizing and working with uncertainty.

If you’re interested in using Bayesian methods, we have two books (Bayesian Data Analysis and Bayesian Workflow) on the topic. But with these methods under our belt, we can now go back to the motivating questions–the more general statistical goals, which exist without reference to any particular models or any particular Bayesian or frequentist methods–and consider them from scratch.

This seems important.

My discussion here is motivated by this question sent in by Opher Donchin:

Here’s one for your blog (although I’m happy to have your take as well).

I’m transitioning our undergrad biomedical engineering course this year from a standard frequentist syllabus to a Bayesian approach. We are mostly following the first 6 chapter of Bayesian Analysis with Python by Osvaldo Martin. In order to get the department to agree to this, I had to promise to teach basic frequentist methods.

Thus, I need to teach frequentist methods to students with a strong Bayesian background. There doesn’t seem to be much available on this. There is of course material comparing the two approaches, but I mean specific material designed to explain the frequentist approach to someone who knows the Bayesian approach.

I’d love to know if anyone knows of resources or has experience of insight or advice.

I replied that I’ll see what the blog commenters suggest, but in the meantime, I recommend chapter 4 of Bayesian Data Analysis as a start.

Donchin responded:

Yes. Chapter 4 is very good on the principles involved. Much of it addresses the Bayesian alternatives to frequentist procedures or the Bayesian perspective on them.

I’m wondering about something more concrete, aimed at a less sophisticated audience. That is, my students will know how to build models, how to interpret the posterior samples, the basics of Bayesian workflow and also model comparison. On the other hand, they will have no knowledge of confidence intervals, maximum likelihood estimates, hypothesis testing, or multiple comparison procedures. Reasonably, my department demands that they be able to read the biomedical literature where such terms are widespread.

I want to give them an understanding of frequentist procedures without getting bogged down in frequentist justifications.

To take an example, I want to explain what an F test calculates when understood within a Bayesian framework. To that end, I can show students a Bayesian model of a normal distribution of group means and a normal likelihood within each group. Then, I can work through what a Bayesian would need to calculate on that model to produce an F statistic and an F test. It has something to do with summaries of posterior distributions of ratios of variances.

What I’m hoping for is somewhere where such questions are worked through in detail so that it could be used for developing lectures.

Of course, chapter 4 of BDA is 10 years old at this point. I imagine some of the ideas may have developed since then.

Ahhh, good point! Chapter 4 of BDA is for statisticians who already know the classical methods and want to understand how these can be understood in light of Bayesian principles and adapted within a Bayesian workflow. But it’s really a completely different task to explain classical methods to students who haven’t already learned them. The idea would be to retcon ideas of classical statistics from a Bayesian angle.

This would be worth doing.

In the meantime, I recommend . . . chapter 4 of Regression and Other Stories, where we go through basic principles of point estimation, hypothesis testing, and uncertainty quantification from an applied perspectives. Also, if you flip through that book, you’ll see other places where we discuss classical procedures from first principles. We don’t have any F tests or multiple comparisons adjustments because I can’t bring myself to care about those things, but a lot else is there, so you might be able to put together much of what you need from that book.

In response to that recommendation, Donchin wrote:

I agree that Chapter 4 of Regression and Other Stories touches on many of the important points, but it is not sufficient for my needs.

I am trying to teach a Bayesian-first undergraduate statistics course to biomedical engineers that also gives a background in frequentist approaches allowing  them to function effectively in environments that require them to use or understand frequentist stats.

This means that the frequentist-realted topics we cover include:

• Estimation: MLE, standard errors, confidence intervals
• Hypothesis testing: p-values, Type I/II errors, power
• Proportion tests and t-tests (independent and paired)
• Effect sizes; multiple comparisons (briefly)
• Linear and multiple regression: least squares, coefficient tests, R2, F-tests
• ANOVA: categorical predictors, interactions, sums of squares, effect sizes
• Pearson correlation and inference
• Repeated-measures and mixed-effects models
• Model comparison: AIC and cross-validation
• Replication crisis / open science / pre-registration (not exactly frequentist, but still)

You can see the syllabus and the lectures in the student-facing version of the course repo at:https://github.com/opherdonchin/StatisticsCourse_36714361

Any further thoughts you might have would be great to hear.

Also, I would be happy to get this some visibility, in hopes that other people would be interested in providing feedback, using some of the material, or just doing it better.

I don’t think I could bring myself to teach a lot of the above topics, except in an “inoculation” sort of way, but I recognize that many students will need it, so if anyone has some good suggestions for Donchin, just leave them here in the comments!