I was checking out the comments at my bloggingheads conversation with Eliezer Yudkowsky, and I noticed the following, from commenter bbbeard:
My sense is that there is a fundamental sickness at the heart of Bayesianism. Bayes’ theorem is an uncontroversial proposition in both frequentist and Bayesian camps, since it can be formulated precisely in terms of event ensembles. However, the fundamental belief of the Bayesian interpretation, that all probabilities are subjective, is problematic — for its lack of rigor. . . .
One of the features of frequentist statistics is the ease of testability. Consider a binomial variable, like the flip of a fair coin. I can calculate that the probability of getting seven heads in ten flips is 11.71875%. I can check this, first of all, with a computer program that generates random numbers uniformly in [0,1) in groups of ten, and keeping tabs on what fraction of samples have exactly seven numbers less than 0.5. Obviously I can do this for any (m,n). I can also take a coin and flip it many times and get an empirical approximation to 11.71875%. At some point a departure from the predicted value may appear, and frequentist statistics give objective confidence intervals that can precisely quantify the degree to which the coin departs from fairness. . . . What is unclear to me is how a Bayesian would map out an experiment, either numerical or empirical, to demonstrate the posterior distribution in the unknown unfair coin experiment. That’s why I ask, “what does the posterior distribution mean”? . . . The Bayesian interpretation is certainly not what we use in physics. Suppose we lived at a time before the speed of light was measured accurately. You could poll a bunch of people, even “experts”, and get a range of guesses about the value of the speed of light. A Bayesian would construct a prior from this information. But what happens when you go do the experiment? . . .
I don’t know that any readers of this blog will need an answer to these questions, but just quickly:
1. No, Bayesian probabilities don’t have to be subjective. See chapter 1 of Bayesian Data Analysis for discussion and examples.
2. Bayesian models can indeed be tested. See chapter 6 of Bayesian Data Analysis.
3. Probability distributions in physics are not so clear as you might think. See the bottom half of page 7 in my Bayesian Analysis discussion here.
OK, I think that just about covers it.
P.S. These definitions (from pages 1-2 of this article) may also be of help:
“Bayesian inference” represents statistical estimation as the conditional distribution of parameters and unobserved data, given observed data. “Bayesian statisticians” are those who would apply Bayesian methods to all problems. (Everyone would apply Bayesian inference in situations where prior distributions have a physical basis or a plausible scientific model, as in genetics.) “Anti-Bayesians” are those who avoid Bayesian methods themselves and object to their use by others.