Christian Robert on the Jeffreys-Lindley paradox; more generally, it’s good news when philosophical arguments can be transformed into technical modeling issues

X writes:

This paper discusses the dual interpretation of the Jeffreys– Lindley’s paradox associated with Bayesian posterior probabilities and Bayes factors, both as a differentiation between frequentist and Bayesian statistics and as a pointer to the difficulty of using improper priors while testing. We stress the considerable impact of this paradox on the foundations of both classical and Bayesian statistics.

I like this paper in that he is transforming what is often seen as a philosophical argument into a technical issue, in this case a question of priors. Certain conventional priors (the so-called spike and slab) have poor statistical properties in settings such as model comparison (in addition to not making sense as prior distributions of any realistic state of knowledge). This reminds me of the way that we nowadays think about hierarchical models. In the old days there was much thoughtful debate about exchangeability and the so-called Stein paradox that partial pooling could lead to improved estimates. Nowadays we realize that the key issue is not “exchangeability” (a close-to-meaningless criterion in that it just is the requirement that the data from the different groups be treated symmetrically) but rather the model that is being used for the distribution of the varying parameters. Switch from a normal distribution to, say, a bimodal distribution, and the Stein-like pooling goes away. Lots of anguished philosophy is replaced by probability modeling.

8 thoughts on “Christian Robert on the Jeffreys-Lindley paradox; more generally, it’s good news when philosophical arguments can be transformed into technical modeling issues

  1. Yes, probability modeling is a fast way to avoid many issues that flourish during philosophical arguments, but
    philosophical ideas are needed to make sensible probability models.

    • Martin:

      I like philosophy too! But I think there’s no doubt that our understanding of hierarchical models is much better when we can argue about “what’s the right distribution to use for these effects?” rather than “are these effects exchangeable?”

      Also, I don’t think the technical issues are allowing us to “avoid” the philosophical questions; rather, they are addressing some real concerns that had been framed as being philosophical.

    • When scientists figure out how to do something, they start leaving off the “philosophy” sections of their papers. It used to be that every Bayesian paper out there had a long opener on why Bayesian models make sense philosophically and/or are useful. You don’t see that nearly so much in today’s papers.

      Philosophy is like artificial intelligence, about which it is said that when we figure out an algorithm for something (vision, chess, robotic manipulation, speech recognition, etc.), it’s no longer AI, it’s engineering.

      Or as a philosopher might argue, a former problem gets “explained away.” Don’t worry, philosophers won’t be out of work — there’s always another layer of abstraction waiting.

    • Martin: I agree or at least one cannot apply statistical ideas to learn from data without a philosophy of science (more or less naive) but many misapprehensions of what is going on statistically (due to not appreciating exactly what prior and or data model is being assumed or their role in attempting to represent adequate what we are trying to learn about in this universe) are turned into problems in philosophy that are (completely) unrelated to what is going on.

      Often these involve extensions to non-finite sets that can be useful approximations but not sensible as literal representations. So in X’s paper, footnote 18 caught my attention as seems very vague if not strange – “equates the values of two density functions” – would seem unrelated to how good an approximation a model is but I have yet to read the paper.

Comments are closed.