I was trying to explain in class how a (Bayesian) statistician reads the formula for a probability distribution. In old-fashioned statistics textbooks you’re told that if you want to compute a conditional distribution from a joint distribution you need to do some heavy math: p(a|b) = p(a,b)/\int p(a’,b)da’.
When doing Bayesian statistics, though, you usually don’t have to do the integration or the division. If you have parameters theta and data y, you first write p(y,theta). Then to get p(theta|y), you don’t need to integrate or divide. All you have to do is look at p(y,theta) in a certain way: Treat y as a constant and theta as a variable. Similarly, if you’re doing the Gibbs sampler and want a conditional distribution, just consider the parameter you’re updating as the variable and everything else as a constant. No need to integrate or divide, you just take the joint distribution and look at it from the right perspective.
Awhile ago Yair told me there’s something called the “Washington read,” where you pick up a book, go straight to the index, and see if, where, and how often you’re mentioned.
It struck me, when explaining Bayesian algebra, that what we’re really doing when we get a conditional distribution is to take a Washington read of the joint distribution, from the perspective of the parameter or parameters of interest.
More generally, I’ve found that an important step in being able to do mathematics for statistics is learning how to focus on different symbols in a formula. In math, all symbols are in some sense equal, whereas in statistics, x and y and pi and theta and sigma and lambda all play different roles. If you don’t get that—if you read formulas in a flat two dimensions without seeing the context or implications or personality of each symbol—you can easily get stuck, sort of like how it would be essentially impossible to read this passage if you had to look up every one of its words in the dictionary.
The “Washington read” for conditional distributions is an example of statistical reading of mathematics. (Another example is that, with rare exceptions, I read “38.24%” or “38.2%” as 38%, or even 40%.)