From “Mathematical simplicity is not always the same as conceptual simplicity” to scale-free parameterization and its connection to hierarchical models

I sent the following message to John Cook:

This post popped up, and I realized that the point that I make (“Mathematical simplicity is not always the same as conceptual simplicity. A (somewhat) complicated mathematical expression can give some clarity, as the reader can see how each part of the formula corresponds to a different aspect of the problem being modeled.”) is the kind of thing that you might say!

Cook replied:

On a related note, I [Cook] am intrigued by dimensional analysis, type theory, etc. It seems alternately trivial and profound.

Angles in radians are ratios of lengths, so they’re technically dimensionless. And yet arcsin(x) is an angle, and so in some sense it’s a better answer.

I’m interested in sort of artificially injecting dimensions where the math doesn’t require them, e.g. distinguishing probabilities from other dimensionless numbers.

To which I responded:

That’s interesting. It relates to some issues in Bayesian computation. More and more I think that the scale should be an attribute of any parameter. For example, suppose you have a pharma model with a parameter theta that’s in micrograms per liter, with a typical value such as 200. Then I think we should parameterize theta relative to some scale: theta = alpha*phi, where alpha is the scale and phi is the scale-free parameter. This becomes clearer if you think of there being many thetas, thus theta_j = alpha * phi_j, for j=1,…,J. The scale factor alpha could be set a priori (for example, alpha = 200 micrograms/L) or it could itself be estimated from the data. This is redundant parameterization but it can make sense from a scientific perspective.

P.S. More here.

2 thoughts on “From “Mathematical simplicity is not always the same as conceptual simplicity” to scale-free parameterization and its connection to hierarchical models

  1. I (currently) think that physical dimensions are just a simple way of determining the physical interpretation of a parameter value, so that either every physically interpreted parameter value should have a dimension (if one wants to include the interpretation in the mathematics, as Andrew suggests) or no parameter should (if one wants to keep track of the physical interpretation independently from the mathematics, as physicists do when expressing everything in units of natural constants, e.g. setting the speed of light to 1). So angles in radians should be thought of as ratios of angles to a unit angle, just as lengths in meters are thought of as ratios of lengths to a unit length. One might then include the reference to the unit angle in the equation or just keep track of the interpretation of the parameter externally.

    For probability, that connects to a point previously made on the blog (I think by Martha (Smith)), according to which the probability axioms are just models that fit better or worse for different physical phenomena (degrees of beliefs, frequencies of measurement results, ratios of sheep, what have you). For each of the different phenomena, the unit for the probability function would be different, since the physical interpretation of the axioms would be different.

  2. I would suggest another related example is the sensitivity analysis in the context of unobserved confounding, in which the confounding assumption is examined by some p-values in some association test, while the scale at which these p-values are compared are mathematically artificial and not always physically related.

Leave a Reply to Yuling Cancel reply

Your email address will not be published. Required fields are marked *