The information principle: the key to a good statistical method is not its underlying philosophy or mathematical reasoning, but rather what information the method allows us to use. Good methods make use of more information. This can come in different ways . . .
The methodological attribution problem: the many useful contributions of a good statistical consultant, or collaborator, will often be attributed to the statistician’s methods or philosophy rather than to the artful efforts of the statistician himself or herself. . . .
Different applications demand different philosophies: consider Rob Kass’s remark: “I tell my students in neurobiology that in claiming statistical significance I get nervous unless the p-value is much smaller than 0.01.” In political science, we are typically not aiming for that level of certainty. (Just to get a sense of the scale of things, there have been barely 100 national elections in all of U.S. history, and political scientists studying the modern era typically start in 1946.)
These appeared in my 2010 article, Bayesian statistics then and now, which is a discussion of an article by Brad Efron, “The future of indirect evidence” and of Rob Kass’s discussion of Efron’s article.
Here’s another line from my paper:
Maximum likelihood, like many classical methods, works pretty well in practice only because practitioners interpret the methods flexibly and do not do the really stupid versions (such as joint maximization of parameters and hyperparameters) that are allowed by the theory.
This is related to the idea that theoretical statistics is the theory of applied statistics, and methods as they are applied in real life are not always the same as what you might think from the formulas alone.
New possible paper idea: “Reflection on flexible statistics: The three meta-principles of statistics”.
Now for even more, and even wilder, creative pondering:
I suck at all things statistics, but I tried to more conceptually grasp things when reading the “Bayesian statistics then and now”. I was wondering whether the meta-principles might in some way be seen as being indicative of having a Bayesian view.
The “information principle” goes something like more information used by the method might mean the method is “good” (?), which might be in line with Bayesian statistics where prior (more) information is used (?).
The “methodological attribution problem” goes something like both the method and the person can influence how something is viewed, and the role of the person should not be overlooked (?). This might be in line with Bayesian statistics in the sense that (according to wikipedia) the degree of belief may be based on prior knowledge (cf. the method) or personal beliefs (cf. the person?).
The “different applications demand different philosophies” goes something like statistical concepts make more sense in certain settings than in others (?) might relate to the Bayesian statistics feature (according to some online sources I came across) that it might be relatively flexible (?).
Anyway, that’s enough conceptual pondering about things I know nothing of…
“The methodological attribution problem: the many useful contributions of a good statistical consultant, or collaborator, will often be attributed to the statistician’s methods or philosophy rather than to the artful efforts of the statistician himself or herself. . . .”
This feels very close – at least to me – to what Egami and Hartman’s refer to as C-Validity (context) when assessing the external validity of a result. The application of ‘Good’ method XYZ in another context, with a different team, might be ‘less’ than good, at least at the margin wrt some other method. It seems that the higher the caliber of the researcher(s) that have done a study on a efficacy of some more complex method, might increase the confidence in the internal validity of the result, perhaps in some sense weaken one’s confidence of its external validity.
According to surveys 6% of people say they can beat up a grizzly bear and when you watch an NFL game there are about 10 people in the crowd who believe they have been abducted by aliens.
This was meant to be here: https://statmodeling.stat.columbia.edu/2025/11/21/what-do-you-think-is-the-ideal-number-of-children-for-a-family-to-have-two-different-statistical-measurement-challenges-arise-from-this-one-question-on-the-general-social-survey/