Philosophy of Biostatistics at http://biostat.mc.vanderbilt.edu/wiki/Main/FrankHarrell and/or http://www.fharrell.com/2017/01/introduction.html

Fundamental Principles of Statistics at http://www.fharrell.com/2017/01/fundamental-principles-of-statistics.html

]]>If you take the forward problem to be parameter to data, ie a data generating model etc, then the inverse problem is data to parameter.

Forward problems are often required to respect certain features of the world based on physical principles, but the induced inverse problem typically does not respect these features. That is what makes ‘inference’ difficult.

]]>If we’re in agreement, then my suggestion is that we focus our public education efforts on appropriate versus inappropriate applications of binary hypothesis testing, and on the arbitrariness of red lines in science, rather than on inherent problems with p-values. I know that you have emphasized these points elsewhere, but I’ve had more than a few collaborators tell me that they know all about how p-values are no good, but then insist on using 95% confidence intervals to decide whether we have a “finding” or not.

Education about p-values should, in my view, stress their proper computation, interpretation, and application, but if the takeaway is “p-values are no good”, then I don’t think any progress has actually been made.

]]>I agree with you. That’s why I wrote an article called, “The problems with p-values are not just with p-values”!

]]>All of this is familiar material. The question is where this reasoning goes wrong, such that p-values become the object of so much criticism. It seems to me that this reasoning is sound, and thus p-values are inappropriate only if deciding between a pair of mutually exclusive, collectively exhaustive hypotheses is not the best formulation of our scientific problem. The fact that many users misinterpret p-values, or report quantities that do not really qualify as p-values, is not the fault of p-values so much as it is the fault of inadequate training in statistics. Similarly, the use of binary hypothesis testing where it is not the best formulation of our scientific problem is driven by a demand for p-values from journals, which is itself driven by a combination of inadequate training in statistics as well as a perceived need for a red line to distinguish findings from no findings. Replacing p-values with something else would not overcome this training deficit, nor would it eliminate the perceived need for a red line.

I agree that, if science was done better, we’d see a lot less p-values, but I don’t think that is because p-values are objectionable in and of themselves, but because users of statistics need better training in statistical methods as well as better goals for scientific research more broadly.

]]>For example, the objections Farrell lists under A, B, G seem misguided to me. Others will, on the other hand, likely strongly agree with them.

In particular one issue I have (in principle) is that I see the desire for what Harrell calls ‘forward probabilities’ (which are usually or used to be called ‘inverse probabilities’!) as in fact striving for more certainty than is really possible. It’s at the heart of the ‘Bayes is not compatible with falsificationist reasoning’ misconception – eg that we always really want the ‘direct’ probability of a model.

]]>