There are two aspects of the problem where we might consider ‘direction’. One is the order in which information arrives and parameters are formulated. This is subject to debate. The other is the direction of the probabilities that are computed, e.g., computing probabilities of the future given the past or given the present state. It is in the sense that I speak of forward probabilities and note that p-values are backwards.

]]>Ditto on the oops

]]>(But I can agree, I suppose, that the Bayesian approach is essentially to ‘directly’ construct an inverse to your forward problem.)

]]>Oops — Harrell, not Farrell.

]]>Philosophy of Biostatistics at http://biostat.mc.vanderbilt.edu/wiki/Main/FrankHarrell and/or http://www.fharrell.com/2017/01/introduction.html

Fundamental Principles of Statistics at http://www.fharrell.com/2017/01/fundamental-principles-of-statistics.html

]]>An inverse problem is defined relative to a forward problem.

If you take the forward problem to be parameter to data, ie a data generating model etc, then the inverse problem is data to parameter.

Forward problems are often required to respect certain features of the world based on physical principles, but the induced inverse problem typically does not respect these features. That is what makes ‘inference’ difficult.

]]>Thanks for the pointer.

If we’re in agreement, then my suggestion is that we focus our public education efforts on appropriate versus inappropriate applications of binary hypothesis testing, and on the arbitrariness of red lines in science, rather than on inherent problems with p-values. I know that you have emphasized these points elsewhere, but I’ve had more than a few collaborators tell me that they know all about how p-values are no good, but then insist on using 95% confidence intervals to decide whether we have a “finding” or not.

Education about p-values should, in my view, stress their proper computation, interpretation, and application, but if the takeaway is “p-values are no good”, then I don’t think any progress has actually been made.

]]>Ram:

I agree with you. That’s why I wrote an article called, “The problems with p-values are not just with p-values”!

]]>All of this is familiar material. The question is where this reasoning goes wrong, such that p-values become the object of so much criticism. It seems to me that this reasoning is sound, and thus p-values are inappropriate only if deciding between a pair of mutually exclusive, collectively exhaustive hypotheses is not the best formulation of our scientific problem. The fact that many users misinterpret p-values, or report quantities that do not really qualify as p-values, is not the fault of p-values so much as it is the fault of inadequate training in statistics. Similarly, the use of binary hypothesis testing where it is not the best formulation of our scientific problem is driven by a demand for p-values from journals, which is itself driven by a combination of inadequate training in statistics as well as a perceived need for a red line to distinguish findings from no findings. Replacing p-values with something else would not overcome this training deficit, nor would it eliminate the perceived need for a red line.

I agree that, if science was done better, we’d see a lot less p-values, but I don’t think that is because p-values are objectionable in and of themselves, but because users of statistics need better training in statistical methods as well as better goals for scientific research more broadly.

]]>To elaborate, many Bayesians (especially Jaynesians) emphasise the need to (formally) ‘condition’ on background information, but almost never put a probability distribution over this. Jaynes (from what I remember) and you (Andrew) often emphasise that we ‘learn the most’ when our model or background information is thrown into doubt by the data. Is this not paradigmatic ‘backwards’ (in Harrell’s sense) reasoning?

]]>For example, the objections Farrell lists under A, B, G seem misguided to me. Others will, on the other hand, likely strongly agree with them.

In particular one issue I have (in principle) is that I see the desire for what Harrell calls ‘forward probabilities’ (which are usually or used to be called ‘inverse probabilities’!) as in fact striving for more certainty than is really possible. It’s at the heart of the ‘Bayes is not compatible with falsificationist reasoning’ misconception – eg that we always really want the ‘direct’ probability of a model.

]]>