Keith said,

“Now, enabling non-statisticians to more fully grasp uncertainty and how statistics (tries to) adequately quantify it – is a real big open question.”

In case it might be useful to others trying to tackle this problem — here’s a link to an attempt I once made at this:

https://web.ma.utexas.edu/users/mks/statmistakes/uncertainty.html

(Some of the external links are broken, but some of those can easily be retrieved by a simple search on the information given.)

]]>These plots might help https://statmodeling.stat.columbia.edu/2019/05/29/concurve-plots-consonance-curves-p-value-functions-and-s-value-functions/

They would at least would show how slowly “support” falls of as one moves away from the point estimate.

Now, enabling non-statistician to more fully grasp uncertainty and how statistics (tries to) adequately quantify it – is a real big open question.

]]>The alarmist side will say “as much as 40% of the salmon in our rivers are bio-engineered fish that have escaped the factory farms.” Activists will hear this as “40% of the salmon in our rivers are GMO FrankenSalmon,” and will repeat something even more extreme.

The apologist side will say that “alarmist fears are overblown, there is no hard evidence that the new salmon have entered our streams, much less that there has been any harm.”

]]>Hi Bob, that would be great! The N-eff needed to do a good job even for 95% intervals can be surprisingly large.

]]>If you want to present a probability distribution and avoid some of these problems, how about converting to a cumulative formulation? Then you can provide probabilities that the true value is beyond a certain threshold (for a few decision-relevant thresholds) rather than trying to summarize directly with a point estimate or interval.

They use sample statistics and get an estimate of population mean length from their sample. The calculate the distribution and are quite confident that they have a good representation of the population mean. But, when they calculate a confidence interval from a sample of proportion, they think the same way. The middle value in the distribution is close to the real value. If the sample size is large, there is of course no big problem. They do not see that wide confidence intervals must be interpreted in a different way.

I’ve seen this, too. The same confusion can arise in an MCMC setting. For example, we report an MCMC standard error in Stan for all posterior expectations reported (e.g., parameters for posterior means and indicators for event probabilities). We also report some quantiles by default, which can be used to define the boundaries of posterior intervals. We’re in the process of moving default reporting to also report on convergence for the quantiles, including tail quantiles, which can show different properties than the convergence of means.

]]>