Skip to content
Search results for p value

“Real men keep p values to themselves”

AT points me to this column by Andrew Leonard, who writes: I [Leonard] asked my readers to explain Felix Salmon’s statement that “I’d say p=0.3 right now that Barack Obama’s first major act as POTUS will be the nationalization of Citigroup.” What follows are some amusing quotes, including this one from Kobi “diagnostics for multiple […]

Getting negative about the critical positivity ratio: when you talk about throwing out the bathwater, really throw out the bathwater! Don’t try to pretend it has some value. Give it up. Let it go. You can do this and still hold on to the baby at the same time!

But maybe it’s all OK? Most of this post is a pretty negative review of a recent book, about which I will apply the well-known saying, “Your manuscript is both good and original; but the part that is good is not original, and the part that is original is not good.” That said, the part […]

The value of thinking about varying treatment effects: coronavirus example

Yesterday we discussed difficulties with the concept of average treatment effect. Part of designing a study is accounting for uncertainty in effect sizes. Unfortunately there is a tradition in clinical trials of making optimistic assumptions in order to claim high power. Here is an example that came up in March, 2020. A doctor was designing […]

Election odds update (Biden still undervalued but not by so much)

Last week I wrote about the discrepancy between our election forecast and the betting odds: Suppose I were to lay $1000 on Biden right now. According to Betfair it seems that, if I win, I make a profit of $840. And our model gives Biden an 88% chance of winning. But we’re modeling Biden vs. […]

The value (or lack of value) of preregistration in the absence of scientific theory

Javier Benitez points us to this 2013 post by psychology researcher Denny Borsboom. I have some thoughts on this article—in particular I want to compare psychology to other social science fields such as political science and economics—but first let me summarize it. Preregistration and open science Borsboom writes: In the past few months, the Center […]

What’s the p-value good for: I answer some questions.

Martin King writes: For a couple of decades (from about 1988 to 2006) I was employed as a support statistician, and became very interested in the p-value issue; hence my interest in your contribution to this debate. (I am not familiar with the p-value ‘reconciliation’ literature, as published after about 2005.) I would hugely appreciate […]

P-value of 10^-74 disappears

Nick Matzke writes: Given the recent discussion of p-values, you or colleagues might find this interesting: Population Genetics: Why structure matters Nick Barton, Joachim Hermisson, Magnus Nordborg One possibility is to compare the population estimates with estimates taken from sibling data, which should be relatively unbiased by environmental differences. In one of many examples of […]

Here are some examples of real-world statistical analyses that don’t use p-values and significance testing.

Joe Nadeau writes: I’ve followed the issues about p-values, signif. testing et al. both on blogs and in the literature. I appreciate the points raised, and the pointers to alternative approaches. All very interesting, provocative. My question is whether you and your colleagues can point to real world examples of these alternative approaches. It’s somewhat […]

Concurve plots consonance curves, p-value functions, and S-value functions

Andrew Vigotsky writes: Now that abandoning significance and embracing uncertainty is in the air, we think this package, which runs in R or Stata, may be of interest to both you and your readers. Concurve plots consonance curves, p-value functions, and S-value functions to allow readers and researchers to get a better feel of the […]

No, its not correct to say that you can be 95% sure that the true value will be in the confidence interval

Hans van Maanen writes: Mag ik je weer een statistische vraag voorleggen? If I ask my frequentist statistician for a 95%-confidence interval, I can be 95% sure that the true value will be in the interval she just gave me. My visualisation is that she filled a bowl with 100 intervals, 95 of which do […]

Thinking about “Abandon statistical significance,” p-values, etc.

We had some good discussion the other day following up on the article, “Retire Statistical Significance,” by Valentin Amrhein, Sander Greenland, and Blake McShane. I have a lot to say, and it’s hard to put it all together, in part because my collaborators and I have said much of it already, in various forms. For […]

A comment about p-values from Art Owen, upon reading Deborah Mayo’s new book

The Stanford statistician writes: One of the fun parts of this was reading some of what Meehl wrote. I’d seen him quoted but had not read him before. What he says reminds me a lot of how p values were presented when I was an undergraduate at Waterloo. They emphasized large p values as a […]

The p-value is 4.76×10^−264

Jerrod Anderson points us to Table 1 of this paper: It seems that the null hypothesis that this particular group of men and this particular group of women are random samples from the same population, is false. Good to know. For a moment there I was worried. On the plus side, as Anderson notes, the […]

Discussion of the value of a mathematical model for the dissemination of propaganda

A couple people pointed me to this article, “How to Beat Science and Influence People: Policy Makers and Propaganda in Epistemic Networks,” by James Weatherall, Cailin O’Connor, and Justin Bruner, also featured in this news article. Their paper begins: In their recent book Merchants of Doubt [New York:Bloomsbury 2010], Naomi Oreskes and Erik Conway describe […]

Answering the question, What predictors are more important?, going beyond p-value thresholding and ranking

Daniel Kapitan writes: We are in the process of writing a paper on the outcome of cataract surgery. A (very rough!) draft can be found here, to provide you with some context: Using standard classification methods (Python sklearn, with synthetic oversampling to address the class imbalance), we are able to predict a poor outcome […]

Don’t define reproducibility based on p-values

Lizzie Wolkovich writes: I just got asked to comment on this article [“Genotypic variability enhances the reproducibility of an ecological study,” by Alexandru Milcu et al. ]—I have yet to have time to fully sort out their stats but the first thing that hit me about it was they seem to be suggesting a way […]

If you want to know about basketball, who ya gonna trust, a mountain of p-values . . . or that poseur Phil Jackson??

Someone points me with amusement to this published article from 2012: Beliefs About the “Hot Hand” in Basketball Across the Adult Life Span Alan Castel, Aimee Drolet Rossi, and Shannon McGillivray University of California, Los Angeles Many people believe in streaks. In basketball, belief in the “hot hand” occurs when people think a player is […]

What you value should set out how you act and that how you represent what to possibly act upon: Aesthetics -> Ethics -> Logic.

This post is by Keith O’Rourke and as with all posts and comments on this blog, is just a deliberation on dealing with uncertainties in scientific inquiry and should not to be attributed to any entity other than the author. As with any critically-thinking inquirer, the views behind these deliberations are always subject to rethinking […]

Best correction ever: “Unfortunately, the correct values are impossible to establish, since the raw data could not be retrieved.”

Commenter Erik Arnesen points to this: Several errors and omissions occurred in the reporting of research and data in our paper: “How Descriptive Food Names Bias Sensory Perceptions in Restaurants,” Food Quality and Preference (2005) . . . The dog ate my data. Damn gremlins. I hate when that happens. As the saying goes, “Each […]

Some natural solutions to the p-value communication problem—and why they won’t work.

John Carlin and I write: It is well known that even experienced scientists routinely misinterpret p-values in all sorts of ways, including confusion of statistical and practical significance, treating non-rejection as acceptance of the null hypothesis, and interpreting the p-value as some sort of replication probability or as the posterior probability that the null hypothesis […]