A parable regarding changing standards on the presentation of statistical evidence

Now, the P-value Sneetches
Had tables with stars.
The Bayesian Sneetches
Had none upon thars.

Those stars weren’t so big. They were really so small.
You might think such a thing wouldn’t matter at all.

But, because they had stars, all the P-value Sneetches
Would brag, “We’re the best kind of Sneetch on the Beaches.
With their snoots in the air, they would sniff and they’d snort
“We’ll have nothing to do with the Bayesian sort!”
And whenever they met some, when they were out walking,
They’d hike right on past them without even talking.

When the P-value children went out to play ball,
Could a Bayesian get in the game… ? Not at all.
You only could play if your tables had stars
And the Bayesian children had none upon thars.

When the P-value Sneetches had frankfurter roasts
Or picnics or parties or PNAS toasts,
They never invited the Bayesian Sneetches.
They left them out cold, in the dark of the beaches.
They kept them away. Never let them come near.
And that’s how they treated them year after year.

Then ONE day, seems… while the Bayesian Sneetches
Were moping and doping alone on the beaches,
Just sitting there wishing their tables had stars…
A stranger zipped up in the strangest of cars!

“My friends,” he announced in a voice clear and keen,
“My name is Savage McJeffreys McBean.
And I’ve heard of your troubles. I’ve heard you’re unhappy.
But I can fix that. I’m the Fix-it-Up Chappie.
I’ve come here to help you. I have what you need.
And my prices are low. And I work at great speed.
And my work is one hundred per cent guaranteed!

Then, quickly Savage McJeffreys McBean
Put together a Bayes Factor machine.
And he said, “You want stars like a Star-Tabled Sneetch… ?
My friends, you can have them for three dollars each!”

“Just pay me your money and hop right aboard!”
So they clambered inside. Then the big machine roared
And it klonked. And it bonked. And it jerked. And it berked
And it bopped them about. But the thing really worked!
When the Bayesian Sneetches popped out, they had stars!
They actually did. They had stars upon thars!

Then they yelled at the ones who had stars at the start,
“We’re exactly like you! You can’t tell us apart.
We’re all just the same, now, you snooty old smarties!
And now we can go to your NPR parties.”

“Good grief!” groaned the ones who had stars at the first.
“We’re still the best Sneetches and they are the worst.
But, now, how in the world will we know,” they all frowned,
“If which kind is what, or the other way round?”

Then came McBean with a very sly wink.
And he said, “Things are not quite as bad as you think.
So you don’t know who’s who. That is perfectly true.
But come with me, friends. Do you know what I’ll do?
I’ll make you, again, the best Sneetches on beaches
And all it will cost you is ten dollars eaches.”

“P-value stars are no longer in style,” said McBean.
“What you need is a trip through my Replication Machine.
This wondrous contraption will take off your stars
So you won’t look like Sneetches who have them on thars.”
And that handy machine
Working very precisely
Removed all the stars from their tables quite nicely.

Then, with snoots in the air, they paraded about
And they opened their beaks and they let out a shout,
“We know who is who! Now there isn’t a doubt.
The best kind of Sneetches are Sneetches without!”

Then, of course, those with stars all got frightfully mad.
To be wearing a star now was frightfully bad.
Then, of course, old Savage McJeffreys McBean
Invited them into his Star-Off machine.

Then, of course from THEN on, as you probably guess,
Things really got into a horrible mess.
All the rest of that day, on those wild screaming beaches,
The fix-it-up Chappie kept fixing up Sneetches.
Off again! On Again!
In again! Out again!
Through the machines they raced round and about again,
Changing their stars every minute or two.
They kept paying money. They kept running through
Until neither the Plain nor the Star-Tables knew
Whether this one was that one… or that one was this one
Or which one was what one… or what one was who.

Then, when every last cent
Of their money was spent,
The Fix-it-Up Chappie packed up
And he went.

And he laughed as he drove
In his car up the beach,
“They never will learn.
No. You can’t teach a Sneetch!”

But McBean was quite wrong. I’m quite happy to say
That the Sneetches got really quite smart on that day,
The day they decided that Sneetches are Sneetches
And no kind of Sneetch is the best on the beaches
That day, all the Sneetches forgot about stars
And whether they had one, or not, upon thars.

[Original is on the web, for example here. I was inspired to construct the above adaptation after thinking of the series of public advice I’ve given over the years regarding prior distributions: first we recommended uniform priors, then scaled-inverse-Wishart and Cauchy and half-Cauchy, now LKJ and normal and half-normal and horseshoe, and who knows what in the future. And I used to recommend p-values and now I don’t. It’s hard to keep up . . .]

18 thoughts on “A parable regarding changing standards on the presentation of statistical evidence

  1. There are supposedly 2 comments here but I see none…

    If you have the number of comments per article available can you post an analysis of it by date? How much of an effect is this “timewarp” (as I’ve seen it called) having on the number of comments? If you have the data (no idea what you are logging), can you also do it by browser share?

  2. I’ve half given up on all of this statistical ‘progress’.

    I do basic experimental work so all of my stuff is pretty much t-tests and ANOVA with varying degrees of p-hacking (on demand for my supervisor/reviewers). You can calculate effect sizes from my t and F statistics and degrees of freedom. You can convert from frequentist to Bayes Factors – there are R packages and online calculators for it.

    Maybe for more sophisticated, larger and more complex datasets these things matter. But for me, the old statistics and the new statistics are the same statistics.

  3. This Tukey paper Christian Hennig pointed out here
    http://statmodeling.stat.columbia.edu/2018/12/04/bayes-statistics-reproducibility-many-serious-problems-statistics-practice-arise-bayesian-inference-not-bayesian-enough-frequentist-evaluation-not-frequentist/#comment-922331

    is most fitting here.

    “any consensus about which challenges [methods] are important will change over time, so what we do in this decade, will not be the same as either what we did a few decades ago, or what we will do a few decades from now. … We live in a paradoxical world, where the only true safety, true though limited, comes from admitting both our uncertainty and the incompleteness with which we are able to meet it.”

Leave a Reply to Sameera Daniels Cancel reply

Your email address will not be published. Required fields are marked *