I was drunk at the podium, and I knew my results weren’t strong

So I left in mid-lecture tempted by a reform song
The plenary hall it shifted as they turned to watch me leave
And I pulled a little p-curve from the pocket in my sleeve

The variation it was stronger to my dichotomizing eyes
Than the light which had blinded me with Fisher’s own half-lies
Yes mid-sunday morning, methodological terrorists sat
Round a mounting pile of fishing, and Ioannadis he spat
And they kicked out a chair and they showed me to sit
Then they started back critiquing in that meta-science pit
They were replicating and failing with a growing sense of dread
While the authors wrote to APS, disputing what they said
Well I watched Kahneman deny it, and I watched Kahneman rescind
In a file drawer disaster, and I saw where I’d been was a science trash bin

Let them boldly claim on unbucking high metrics
They are pure noise nudge stories which merely entertain courses
That these log heads teach just to spread the thought they’re right
In their NHSTy-induced unreproducible night
Yes I thought I saw new light, an informed prior which dimmed
The preached F-stats with which ESP tests prove whims
Oh the fear-mongering it paled next to this data-policing chorus
Calling undisclosed freedoms, dense as a forest

There were DOIs linked in sympathy, gilded the glaring
Of these TED-loathe companions, to bypass their theorizing
Some need for an answer that was kin to truthiness
Which coarse steps once preregistered, brought faith and brought trust
I saw the dependence, yes an inherent weakness
With holdouts withholded data and hindered all frankness
That open science supported what designs couldn’t stand
On their own from measurement error, whether or not they’re run again
They are slave to prescriptions, but to them alone
Each of them bootstrapping fear of assuming unknowns

Well wherever folks gather, to trust peer review
They are each one a sinner, they are each one a fool
For if I mine my data, and I submit my song
I have no analysis companion, a-trailing along
To imagine a sharing of data I earned
To question generalizations I’ve drawn without concern

Truth’s in my story, and truth’s in the flood
The reform is inside of you, outside of you is made of mud

 

(I wrote this as an adaption of the lyrics of Will Oldham’s “I was drunk at the pulpit”, recorded during a John Peel session in 2002 which I heard over the holidays and immediately started mentally rewriting. Something about the renunciatory message reminded me of science reform, I guess. Many parts of my version were inspired by posts on this blog, and by critiques of proposed methodological reforms. Though not those dark last couple lines!)

5 thoughts on “I was drunk at the podium, and I knew my results weren’t strong

  1. The line near the end about sinners reminds me of something from a post a few years ago:

    As they say in AA (or someplace like that), it’s only after you admit you’re a sinner that you can be redeemed. I know that I’m a sinner. I make statistical mistakes all the time. It’s unavoidable.

    • Yeah, I think no one wants to admit that they make mistakes all the time since no one else is really admitting it, and at least in academia. So unless you’re absolutely sure no one will ever use it against you if often seems safer not to risk talking about your mistakes and bad decisions, at least before tenure. But good data analysis while being human is hard, just like figuring out what’s going to fix science is hard. More widespread acceptance that we are all often fumbling and tripping around when we do both would be great.

Leave a Reply to Andrew Cancel reply

Your email address will not be published. Required fields are marked *