Skip to content
Archive of entries posted by

Should we be suspicious of the vote counting in Bolivia?

I recently was sent two documents regarding the recent Bolivian presidential election. 1. Andrés Castro pointed me to this report from the Organization of American Stats reporting audit results from the election. The conclusions of this report are pretty harsh. For example: Given all the irregularities observed, it is impossible to guarantee the integrity of […]

A reduction in error rate of 400-600%: Pretty good, huh?

In comments to the previous post, Alexey Guzey points to this bit from his post on sleep legend Matthew Walker: In The Lancet, Walker writes: pilot studies have shown that when you limit trainee doctors to no more than a 16 h shift, with at least an 8 h rest opportunity before the next shift, […]

“Why We Sleep” update: some thoughts while we wait for Matthew Walker to respond to Alexey Guzey’s criticisms

So. It’s been a week since Alexey Guzey posted his wonderfully-titled article, “Matthew Walker’s ‘Why We Sleep’ Is Riddled with Scientific and Factual Errors.” I few days ago I reviewed Guzey’s post, and I summarized: I’ve not read Walker’s book and I don’t know anything about sleep research, so I won’t try to judge Guzey’s […]

“Whether something is statistically significant is itself a very random feature of data, so in this case you’re essentially outsourcing your modeling decision to a random number”

I happened to come across a post of mine that’s not scheduled until next April, and I noticed the above line, which I really liked, so I’m sharing it with you right now here. The comment relates to a common procedure in statistics, where researchers decide exclude potentially important interactions from their models, just because […]

No, Bayes does not like Mayor Pete. (Pitfalls of using implied betting market odds to estimate electability.)

Asher Meir points to this amusing post from Greg Mankiw, who writes: Who has the best chance of beating Donald Trump? A clue can be found using Bayes Theorem. Here is the logic. Let A be the event that a candidate wins the general election, and B be the event that a candidate wins his […]

Psychological Methods Feed

Someone writes: I’m emailing you about an email service I provide for some of the best blogs and podcasts on psychological methods. People can sign up for free and receive daily/real time emails containing the blog post(s). Below is the list. I had no idea there were so many blogs that discussed psychological methods. Oscar […]

What happens when frauds are outed because of whistleblowing?

Ashwin Malshe: I would like to bring to your attention a recent controversy in accounting research. It relates to how extreme values may drive results in observational studies. However, this issue is more complex in this specific case because the event (corporate whistleblowing) is rather rare, showing up only about 20% of the time in […]

What does a “statistically significant difference in mortality rates” mean when you’re trying to decide where to send your kid for heart surgery?

Keith Turner writes: I am not sure if you caught the big story in the New York Times last week about UNC’s pediatric heart surgery program, but part of the story made me interested to know if you had thoughts: Doctors were told that the [mortality] rate had improved in recent years, but the program […]

What’s the evidence on the effectiveness of psychotherapy?

Kyle Dirck points us to this article by John Sakaluk, Robyn Kilshaw, Alexander Williams, and Kathleen Rhyner in the Journal of Abnormal Psychology, which begins: Empirically supported treatments (or therapies; ESTs) are the gold standard in therapeutic interventions for psychopathology. Based on a set of methodological and statistical criteria, the APA [American Psychological Association] has […]

Break out the marshmallows, friends: Ego depletion is due to change sign!

In a paper amusingly titled, “Ego depletion may disappear by 2020,” Miguel Vadillo (link from Kevin Lewis) writes: Ego depletion has been successfully replicated in hundreds of studies. Yet the most recent large-scale Registered Replication Reports (RRR), comprising thousands of participants, have yielded disappointingly small effects, sometimes even failing to reach statistical significance. Although these […]

Stan saves Australians $20 billion

Jim Savage writes: Not sure if you knew, but Stan was used in the Australian Productivity Commission’s review of the Australian retirement savings system. Their review will likely affect the regulation on $2 trillion of retirement savings, possibly saving Australians around $20-50 billion in fees over the next decade. OK, we can now officially say […]

My talk at Yale this Thursday

It’s the Quantitative Research Methods Workshop, 12:00-1:15 p.m. in Room A002 at ISPS, 77 Prospect Street Slamming the sham: A Bayesian model for adaptive adjustment with noisy control data Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University It is not always clear how to adjust for control data in causal inference, […]

Is Matthew Walker’s “Why We Sleep” Riddled with Scientific and Factual Errors?

Asher Meir points to this hilarious post by Alexey Guzey entitled, Matthew Walker’s “Why We Sleep” Is Riddled with Scientific and Factual Errors. Just to start with, the post has a wonderful descriptive title. And the laffs start right away: Positively Nabokovian, I’d say. I mean it. The above table of contents makes me want […]

In research as in negotiation: Be willing to walk away, don’t paint yourself into a corner, leave no hostages to fortune

There’s a saying in negotiation that the most powerful asset is the ability to walk away from the deal. Similarly, in science (or engineering, business decision making, etc.), you have to be willing to give up your favorite ideas. When I look at various embarrassing examples in science during the past decade, a common thread […]

Why do a within-person rather than a between-person experiment?

Zach Horne writes: A student of mine was presenting at the annual meeting of the Law and Society Association. She sent me this note after she gave her talk: I presented some research at LSA which used a within subject design. I got attacked during the Q&A session for using a within subjects design and […]

Ballot order effects in the news; I’m skeptical of the claimed 5% effect.

Palko points us to this announcement by Marc Elias: BREAKING: In major court victory ahead of 2020, Florida federal court throws out state’s ballot order law that lists candidates of the governor’s party first on every ballot for every office. Finds that it gave GOP candidates a 5% advantage. @AndrewGillum lost in 2018 by .4% […]

Should we mind if authorship is falsified?

In a typically thought-provoking piece, Louis Menand asks, “Should we mind if a book is a hoax?” In his article, Menand (whose father taught the best course I ever took at MIT, in which we learned that eternal vigilance is the price of liberty) focuses on imaginative literature written by white people but attributed to […]

I (inadvertently) misrepresented others’ research in a way that made my story sound better.

During a recent talk (I think it was this one on statistical visualization), I spent a few minutes discussing a political science experiment involving social stimuli and attitudes toward redistribution. I characterized the study as being problematic for various reasons (for background, see this post), and I remarked that you shouldn’t expect to learn much […]

Is “abandon statistical significance” like organically fed, free-range chicken?

The question: is good statistics scalable? This comes up a lot in discussions on abandoning statistical significance, null-hypothesis significance testing, p-value thresholding, etc. I recommend accepting uncertainty, but what if it’s decision time—what to do? How can the world function if the millions of scientific decisions currently made using statistical significance somehow have to be […]

Dow Jones probability calculation

Here’s a cute one for your intro probability class. Karen Langley from the Wall Street Journal asks: What is the probability of the Dow Jones Industrial Average closing unchanged from the day before, as it did yesterday? To answer this question we need to know two things: 1. How much does the Dow Jones average […]