Skip to content
Archive of posts filed under the Decision Theory category.

Glenn Shafer tells us about the origins of “statistical significance”

Shafer writes: It turns out that Francis Edgeworth, who introduced “significant” in statistics, and Karl Pearson, who popularized it in statistics, used it differently than we do. For Edgeworth and Pearson, “being significant” meant “signifying”. An observed difference was significant if it signified a real difference, and you needed a very small p-value to be […]

Josh Miller’s alternative, more intuitive, formulation of Monty Hall problem

Here it is: Three tennis players. Two are equally-matched amateurs; the third is a pro who will beat either of the amateurs, always. You blindly guess that Player A is the pro; the other two then play. Player B beats Player C. Do you want to stick with Player A in a Player A vs. […]

Deterministic thinking (“dichotomania”): a problem in how we think, not just in how we act

This has come up before: – Basketball Stats: Don’t model the probability of win, model the expected score differential. – Econometrics, political science, epidemiology, etc.: Don’t model the probability of a discrete outcome, model the underlying continuous variable – Thinking like a statistician (continuously) rather than like a civilian (discretely) – Message to Booleans: It’s […]

Here’s a puzzle: Why did the U.S. doctor tell me to drink more wine and the French doctor tell me to drink less?

This recent post [link fixed], on the health effects of drinking a glass of wine a day, reminds me of a story: Several years ago my cardiologist in the U.S. recommended that I drink a glass of red wine a day for health reasons. I’m not a big drinker—probably I average something less than 100 […]

It’s not just p=0.048 vs. p=0.052

Peter Dorman points to this post on statistical significance and p-values by Timothy Taylor, editor of the Journal of Economic Perspectives, a highly influential publication of the American Economic Association. I have some problems with what Taylor writes, but for now I’ll just take it as representing a certain view, the perspective of a thoughtful […]

Calibration and sharpness?

I really liked this paper, and am curious what other people think before I base a grant application around applying Stan to this problem in a machine-learning context. Gneiting, T., Balabdaoui, F., & Raftery, A. E. (2007). Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(2), 243–268. Gneiting […]

“I am a writer for our school newspaper, the BHS Blueprint, and I am writing an article about our school’s new growth mindset initiative.”

Caleb VanArragon writes: I am a student at Blaine High School in Blaine, Minnesota. I am a writer for our school newspaper, the BHS Blueprint, and I am writing an article about our school’s new growth mindset initiative. I was wondering if you would be willing to answer a couple of questions about your study […]

Beyond Power Calculations: Some questions, some answers

Brian Bucher (who describes himself as “just an engineer, not a statistician”) writes: I’ve read your paper with John Carlin, Beyond Power Calculations. Would you happen to know of instances in the published or unpublished literature that implement this type of design analysis, especially using your retrodesign() function [here’s an updated version from Andy Timm], […]

What can be learned from this study?

James Coyne writes: A recent article co-authored by a leading mindfulness researcher claims to address the problems that plague meditation research, namely, underpowered studies; lack of or meaningful control groups; and an exclusive reliance on subjective self-report measures, rather than measures of the biological substrate that could establish possible mechanisms. The article claims adequate sample […]

Here are some examples of real-world statistical analyses that don’t use p-values and significance testing.

Joe Nadeau writes: I’ve followed the issues about p-values, signif. testing et al. both on blogs and in the literature. I appreciate the points raised, and the pointers to alternative approaches. All very interesting, provocative. My question is whether you and your colleagues can point to real world examples of these alternative approaches. It’s somewhat […]

Conditional probability and police shootings

A political scientist writes: You might have already seen this, but in case not: PNAS published a paper [Officer characteristics and racial disparities in fatal officer-involved shootings, by David Johnson, Trevor Tress, Nicole Burkel, Carley Taylor, and Joseph Cesario] recently finding no evidence of racial bias in police shootings: Jonathan Mummolo and Dean Knox noted […]

You are invited to join Replication Markets

Anna Dreber writes: Replication Markets (RM) invites you to help us predict outcomes of 3,000 social and behavioral science experiments over the next year. We actively seek scholars with different voices and perspectives to create a wise and diverse crowd, and hope you will join us. We invite you – your students, and any other […]

Swimming upstream? Monitoring escaped statistical inferences in wild populations.

Anders Lamberg writes: In my mails to you [a few years ago], I told you about the Norwegian practice of monitoring proportion of escaped farmed salmon in wild populations. This practice results in a yearly updated list of the situation in each Norwegian salmon river (we have a total of 450 salmon rivers, but not […]

Pre-results review: Some results

Aleks Bogdanoski writes: I’m writing from the Berkeley Initiative for Transparency in the Social Sciences (BITSS) at UC Berkeley with news about pre-results review, a novel form of peer review where journals review (and accept) research papers based on their methods and theory — before any results are known. Pre-results review is motivated by growing […]

On deck through the end of 2019

Here’s what’s scheduled for the next six months: This is a great example for a statistics class, or a class on survey sampling, or a political science class How to read (in quantitative social science). And by implication, how to write. Causal inference with time-varying exposures Reproducibility problems in the natural sciences If you want […]

It’s a lot of pressure to write a book!

Regression and Other Stories is almost done, and I was spending a couple hours going through it starting from page 1, cleaning up imprecise phrasings and confusing points. . . . One thing that’s hard about writing a book is that there are so many places you can go wrong. A 500-page book contains something […]

How much is your vote worth?

Tyler Cowen writes: If it were legal, and you tried to sell your vote and your vote alone, you might not get much more than 0.3 cents. It depends where you live. If you’re not voting in any close elections, then the value of your vote is indeed close to zero. For example, I am […]

Tony nominations mean nothing

Someone writes: I searched up *Tony nominations mean nothing* and I found nothing. So I had to write this. There are currently 41 theaters that the Tony awards accept when nominating their choices. If we are being as generous as possible, we could say that every one of those theaters will be hosting a performance […]

Still at work on the piranha theorems

We’re still at work on the piranha theorems. But, in the meantime, I happened to show somebody this: There can be some large and predictable effects on behavior, but not a lot, because, if there were, then these different effects would interfere with each other, and as a result it would be hard to see […]

Why edit a journal? More generally, how to contribute to scientific discussion?

The other day I wrote: Journal editing is a volunteer job, and people sign up for it because they want to publish exciting new work, or maybe because they enjoy the power trip, or maybe out of a sense of duty—but, in any case, they typically aren’t in it for the controversy. Jon Baron, editor […]