A coauthor writes: I really like the paper [we are writing] as it is. My only criticism of it perhaps would be that we present this great new method and discuss all of its merits, but we do not really discuss when it fails / what its downsides are. Are there any cases where the […]
Archive of posts filed under the Decision Theory category.
The best is the enemy of the good. It is also the enemy of the not so good.
This post is by Phil Price, not Andrew. The Ocean Cleanup Project’s device to clean up plastic from the Great Pacific Garbage Patch is back in the news because it is back at work and is successfully collecting plastic. A bunch of my friends are pretty happy about it and have said so on social […]
What’s the p-value good for: I answer some questions.
Martin King writes: For a couple of decades (from about 1988 to 2006) I was employed as a support statistician, and became very interested in the p-value issue; hence my interest in your contribution to this debate. (I am not familiar with the p-value ‘reconciliation’ literature, as published after about 2005.) I would hugely appreciate […]
Automation and judgment, from the rational animal to the irrational machine
Virgil Kurkjian writes: I was recently going through some of your recent blog posts and came across Using numbers to replace judgment. I recently wrote something about legible signaling which I think helps shed some light on exactly what causes the bureaucratization of science and maybe what we can do about it. In short I […]
Glenn Shafer: “The Language of Betting as a Strategy for Statistical and Scientific Communication”
Glenn Shafer writes: I have joined the immense crowd writing about p-values. My proposal is to replace them with betting outcomes: the factor by which a bet against the hypothesis multiplies the money it risks. This addresses the desideratum you and Carlin identify: embrace all the uncertainty. No one will forget that the outcome of […]
My talk at the Brookings Institution this Fri 11am
The replication crisis in science: Does it matter for policy? Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University I argue that policy analysts should care about the replication crisis for three reasons: (1) High-profile policy claims have been systematically exaggerated; (2) This has implications for how to conduct and interpret new […]
Are statistical nitpickers (e.g., Kaiser Fung and me) getting the way of progress or even serving the forces of evil?
As Ira Glass says, today we have a theme and some variations on this theme. Statistical nitpickers: Do they cause more harm than good? I’d like to think we cause more good than harm, but today I want to consider the counter-argument, that, even when we are correct on the technical merits, we statisticians should […]
The Map Is Not The Territory
This post is by Phil Price, not Andrew. My wife and I are building a new house, or, rather, paying trained professionals to build one for us. We are trying to make the house as environmentally benign as we reasonably can: ducted mini-split heating, heat pump water heater, solar panels, heat-recovery ventilator, sustainably harvested lumber, […]
Econ corner: A rational reason (beyond the usual “risk aversion” or concave utility function) for wanting to minimize future uncertainty in a decision-making setting
Eric Rasmusen sends along a paper, Option Learning as a Reason for Firms to Be Averse to Idiosyncratic Risk, and writes: It tries to distinguish between two kinds of risk. The distinction is between uncertainty that the firm will learn about, and uncertainty that will be bumping the profit process around forever. It’s not the […]
Glenn Shafer tells us about the origins of “statistical significance”
Shafer writes: It turns out that Francis Edgeworth, who introduced “significant” in statistics, and Karl Pearson, who popularized it in statistics, used it differently than we do. For Edgeworth and Pearson, “being significant” meant “signifying”. An observed difference was significant if it signified a real difference, and you needed a very small p-value to be […]
Josh Miller’s alternative, more intuitive, formulation of Monty Hall problem
Here it is: Three tennis players. Two are equally-matched amateurs; the third is a pro who will beat either of the amateurs, always. You blindly guess that Player A is the pro; the other two then play. Player B beats Player C. Do you want to stick with Player A in a Player A vs. […]
Deterministic thinking (“dichotomania”): a problem in how we think, not just in how we act
This has come up before: – Basketball Stats: Don’t model the probability of win, model the expected score differential. – Econometrics, political science, epidemiology, etc.: Don’t model the probability of a discrete outcome, model the underlying continuous variable – Thinking like a statistician (continuously) rather than like a civilian (discretely) – Message to Booleans: It’s […]
Here’s a puzzle: Why did the U.S. doctor tell me to drink more wine and the French doctor tell me to drink less?
This recent post [link fixed], on the health effects of drinking a glass of wine a day, reminds me of a story: Several years ago my cardiologist in the U.S. recommended that I drink a glass of red wine a day for health reasons. I’m not a big drinker—probably I average something less than 100 […]
It’s not just p=0.048 vs. p=0.052
Peter Dorman points to this post on statistical significance and p-values by Timothy Taylor, editor of the Journal of Economic Perspectives, a highly influential publication of the American Economic Association. I have some problems with what Taylor writes, but for now I’ll just take it as representing a certain view, the perspective of a thoughtful […]
Calibration and sharpness?
I really liked this paper, and am curious what other people think before I base a grant application around applying Stan to this problem in a machine-learning context. Gneiting, T., Balabdaoui, F., & Raftery, A. E. (2007). Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(2), 243–268. Gneiting […]
“I am a writer for our school newspaper, the BHS Blueprint, and I am writing an article about our school’s new growth mindset initiative.”
Caleb VanArragon writes: I am a student at Blaine High School in Blaine, Minnesota. I am a writer for our school newspaper, the BHS Blueprint, and I am writing an article about our school’s new growth mindset initiative. I was wondering if you would be willing to answer a couple of questions about your study […]
Beyond Power Calculations: Some questions, some answers
Brian Bucher (who describes himself as “just an engineer, not a statistician”) writes: I’ve read your paper with John Carlin, Beyond Power Calculations. Would you happen to know of instances in the published or unpublished literature that implement this type of design analysis, especially using your retrodesign() function [here’s an updated version from Andy Timm], […]
What can be learned from this study?
James Coyne writes: A recent article co-authored by a leading mindfulness researcher claims to address the problems that plague meditation research, namely, underpowered studies; lack of or meaningful control groups; and an exclusive reliance on subjective self-report measures, rather than measures of the biological substrate that could establish possible mechanisms. The article claims adequate sample […]
Here are some examples of real-world statistical analyses that don’t use p-values and significance testing.
Joe Nadeau writes: I’ve followed the issues about p-values, signif. testing et al. both on blogs and in the literature. I appreciate the points raised, and the pointers to alternative approaches. All very interesting, provocative. My question is whether you and your colleagues can point to real world examples of these alternative approaches. It’s somewhat […]
Conditional probability and police shootings
A political scientist writes: You might have already seen this, but in case not: PNAS published a paper [Officer characteristics and racial disparities in fatal officer-involved shootings, by David Johnson, Trevor Tress, Nicole Burkel, Carley Taylor, and Joseph Cesario] recently finding no evidence of racial bias in police shootings: Jonathan Mummolo and Dean Knox noted […]