Tuesday 28 Apr 2026, 4pm in CoDa E160:
What to do when your estimate is 1 standard error away from 0?
Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University
We provide a new answer to this simple yet very important question. Thinking clearly about this problem leads us to bring in many ideas in statistical analysis and computing, including causal identification, meta-analysis, Mister P, expectation propagation, decision analysis, experimental design, and the fundamental unity of Bayesian and frequentist statistics. We demonstrate our approach in examples from many applications, including medicine, social science, business, sports, and public policy.
This work is joint with Witold Więcek and Erik van Zwet.
In addition to all the above, I’ll probably drift into some related general topics such as the role of experimentation in science and engineering and the limitations of thinking about policy analysis in terms of causal inference.
Will there be a zoom link?
I don’t think so, but if it goes well maybe I’ll deliver a similar talk elsewhere.
What is the importance of this question?
Anon:
It’s an important question because it happens all the time, that an estimate is 1 standard error away from 0, and so it’s good to know what to do in this very common scenario.
Apologies for the nitpick, but the title of the talk isn’t a question (it’s a noun clause), so it shouldn’t end with a question mark.
@Andrew Gelman:
I am not sure I fully see the relevance of the problem. So, an estimator is one standard deviation away from zero. Assuming the model describes reality decently well, this means we can be quite confident the sign is correct. However, we are not particularly sure how large the ‘true’ value we are trying to estimate is (‘type M error’). But what else can we conclude from ‘one standard deviation away from zero’? ‘Tis not an awful lot of information to work with. To understand the relevance of one SD, we need to consider the context, i.e. the research question, non? Do you have a post or a paper that illustrates the point you are getting at?
I’d like to know “what to do” when the estimator is one standard deviation away from zero, but not two standard deviations away from zero. Would we write “we are quite confident the sign is correct, but not too confident”? While one can provide 50, 68, and 95 % credible intervals in a paper’s results section, how do we write about such a result in the discussion section? For a slope estimator, “We found a negative trend with 68 % probability but not 95 % probability” and provide that caveat every time we write about the trend? I never wish to dichotomize a result, but writing “we are quite confident” or “we have some evidence” is so vague. I find writing clearly and succinctly about uncertainty very challenging. I’d love to hear any tips!
Raphael:
1. Regarding you not seeing the relevance of the problem: Again, it’s relevant to me–I see estimates that are 1 standard deviation away from 0 all the time, and it’s very relevant to me what to do when this happens!
2. Yes, you do need to consider the context. That’s part of “what to do.” We do have a paper on the topic but it’s not finished yet. I’ll post it when it’s done.
appreciate any related papers/materials – thank you
I thought the answer was obvious. Assemble two underpowered papers with similar findings. Contact those authors. Jam the results together and reanalyze. Call it a meta-analysis. Publish.