This is Jessica. Recently I asked a question about when a model developer should try to estimate the relationship between model predictions and the observed behavior that results when people have access to the model predictions. Kenneth Tay suggested a recent machine learning paper on Performative Prediction by Juan Perdomo Tijana Zrnic. Celestine Mendler-Dunner and […]
Archive of posts filed under the Decision Theory category.
Raymond Smullyan on Ted Cruz, Al Sharpton, and those scary congressmembers
Palko shares this fun logic puzzle from the great Raymond Smullyan which also has obvious implications for modern politics: Inspector Craig of Scotland Yard was called to Transylvania to solve some cases of vampirism. Arriving there, he found the country inhabited both by vampires and humans. Vampires always lie and humans always tell the truth. […]
Open data and quality: two orthogonal factors of a study
It’s good for a study to have open data, and it’s good for the study to be high quality. If for simplicity we dichotomize these variables, we can find lots of examples in all four quadrants: – Unavailable data, low quality: The notorious ESP paper from 2011 and tons of papers published during that era […]
“Prediction Markets in a Polarized Society”
Rajiv Sethi writes about some weird things in election prediction markets, such as Donald Trump being given a one-in-eight chance of being the election winner . . . weeks after he’d lost the election. Sethi writes: There’s a position size limit of $850 dollars per contract in this market, which also happens to have hit […]
Which sorts of posts get more blog comments?
Paul Alper writes: Some of your blog postings elicit many responses and some, rather few. Have you ever thought of displaying some sort of statistical graph illustrating the years of data? For example, sports vs. politics, or responses for one year vs. another (time series), winter vs. summer, highly technical vs. breezy. I’ve not done […]
More on that credulity thing
I see five problems here that together form a feedback loop with bad consequences. Here are the problems: 1. Irrelevant or misunderstood statistical or econometric theory; 2. Poorly-executed research; 3. Other people in the field being loath to criticize, taking published or even preprinted claims as correct until proved otherwise; 4. Journalists taking published or […]
Rob Tibshirani, Yuling Yao, and Aki Vehtari on cross validation
Rob Tibshirani writes: About 9 years ago I emailed you about our new significance result for the lasso. You wrote about in your blog. For some reason I never saw that full blog until now. I do remember the Stanford-Berkeley Seminar in 1994 where I first presented the lasso and you asked that question. Anyway, […]
Relative vs. absolute risk reduction . . . 500 doctors want to know!
Some stranger writes: What are your thoughts on this paper? Especially the paragraph on page 6 “Similar to the critical appraisal ….. respectively”. There are many of us MD’s who are quite foxed. If you blog about it, please don’t mention my name and just say a doctor on a 500-member listserv asked you about […]
Understanding the value of bloc voting, using the Congressional Progressive Caucus as an example:
Daniel Stock writes: I’m a public policy PhD student, interested in economic policy and a bit of political science. I recently saw that the Congressional Progressive Caucus instituted bloc voting rules a few months ago: if at least two thirds of them agree on a bill or amendment, then all CPC members are bound to […]
Call for a moratorium on the use of the term “prisoner’s dilemma”
Palko writes: I’m not sure what the best way to get the ball rolling here would be (perhaps a kickstarter?) but we need to have a strictly enforced rule that no journalist or pundit is allowed to mention the prisoner’s dilemma for the next five or ten years, however long it takes to learn to […]
I have these great April Fools ideas but there’s no space for them in the margin of this blog
Not really. Actually I have no good April Fools ideas this year. Usually I write an April Fools post months in advance, but it’s been such an overwhelming year in so many ways with a pandemic, an attempted overthrow of the government, and lots more, that somehow the inspiration never came. Bu we should have […]
Many years ago, when he was a baby economist . . .
Jonathan Falk writes: Many years ago, when I was a baby economist, a fight broke out in my firm between two economists. There was a question as to whether a particular change in the telecommunications laws had spurred productivity improvements or not. There a trend of x% per year in productivity improvements that had gone […]
What is the landscape of uncertainty outside the clinical trial’s methods?
I live in the province of British Columbia in the country of Canada (right, this post is not by Andrew, it is by Lizzie). Recently one of our top provincial health officials, Dr. Bonnie Henry, has received extra scrutiny based on her decision to delay second doses of the vaccine. The general argument against this […]
Bayesian methods and what they offer compared to classical econometrics
A well-known economist who wishes to remain anonymous writes: Can you write about this agent? He’s getting exponentially big on Twitter. The link is to an econometrician, Jeffrey Wooldridge, who writes: Many useful procedures—shrinkage, for example—can be derived from a Bayesian perspective. But those estimators can be studied from a frequentist perspective, and no strong […]
A new approach to pandemic control by informing people of their social distance from exposure
Po-Shen Lo, a mathematician who works in graph theory, writes about a new approach he devised for pandemic control. He writes: The significance of this new approach is potentially very high, because it not only can improve the current situation, but it would permanently add a new orthogonal tool to the toolbox for pandemic control, […]
Here is how you should title the next book you write.
I was talking with someone about book titles. I liked the title Red State Blue State Rich State Poor State when I came up with it, but the book did not sell as well as I hoped (not that I thought it would sell enough to make me lots of money; I’m just using sales […]
The Mets are hiring
Des McGowan writes: We are looking to hire multiple full time analysts/senior analysts to join the Baseball Analytics department at the New York Mets. The roles will involve building, testing, and presenting statistical models that inform decision-making in all facets of Baseball Operations. These positions require a strong background in complex statistics and data analytics, […]
Statisticians don’t use statistical evidence to decide what statistical methods to use. Also, The Way of the Physicist.
David Bailey, a physicist at the University of Toronto, writes: I thought you’d be pleased to hear that a student in our Advanced Physics Lab spontaneously used Stan to analyze data with significant uncertainties in both x and y. We’d normally expect students to use python and orthogonal distance regression, and STAN is never mentioned […]
Is the right brain hemisphere more analog and Bayesian?
Oliver Schultheiss writes: I recently commented one of your posts (I forgot which one) with a reference to evidence suggesting that the right brain hemisphere may be in a better position to handle numbers and probabilistic predictions. Yesterday I came across the attached paper by Filipowicz, Anderson, & Danckert (2016) that may be of some […]
“Our underpowered trial provides no indication that X has a positive or negative effect on Y”
It’s rare to see researchers say flat-out that an experimental result leaves them uncertain. There seems to be such a temptation to either declare victory with statistical significance (setting the significance level to 0.1 if necessary to clear the bar) or to claim that weak and noisy results are “suggestive” or, conversely, to declare non-significance […]