Skip to content
Archive of entries posted by

Instead of replicating studies with problems, let’s replicate the good studies. (Consider replication as an honor, not an attack.)

Commenter Thanatos Savehn pointed to an official National Academy of Sciences report on Reproducibility and Replicability that included the following “set of criteria to help determine when testing replicability may be warranted”: 1) The scientific results are important for individual decision-making or for policy decisions. 2) The results have the potential to make a large […]

Participate in Brazilian Reproducibility Initiative—even if you don’t live in South America!

Anna Dreber writes: There’s a big reproducibility initiative in Brazil on biomedical research led by Olavo Amaral and others, which is an awesome project where they are replicating 60 studies in Brazilian biomedical research. We (as usual lots of collaborators) are having a prediction survey and prediction markets for these replications – would it be […]

To do: Construct a build-your-own-relevant-statistics-class kit.

Alexis Lerner, who took a couple of our courses on applied regression and communicating data and statistics, designed a new course, “Jews: By the Numbers,” at the University of Toronto: But what does it mean to work with data and statistics in a Jewish studies course? For Lerner, it means not only teaching her students […]

The hot hand and playing hurt

So, was chatting with someone the other day and it came up that I sometimes do sports statistics, and he told me how he read that someone did some research finding that the hot hand in basketball isn’t real . . . I replied that the hot hand is real, and I recommended he google […]

Hey! Participants in survey experiments aren’t paying attention.

Gaurav Sood writes: Do survey respondents account for the hypothesis that they think people fielding the survey have when they respond? The answer, according to Mummolo and Peterson, is not much. Their paper also very likely provides the reason why—people don’t pay much attention. Figure 3 provides data on manipulation checks—the proportion guessing the hypothesis […]

When Prediction Markets Fail

A few years ago, David Rothschild and I wrote: Prediction markets have a strong track record and people trust them. And that actually may be the problem right now. . . . a trader can buy a contract on an outcome, such as the Democratic nominee to win the 2016 presidential election, and it will […]

Australian polls failed. They didn’t do Mister P.

Neil Diamond writes: Last week there was a federal election in Australia. Contrary to expectations and to opinion polls, the Government (a coalition between the Liberal (actually conservative) and National parties, referred to as LNP or the Coalition) was returned with an increased majority defeating the Australian Labor Party (ALP or Labor, no “u”). Voting […]

Battle for the headline: Hype and the effect of statistical significance on the ability of journalists to engage in critical thinking

A few people pointed me to this article, “Battle for the thermostat: Gender and the effect of temperature on cognitive performance,” which received some uncritical press coverage here and here. And, of course, on NPR. “543 students in Berlin, Germany” . . . good enuf to make general statements about men and women, I guess! […]

Padres need Stan

Cody Zupnick writes: I’m working in baseball research for the San Diego Padres, and we’re looking for new people, potentially with Stan experience. Would you mind seeing if any of your readers have any interest? Cool!

Epic Pubpeer thread continues

Here. (background here and here)

“I’m sick on account I just ate a TV dinner.”

I recently read “The Shadow in the Garden,” a book by James Atlas that’s a mix of memoir about his experiences as a biographer of poet Delmore Schwartz and novelist Saul Bellow, and various reflections and anecdotes about biography-writing more generally. I enjoyed the book so much that I’m pretty much just gonna have a […]

Have prices have risen more quickly for people at the bottom of the income distribution than for those at the top? Lefty window-breakers wait impatiently while economists struggle to resolve this dispute.

Palko points us to this post by Mike Konczal pointing to this news article by Annie Lowrey reporting on research by Christopher Wimer, Sophie Collyer, and Xavier Jaravel finding that “prices have risen more quickly for people at the bottom of the income distribution than for those at the top.” This new result counters an […]

Columbia statistics department is hiring!

Official announcement is below. Please please apply to these faculty and postdoc positions. We really need some people who do serious applied work, especially in social sciences. Obv these will be competitive, but please give it a shot, because we’d like to have some strong applied candidates in the mix for all of these positions. […]

The incentives are all wrong (causal inference edition)

I was talking with some people the other day about bad regression discontinuity analyses (see this paper for some statistical background on the problems with these inferences), examples where the fitted model just makes no sense. The people talking with me asked the question: OK, we agree that the published analysis was no good. What […]

“The paper has been blind peer-reviewed and published in a highly reputable journal, which is the gold standard in scientific corroboration. Thus, all protocol was followed to the letter and the work is officially supported.”

Robert MacDonald points us to this news article by Esther Addley: It’s another example of what’s probably bad science being published in a major journal, where other researchers point out its major flaws and the author doubles down. In this case, the University of Bristol has an interesting reaction. It’s pulled down its article praising […]

How to teach sensible elementary statistics to lower-division undergraduates?

Kevin Carlson writes: Though my graduate education is in mathematics, I teach elementary statistics to lower-division undergraduates. The traditional elementary statistics curriculum culminates in confidence intervals and hypothesis tests. Most students can learn to perform these tests, but few understand them. It seems to me that there’s a great opportunity to reform the elementary curriculum […]

Hey, Stan power users! PlayStation is Hiring.

Imad writes: The Customer Lifecycle Management team at PlayStation is looking to hire a Senior Data Modeler (i.e. Data Scientist). DM me if you like building behavioral models and working with terabytes of data. You’ll have the opportunity use whatever tools you want (e.g. Stan) to build your models. I’m not into videogames myself, but […]

The dropout rate in his survey is over 60%. What should he do? I suggest MRP.

Alon Honig writes: I work for a cpg company that conducts longitudinal surveys for analysis of customer behavior. In particular they wanted to know how people are interacting with our product. Unfortunately the designers of these surveys put so many questions (100+) that the dropout rate (those that did not complete the survey) was over […]

The climate economics echo chamber: Gremlins and the people (including a Nobel prize winner) who support them

Jay Coggins, a professor of applied economics at the university of Minnesota, writes in with some thoughts about serious problems of within the field of environmental economics: Your latest on Tol [a discussion of a really bad paper he published in The Review of Environmental Economics and Policy, “the official journal of the Association of […]

I’m no expert

A journalist contacted me and wanted me to answer some questions. I said, sure, send them over by email, and here’s what came: ** The European Union has announced that the Special Financial Mechanism (SPV) will be implemented soon. What is your assessment of this mechanism? And how much do you think PSV could help […]