Skip to content
Archive of posts filed under the Miscellaneous Science category.

Let’s do preregistered replication studies of the cognitive effects of air pollution—not because we think existing studies are bad, but because we think the topic is important and we want to understand it better.

In the replication crisis in science, replications have often been performed of controversial studies on silly topics such as embodied cognition, extra-sensory perception, and power pose. We’ve been talking recently about replication being something we do for high-quality studies on important topics. That is, the point of replication is not the hopeless endeavor of convincing […]

The value (or lack of value) of preregistration in the absence of scientific theory

Javier Benitez points us to this 2013 post by psychology researcher Denny Borsboom. I have some thoughts on this article—in particular I want to compare psychology to other social science fields such as political science and economics—but first let me summarize it. Preregistration and open science Borsboom writes: In the past few months, the Center […]

Woof! for descriptive statistics

Gary Smith points to this news article in the Economist, “WOOF, CAKE, BOOM: stocks with catchy tickers beat the market,” which reports: In a study published in 2009 by Gary Smith, Alex Head and Julia Wilson of Pomona College in California, a group of people were asked to pick American public companies with “clever” tickers. […]

Junk Science Then and Now

Many years ago, Martin Gardner wrote a book, Fads and Fallacies in the Name of Science, featuring chapters on flat earth and eccentric astronomy theories, UFO’s, alternative physics, dowsing, creationism, Lysenkoism, pyramid truthers, medical quacks, food faddists, ESP, etc. The Wikipedia page summarizes Gardner’s book as follows: Fads and Fallacies in the Name of Science […]

As usual, I agree with Paul Meehl: “It is not a reform of significance testing as currently practiced in soft-psych. We are making a more heretical point than any of these: We are attacking the whole tradition of null-hypothesis refutation as a way of appraising theories.”

Javier Benitez sends along the about quote from Meehl’s 1990 article, Appraising and Amending Theories: The Strategy of Lakatosian Defense and Two Principles that Warrant It: I wish that Shalizi and I had known about that Meehl article when we were writing this. When we wrote that article, we were thinking about Bayesian statistics and […]

Different challenges in replication in biomedical vs. social sciences

Summary – In biological sciences, it might be reasonable to expect real effects to replicate, but carrying out the measurement required to study this replication is difficult for technical reasons. – In social sciences, it might be straightforward to replicate the data collection, but effects of interest could vary so much by context that replication […]

Recent unpublished papers

You perhaps notice our published papers when they appear in journals: [2020] Expectation propagation as a way of life: A framework for Bayesian inference on partitioned data. {\em Journal of Machine Learning Research} {\bf 21}, 1–53. (Aki Vehtari, Andrew Gelman, Tuomas Sivula, Pasi Jylanki, Dustin Tran, Swupnil Sahai, Paul Blomstedt, John P. Cunningham, David Schiminovich, […]

Are GWAS studies of IQ/educational attainment problematic?

Nick Matzke writes: I wonder if you or your blog-colleagues would be interested in giving a quick blog take on the recent studies that do GWAS (Genome-Wide-Association Studies) on “traits” like IQ, educational attainment, and income? Matzke begins with some background: The new method for these studies is to claim that a “polygenic score” can […]

Will decentralised collaboration increase the robustness of scientific findings in biomedical research? Some data and some causal questions.

Mark Tuttle points to this press release, “Decentralising science may lead to more reliable results: Analysis of data on tens of thousands of drug-gene interactions suggests that decentralised collaboration will increase the robustness of scientific findings in biomedical research,” and writes: In my [Tuttle’s] opinion, the explanation is more likely to be sociological – group […]

Four projects in the intellectual history of quantitative social science

1. The rise and fall of game theory. My impression is that game theory peaked in the late 1950s. Two classics from that area are Philip K. Dick’s “Solar Lottery” and R. Duncan Luce and Howard Raiffa’s “Games and Decisions.” The latter is charming in its retro attitude that all that remained were some minor […]

Of Manhattan Projects and Moonshots

Palko writes: I think we have reversed the symbolic meaning of a Manhattan project and a moonshot. He explains: The former has come to mean a large, focus, and dedicated commitment to rapidly addressing a challenging but solvable problem. The second has come to mean trying to do something so fantastic it seems impossible. But, […]

The Generalizer

I just saw Beth Tipton speak at the Institute of Education Sciences meeting on The Generalizer, a tool that she and her colleagues developed for designing education studies with the goal of getting inferences for the population. It’s basically MRP, but what is innovative here is the application of these ideas at the design stage. […]

It happens all the time

Under the subject line, “Here is another one for your archive,” someone points me to a news article and writes: What would have happened had the guy not discovered his coding error? Or what if he had, but the results were essentially unchanged? My guess if that nothing would happen until someone got the data […]

Science is science writing; science writing is science

Meehan Crist writes: There is a belief, particularly prevalent among scientists, that science writing is more or less glorified PR – scientists do the intellectual work of discovery and writers port their findings from lab to public – but [Rachel Carson’s 1962 book] Silent Spring is a powerful reminder that great science writing can expand […]

Horns! Have we reached a new era in skeptical science journalism? I hope so.

Pointing us to this news article from Aylin Woodward, “No, we’re probably not growing horns from our heads because of our cellphone use — here’s the real science,” Jordan Anaya writes: I haven’t looked into it, but seems like your basic terrible study with an attention grabbing headline. Pretty much just mention cell phone use […]

Judith Rich Harris on the garden of forking paths

Ethan Ludwin-Peery writes: I finally got around to reading The Nurture Assumption and I was surprised to find Judith Rich Harris quite lucidly describing the garden of forking paths / p-hacking on pages 17 and 18 of the book. The edition I have is from 2009, so it predates most of the discussion of these […]

“There is this magic that our DNA enables”

I was just at a talk where a computer scientist was making dramatic claims for the value of human decision making. This is typically a crowd-pleasing position to take—after all, we in the audience are humans and we want to hear how great we are. What really riled me was when the speaker said, “There […]

The tone problem in psychology

Are you tone deaf? Find out here. P.S. Link updated. I guess things can change in 6 months!

Don’t believe people who say they can look at your face and tell that you’re lying.

Kevin Lewis points us to this article, Lessons From Pinocchio: Cues to Deception May Be Highly Exaggerated, by Timothy Luke, which begins: Deception researchers widely acknowledge that cues to deception—observable behaviors that may differ between truthful and deceptive messages—tend to be weak. Nevertheless, several deception cues have been reported with unusually large effect sizes, and […]

“Why We Sleep” update: some thoughts while we wait for Matthew Walker to respond to Alexey Guzey’s criticisms

So. It’s been a week since Alexey Guzey posted his wonderfully-titled article, “Matthew Walker’s ‘Why We Sleep’ Is Riddled with Scientific and Factual Errors.” I few days ago I reviewed Guzey’s post, and I summarized: I’ve not read Walker’s book and I don’t know anything about sleep research, so I won’t try to judge Guzey’s […]