Skip to content
Search results for file drawer

Interrogating p-values

This article is a discussion of a paper by Greg Francis for a special issue, edited by E. J. Wagenmakers, of the Journal of Mathematical Psychology. Here’s what I wrote: Much of statistical practice is an effort to reduce or deny variation and uncertainty. The reduction is done through standardization, replication, and other practices of […]

“A Vast Graveyard of Undead Theories: Publication Bias and Psychological Science’s Aversion to the Null”

Erin Jonaitis points us to this article by Christopher Ferguson and Moritz Heene, who write: Publication bias remains a controversial issue in psychological science. . . . that the field often constructs arguments to block the publication and interpretation of null results and that null results may be further extinguished through questionable researcher practices. Given […]

Likelihood Ratio ≠ 1 Journal

Dan Kahan writes: The basic idea . . . is to promote identification of study designs that scholars who disagree about a proposition would agree would generate evidence relevant to their competing conjectures—regardless of what studies based on such designs actually find. Articles proposing designs of this sort would be selected for publication and only […]

Proposals for alternative review systems for scientific work

I recently became aware of two new entries in the ever-popular genre of, Our Peer-Review System is in Trouble; How Can We Fix It? Political scientist Brendan Nyhan, commenting on experimental and empirical sciences more generally, focuses on the selection problem that positive rather then negative findings tend to get published, leading via the statistical […]

Support of the Null Hypothesis

Timothy Teräväinen pointed to an interesting journal, the Journal of Articles in Support of the Null Hypothesis: In the past other journals and reviewers have exhibited a bias against articles that did not reject the null hypothesis. We seek to change that by offering an outlet for experiments that do not reach the traditional significance […]

Pizzagate and Kahneman, two great flavors etc.

[cat picture] 1. The pizzagate story (of Brian Wansink, the Cornell University business school professor and self-described “world-renowned eating behavior expert for over 25 years”) keeps developing. Last week someone forwarded me an email from the deputy dean of the Cornell business school regarding concerns about some of Wansink’s work. This person asked me to […]

Good advice can do you bad

Here are some examples of good, solid, reasonable statistical advice which can lead people astray. Example 1 Good advice: Statistical significance is not the same as practical significance. How it can mislead: People get the impression that a statistically significant result is more impressive if it’s larger in magnitude. Why it’s misleading: See this classic […]

Creationist article Article with creationist language published in Plos-One

Dan Gianola pointed me to this one. It’s an article by Ming-Jin Liu, Cai-Hua Xiong, Le Xiong, and Xiao-Lin Huang with the innocuous title, “Biomechanical Characteristics of Hand Coordination in Grasping Activities of Daily Living,” and a boring abstract: Hand coordination can allow humans to have dexterous control with many degrees of freedom to perform […]

At this point, even Tom Cruise is skeptical about claims of social priming. (Click to find out why)

The blogger known as Neuroskeptic writes: Can the thought of money make people more conservative? The idea that mere reminders of money can influence people’s attitudes and behaviors is a major claim within the field of social priming – the study of how our behavior is unconsciously influenced by seemingly innocuous stimuli. However, social priming […]

Constructing an informative prior using meta-analysis

Chris Guure writes: I am trying to construct an informative prior by synthesizing or collecting some information from literature (meta-analysis) and then to apply that to a real data set (it is longitudinal data) for over 20 years follow-up. In constructing the prior using the meta-analysis data, the issue of publication bias came up. I […]

Difficulties in making inferences about scientific truth from distributions of published p-values

Jeff Leek just posted the discussions of his paper (with Leah Jager), “An estimate of the science-wise false discovery rate and application to the top medical literature,” along with some further comments of his own. Here are my original thoughts on an earlier version of their article. Keith O’Rourke and I expanded these thoughts into […]

How to fix the tabloids? Toward replicable social science research

This seems to be the topic of the week. Yesterday I posted on the sister blog some further thoughts on those “Psychological Science” papers on menstrual cycles, biceps size, and political attitudes, tied to a horrible press release from the journal Psychological Science hyping the biceps and politics study. Then I was pointed to these […]

Another one of those “Psychological Science” papers (this time on biceps size and political attitudes among college students)

Paul Alper writes: Unless I missed it, you haven’t commented on the recent article of Michael Bang Peterson [with Daniel Sznycer, Aaron Sell, Leda Cosmides, and John Tooby]. It seems to have been reviewed extensively in the lay press. A typical example is here. This review begins with “If you are physically strong, social science […]

Against optimism about social science

Social science research has been getting pretty bad press recently, what with the Excel buccaneers who didn’t know how to handle data with different numbers of observations per country, and the psychologist who published dozens of papers based on fabricated data, and the Evilicious guy who wouldn’t let people review his data tapes, etc etc. […]

More proposals to reform the peer-review system

Chris Said points us to two proposals to fix the system for reviewing scientific papers. Both the proposals are focused on biological research. Said writes:

Stabilizing feedback as a bad thing in scientific inference

Someone writes: