Here’s what’s scheduled for the next six months:
This is a great example for a statistics class, or a class on survey sampling, or a political science class
How to read (in quantitative social science). And by implication, how to write.
Causal inference with time-varying exposures
Reproducibility problems in the natural sciences
If you want a vision of the future, imagine a computer, calculating the number of angels who can dance on the head of a pin—forever.
Collinearity in Bayesian models
Inshallah
“Did Austerity Cause Brexit?”
“Widely cited study of fake news retracted by researchers”
Causal inference using repeated cross sections
Calibrating patterns in structured data: No easy answers here.
Healthier kids: Using Stan to get more information out of pediatric respiratory data
Gigerenzer: “The Bias Bias in Behavioral Economics,” including discussion of political implications
Endless citations to already-retracted articles
Update on keeping Mechanical Turk responses trustworthy
Blindfold play and sleepless nights
What does it take to repeat them?
“The most mysterious star in the galaxy”
Gendered languages and women’s workforce participation rates
What’s published in the journal isn’t what the researchers actually did.
Alison Mattek on physics and psychology, philosophy, models, explanations, and formalization
Votes vs. $
“Developing Digital Privacy: Children’s Moral Judgments Concerning Mobile GPS Devices”
Plaig!
From deviance, DIC, AIC, etc., to leave-one-out cross-validation
Of book reviews and selection bias
Swimming upstream? Monitoring escaped statistical inferences in wild populations.
Concerned about demand effects in psychology experiments? Incorporate them into the design.
Just forget the Type 1 error thing.
“This is a case where frequentist methods are simple and mostly work well, and the Bayesian analogs look unpleasant, requiring inference on lots of nuisance parameters that frequentists can bypass.”
The garden of forking paths
This one goes in the Zombies category, for sure.
Allowing intercepts and slopes to vary in a logistic regression: how does this change the ROC curve?
A weird new form of email scam
Holes in Bayesian Philosophy: My talk for the philosophy of statistics conference this Wed.
Hey, look! The R graph gallery is back.
The intellectual explosion that didn’t happen
Deterministic thinking meets the fallacy of the one-sided bet
Are GWAS studies of IQ/educational attainment problematic?
Attorney General of the United States less racist than Nobel prize winning biologist
Here are some examples of real-world statistical analyses that don’t use p-values and significance testing.
They added a hierarchical structure to their model and their parameter estimate changed a lot: How to think about this?
“Beyond ‘Treatment Versus Control’: How Bayesian Analysis Makes Factorial Experiments Feasible in Education Research”
As always, I think the best solution is not for researchers to just report on some preregistered claim, but rather for them to display the entire multiverse of possible relevant results.
Replication police methodological terrorism stasi nudge shoot the messenger wtf
Separated at birth?
What can be learned from this study?
“I feel like the really solid information therein comes from non or negative correlations”
“The issue of how the report the statistics is one that we thought about deeply, and I am quite sure we reported them correctly.”
Is there any scientific evidence that humans don’t like uncertainty?
You should (usually) log transform your positive data
Coney Island
Yes, you can include prior information on quantities of interest, not just on parameters in your model
More on why Cass Sunstein should be thanking, not smearing, people who ask for replications
The importance of talking about the importance of measurement: It depends on the subfield
More on the piranha problem, the butterfly effect, unintended consequences, and the push-a-button, take-a-pill model of science
“No, cardiac arrests are not more common on Monday mornings, study finds”
Beyond Power Calculations: Some questions, some answers
When people make up victim stories
“I am a writer for our school newspaper, the BHS Blueprint, and I am writing an article about our school’s new growth mindset initiative.”
Is the effect they found too large to believe? (the effect of breakfast micronutrients on social decisions)
“It just happens to be in the nature of knowledge that it cannot be conserved if it does not grow.”
He says it again, but more vividly.
The Wife
More golf putting, leading to a discussion of how prior information can be important for an out-of-sample prediction or causal inference problem, even if it’s not needed to fit existing data
A world of Wansinks in medical research: “So I guess what I’m trying to get at is I wonder how common it is for clinicians to rely on med students to do their data analysis for them, and how often this work then gets published”
It’s not just p=0.048 vs. p=0.052
Why didn’t they say they were sorry when it turned out they’d messed up?
Here’s why you need to bring a rubber band to every class you teach, every time.
Here’s a puzzle: Why did the U.S. doctor tell me to drink more wine and the French doctor tell me to drink less?
Was Thomas Kuhn evil? I don’t really care.
Exchange with Deborah Mayo on abandoning statistical significance
My math is rusty
Deterministic thinking (“dichotomania”): a problem in how we think, not just in how we act
I think that science is mostly “Brezhnevs.” It’s rare to see a “Gorbachev” who will abandon a paradigm just because it doesn’t do the job. Also, moving beyond naive falsificationism
“Boston Globe Columnist Suspended During Investigation Of Marathon Bombing Stories That Don’t Add Up”
Question on multilevel modeling reminds me that we need a good modeling workflow (building up your model by including varying intercepts, slopes, etc.) and a good computing workflow
Harking, Sharking, Tharking
“Statistical Inference Enables Bad Science; Statistical Thinking Enables Good Science”
Laplace Calling
Challenge of A/B testing in the presence of network and spillover effects
The State of the Art
Bank Shot
They misreport their experiments and don’t fess up when they’ve been caught.
“Persistent metabolic youth in the aging female brain”??
Junk science and fake news: Similarities and differences
We should all routinely criticize our own work.
Controversies in the theory of measurement in mathematical psychology
Here’s a supercool controversy for ya
“Less Wow and More How in Social Psychology”
“Troubling Trends in Machine Learning Scholarship”
P-value of 10^-74 disappears
“What is the conclusion of a clinical trial where p=0.6?”
Kaiser Fung suggests “20 paper ideas pre-approved for prestigious journals”
More on that 4/20 road rage dude
Are statistical nitpickers (e.g., Kaiser Fung and me) getting the way of progress or even serving the forces of evil?
BizStat: Modeling performance indicators for deals
Glenn Shafer: “The Language of Betting as a Strategy for Statistical and Scientific Communication”
Automation and judgment, from the rational animal to the irrational machine
What’s the p-value good for: I answer some questions.
On the term “self-appointed” . . .
When presenting a new method, talk about its failure modes.
Poetry corner
How to think scientifically about scientists’ proposals for fixing science
The status-reversal heuristic
“Here’s an interesting story right in your sweet spot”
The real lesson learned from those academic hoaxes: a key part of getting a paper published in a scholarly journal is to be able to follow the conventions of the journal. And some people happen to be good at that, irrespective of the content of the papers being submitted.
His data came out in the opposite direction of his hypothesis. How to report this in the publication?
He’s looking for a Bayesian book
My best thoughts on priors
Bayesian analysis of data collected sequentially: it’s easy, just include as predictors in the model any variables that go into the stopping rule.
“Bullshitters. Who Are They and What Do We Know about Their Lives?”
“Causal Processes in Psychology Are Heterogeneous”
“Any research object with a strong and obvious series of inconsistencies may be deemed too inaccurate to trust, irrespective of their source. In other words, the description of inconsistency makes no presumption about the source of that inconsistency.”
Many Ways to Lasso
Afternoon decision fatigue
What happens to your metabolism when you eat ultra-processed foods?
Software for multilevel conjoint analysis in marketing
I’m no expert
“Everybody wants to be Jared Diamond”
The dropout rate in his survey is over 60%. What should he do? I suggest MRP.
How to teach sensible elementary statistics to lower-division undergraduates?
“The paper has been blind peer-reviewed and published in a highly reputable journal, which is the gold standard in scientific corroboration. Thus, all protocol was followed to the letter and the work is officially supported.”
The incentives are all wrong (causal inference edition)
“Men Appear Twice as Often as Women in News Photos on Facebook”
“Non-disclosure is not just an unfortunate, but unfixable, accident. A methodology can be disclosed at any time.”
Battle for the headline: Hype and the effect of statistical significance on the ability of journalists to engage in critical thinking
Australian polls failed. They didn’t do Mister P.
When Prediction Markets Fail
Hey! Participants in survey experiments aren’t paying attention.
To do: Construct a build-your-own-relevant-statistics-class kit.
Consider replication as an honor, not an attack.
Is “abandon statistical significance” like organically fed, free-range chicken?
Should we mind if authorship is falsified?
Why do a within-person rather than a between-person experiment?
In research as in negotiation: Be willing to walk away, don’t paint yourself into a corner, leave no hostages to fortune
Stan saves Australians $20 billion
What’s the evidence on the effectiveness of psychotherapy?
What does a “statistically significant difference in mortality rates” mean when you’re trying to decide where to send your kid for heart surgery?
What happens when frauds are outed because of whistleblowing?
How much granularity do you need in your Mister P?
This awesome Pubpeer thread is about 80 times better than the original paper
I’m still struggling to understand hypothesis testing . . . leading to a more general discussion of the role of assumptions in statistics
Structural equation modeling and Stan
Why “bigger sample size” is not usually where it’s at.
Public health researchers: “Death by despair” is a thing, but not the biggest thing
No, Bayes does not like Mayor Pete. (Pitfalls of using implied betting market odds to estimate electability.)
In short, adding more animals to your experiment is fine. The problem is in using statistical significance to make decisions about what to conclude from your data.
What comes after Vixra?
Don’t believe people who say they can look at your face and tell that you’re lying.
What’s wrong with Bayes; What’s wrong with null hypothesis significance testing
What’s wrong with Bayes
What’s wrong with null hypothesis significance testing
“Pfizer had clues its blockbuster drug could prevent Alzheimer’s. Why didn’t it tell the world?”
Are you tone deaf? Find out here.
“There is this magic that our DNA enables”
How to think about “medical reversals”?
The checklist manifesto and beyond
“Deep Origins” and spatial correlations
“Inferential statistics as descriptive statistics”
Judith Rich Harris on the garden of forking paths
What happened to the hiccups?
What does it mean when they say there’s a 30% chance of rain?
Causal inference and within/between person comparisons
‘Sumps and rigor
Elon Musk and George Lucas
Horns! Have we reached a new era in skeptical science journalism? I hope so.
Causal inference, adjusting for 300 pre-treatment predictors
External vs. internal validity of causal inference from natural experiments: The example of charter school lottery studies
“What if your side wins?”
“But when we apply statistical models, do we need to care about whether a model can retrieve the relationship between variables?”
How did our advice about research ethics work out, four years later?
Fitting big multilevel regressions in Stan?
The last-mile problem in machine learning
The Role of Statistics in a Deep-Learning World
Create your own community (if you need to)
I wrote most of these awhile ago and I don’t remember what many of them are about. So I’m in as much suspense as you are.
Forgive me if this has been covered extensively already, but why are the posts on a (6 month?) delay again?
I did a quick google, and found some comments on other posts where benefits of the delays are mentioned, but I didn’t find anything conclusive about why.
Thank you.
Patrick:
180 posts are too much for anyone to read in one day. I schedule them to appear daily so readers and commenters have a chance to digest each post.
Hi Andrew, thanks for the response.
Presumably they weren’t all written in a single day though; why not release them most contemporaneous with when they were written? Still spaced out, but without the lag.
Not criticizing, just curiosity on my part.
If Andrew didn’t maintain a backlog he’d be under daily pressure to write.
He does sometimes move a post up or interject a freshly written one if there’s a particularly postworthy incident.
I’ve been waiting for the comparing theoretical predictions to galactic rotation rate data across multiple papers post to show up. Is it still coming?
Sounds great!
I assume this is a top to bottom queue, is that correct? Looking forward to “Swimming upstream? Monitoring escaped statistical inferences in wild populations.” Aren’t too many things that this would likely refer to.
Roy:
They’re in chronological order.