In an abstract entitled, “Statistical dust and sweeping claims about maternal warmth,” John Richters and Everett Waters write:
Alley and colleagues draw on mediation analyses of longitudinal data from Millennium Cohort Study to argue that their findings “highlight the critically important role that childhood maternal warmth plays in shaping mental and physical health into late adolescence” (p. 716), and “suggest public health interventions aimed at increasing maternal warmth “may be particularly effective in positively impacting adolescent health” (p. 714).
Although the article is dense with tabularized information about key study variables, readers will search in vain for evidence to justify the authors’ conclusions and recommendations related to maternal warmth. What they will find instead are minuscule direct and mediated path coefficients (betas) linking maternal warmth to adolescent outcomes that amount to uninterpretable and unactionable statistical dust. The authors tell us as much in their seductively (if unintentionally) misleading statement that “Social safety at 14 years of age mediated 20% to 100% of the effect of early maternal warmth on physical health, psychological distress, and psychiatric problems at 17 years of age (b = 0.01-0.15; P < .001 for all)” (p. 709). A more straightforward, precise, and informative description of this finding is that social safety at age 14 mediated 20% of maternal warmth’s .01 effect on physical health, 60% of its .01 effect on psychiatric problems, and 100% of its .15 effect on psychological distress at age 17, for a total indirect effect of maternal warmth on subsequent outcome measures of less than 1%. The interpretability of these findings is further compromised by the extreme distributional skew of the 3-item social safety schema latent variable, with the vast majority of adolescents reporting that they had family and friends who helped them feel safe, secure, and happy (86%), someone they could turn to with problems (79%), and someone to whom they felt close (89%). The maternal warmth and harsh parenting measures are also marked by extreme distributional skew, with trained observers reporting that 86% of the mothers exhibited all 5 maternal warmth behaviors and 91% exhibited no harsh parenting behavior during naturalistic play with their children.
The authors do themselves and the readership of JAMA Psychiatry a great disservice by slipping through the normative scientific membrane and conflating statistical with theoretical and practical significance. This is an especially troubling breach within the context of contemporary concerns and public skepticism about the reliability and credibility of social and behavioral sciences research.
References
1. Alley J, Tsomokos DI, Mengelkoch S, Slavich GM. Childhood maternal warmth, social safety schemas, and adolescent mental and physical health. JAMA Psychiatry.
2025;82(7):709-717. doi:10.1001/jamapsychiatry.2025.08152. Bogdan PC. One decade into the replication crisis, how have psychological results changed? Adv Methods Pract Psychol Sci. 2025;8(2):25152459251323480.
doi:10.1177/251524592513234803. Murray EJ, Swanson SA. Causal inference in observational psychiatry: What do we need to know? JAMA Psychiatry. 2023;80(6):539-540.
doi:10.1001/jamapsychiatry.2023.03434. Richters JE. Incredible utility: The lost causes and causal debris of psychological science. Basic Appl Soc Psychol. 2021;43(6):366-405.
doi:10.1080/01973533.2021.1994229
Ahhhh, JAMA!
P.S. JAMA’s not all bad. My colleagues and I recently published a short paper there! Just about all journals are a mix of good and bad.
Figures don’t lie, but liars can figure.
Just incredible that such extravagant claims aren’t stopped in peer review. The “finding” is of course one that many in the field would like to be true, and that will help it slip past.
Claiming clinical or public health relevance is often required by journals, but should probably be reserved for systematic reviews. And authors should be required to include simulation-based illustration of the magnitude of change implied by their estimates, tied explicitly to the measurement content, just like the authors here illustrate.
On the other hand, more standards often seem to invite more of the same threshold-seeking behaviour, p <. 05 is just a pretty old standard after all.
This makes the second time in a week that I have read a more thoughtful, articulate and appropriate interpretation of a study in a commentary piece than the article itself. It is sad that we have created a system that panders to overinterpretation of ‘statistical dust’ (great phrase) rather than lines of inquiry that really change outcomes.
Quote from the blog post: “In an abstract entitled, “Statistical dust and sweeping claims about maternal warmth,” John Richters and Everett Waters write:”
I can’t be the only one pondering the possibilities of some alternative titles given the original use of “dust” and “sweeping”. I wondered whether the following might be a great option:
“Cleaning up statistical dust and sweeping claims about maternal warmth”
“Cleaning up statistical dust and sweeping claims about maternal warmth”
Thanks for the idea, but here’s why I don’t think it works:
OUR TITLE relies on a juxtaposition of scale: “dust” (microscopic, trivial, negligible effect sizes) versus “sweeping” (grand, overarching, definitive theoretical conclusions). It’s a static observation about the authors’ claims. It signals not what our Comment will do, but what the problem is, which invites readers to discover the solution. It’s scathing, passive, and clinically detached—which I think makes it more devastating than an active interventionist title.
YOUR ALTERNATIVE introduces a grammatical and semantic mismatch in the form of a verb-object clash: You can “clean up” dust—that works perfectly. But you can’t “clean up” claims. You can refute, revise, qualify, or sweep them away, but you can’t clean them up. The problem is broken parallelism: Our title balances two distinct concepts (tiny evidence vs. huge assertions). Your alternative forces a single verb (“cleaning up”) to govern both objects, which creates a syntactical stumble the moment the reader hits “sweeping claims.”
Thanks for the reply!
I like wordplay and my mind just got activitated seeing the words “dust” and “sweeping”. I don’t know anything about statistics and the actual content of your manuscript, but given your reply I think I estimated the meaning of “statistical dust” part appropriately enough.
I am not sure I agree with your use of “static observation” and “passive” and “clinically detached” descriptions of your title in the comment here now, if I am understanding correctly. I think “sweeping” might be a word that’s connected to some sort of relatively subjective assessments (or perhaps even more than that), and even “statistical dust” might also show some direction of how the authors might view (some of the) the statistical findings mentioned. Not saying anything about whether that’s good or bad in this case, just that I note that I might find these words less static or passive or clinically detached than you in this case (if I am understanding correctly).
As a side note: in general I think I am a fan of being very non-directional and pretty basic and objective in titles. I remember having to come up with my title of my research master thesis-turned-into-a-manuscript and one of my co-authors mentioned how he liked the clear and simple title (or something like that): “Associations between psychologists’ thinking styles and accuracy on a diagnostic classification task”. However, as I said, given the use of words like “dust” and “sweeping” my mind got activated, possibly also in connection to the unusually short manuscript, where I think a more “creative” title might work for some reason.
I also think the juxtaposation of “dust” and “sweeping” is partly lost in my view and experience because they also need the adjective (is that the correct term) to properly work (?). Technically it’s “statistical dust” VS. “sweeping claims” in my mind when I read it, not “dust” VS. “sweeping”. The juxtaposition you mention is technically between a noun and an adjective then if I am not mistaken (“dust” VS. “sweeping”).
That’s also why I think a more loose use of the “clean up” might be fitting here. I think I would initially agree with your assessment much more when it would be something like “statistical dust” VS. “inferential cobwebs”, where “dust” and “cobwebs” are 1) more similar to me than “dust” and “claims” and 2) the connection to cleaning is more obvious.
According to an online dictionary “clean up” can be defined as “to rid (something) of dirt, filth, or other impurities” and “to make (someone or something) orderly or presentable” and “(tr) to rid (a place) of undesirable people or conditions”. I reason among these three options, there must be connections to what to do, or what your manuscript points to, concerning both “statistical dust” and “sweeping claims”.
Again, thanks so much for the reply. I enjoyed hearing your thought process. I very much enjoy wordplay and titles of papers. They are sometimes a topic on this blog here, and I usually find them interesting to read.
Thanks. Interesting take. Given your interest in word play, you might find my 2021 Incredible utility … article of interest as a gold mine for linguistic quibbles.
http://osf.io/pfqb5
I have an interest in wordplay, not so much in reading papers!
I am also a pretty simple man in a way, your paper seems very complicated for me. If I were to try and spot the linguistice quibbles, I would have to read a lot, and think a whole lot more. I am kind of trying to avoid thinking a whole lot at the moment. I think many of the possible wordplay and quibbles might be too complicated or obscure for me. I’m sorry.
My mind works strangely I think, for instance when it comes to reading papers, I have to often jump from end to beginning, read parts very intensely, have a break for a few days, come back to the paper, read a short section then ponder it, etc. I think sometimes I have to sort of re-restructure, and translate, sections so I can even try and understand what the main point of a paper is, or what the important and less important parts are. This is especially the case with topics that are hard to understand for me, or when relatively difficult words are used, or complicated sentences.
I quickly looked at your paper and noticed twice that you listed some word in different languages (?) if I am not mistaken. That’s interesting, but that will take me a while to even try and understand why you might have used that, how it might relate to the main message of the paper (if at all), and why you used it at some section and not in another (if that’s even the case), etc. As far as just the title, I can’t quickly judge it as I reason a lot depends on the entire paper and message of/in it and I don’t have the energy to read the entire paper at the moment. I reason others might be interested in it, and will have more energy and use for reading it!
I only do “dust” and “sweeping” in titles of short papers today!
Again, my brain just got activated because of the combination of these words.
I (still) think and reason your use of “dust” and “sweeping” might already indicate, or hint at, some judgement or assessment.
I (still) think and reason the juxtaposition of “dust” and “sweeping” should also sort of keep an eye on the total comparison of “statistical dust” and “sweeping claims”.
I (still) think and reason that the “cleaning up” multiple interpretations might be suitable to connect both “statistical dust” and “sweeping claims”.
I (still) think you can not literally “clean up statistical dust” with a broom or dustpan, so the “cleaning up” connection to “dust” already implies some specific form or version of “cleaning up”, one that may also apply to “sweeping claims”.
Anyway, should I want to write another manuscript, perhaps I now have a title for it:
“Incredibly useful, or incredibly useless: The lost art, or what’s at the heart, of titles in Psychological Science”
Dear Andrew, in a previous post (I think it was in March) you wrote that you may publish the more technical version of your JAMA article on the new FDA draft about Bayesian trials as a separate article. I would be really interested in reading it. Is there any update on that?