Dale Lehman sends an email with subject line “A new low in science”:
I watched part of the painful hearing and here is a story on it.
It is a new low in both statistical studies, policymakers reactions, and public reactions. Of course, the study could be correct, but without any review and without any data, it is simply out of line with many other studies. And, in the hearing the explanations amount to a vast conspiracy where thousands of scientists and clinicians are in a grand scheme (mostly out of fear for their jobs – an irony, given how this administration has used fear about employment as a tool against government employees) to prevent the truth from getting out. The hearing itself was a travesty with virtually none of the subcommittee members present – only Ron Johnson (who outdid himself) and Blumenthal who left (I think out of frustration). I don’t know what bothered me the most – the tone of the witnesses, Johnson’s tirade, or the public applauding the conspiracy statements.
I replied that the bit about “the study could be correct” reminds me of the distinction between evidence and truth which we’ve discussed on the blog.
I’m not sure that this is “a new low in science.” Some other recent lows in science have included members of the elite media promoting ridiculous UFO-as-space-aliens speculation, and celebrity academics here and here promoting junk-science claims of mind-body healing.
The vaccine crap is much worse from a policy perspective—space aliens and mind-body healing are mostly just a waste of time—but, as we’ve discussed, I have a horrible feeling sometimes that heavily promoted crap on space aliens, cold showers, mind-body healing, schoolyard evolutionary psychology, extra-sensory perception, magic golf balls, air rage, himmicanes, subliminal smiley faces, etc etc etc, has softened the ground so that the seeds of more evil trees can now planted and take root.
All that junk science over the past twenty years has been promoted by leading academics. Prominent professors from Harvard, Stanford, Columbia, Chicago, etc. have been promoting magical thinking under the guise of science. Perhaps no surprise that politicians and non-academic hucksters want to get into the game too.
P.S. I asked Lehman whether it was ok for me to quote him, and he said, “Sure. But if it is delayed by 6 months, there will probably be worse examples to come.” I replied that I’ve been doing less blogging lately and the lag is only 4 months now.
P.P.S. Between when I wrote the above post and when it appeared, I posted something else with a similar theme: 25,000 lives saved per ship sunk, $100,000 per citation, a probability of 10^-90 of a decisive vote . . . Is there a through line from B.S. numbers in junk science to B.S. numbers coming from the government?
Part of this may just be me trying to justify my own existence: I write about bad science and I’m concerned about bad policy, so it’s natural for me to see a connection. But I do see something real here. Many of the celebrated leaders of our pop-science establishment–professors with elevated titles at major universities, members of the National Academy of Sciences, PBS and Ted stalwarts, etc.–have at best a tolerance and at worst an active taste for junk science. And many of our political leaders will say the most ridiculous, immediately refutable things. I can’t do much about this, but I can scream. Oh yes, I can do that.
Space Aliens and ESP have been way way more prominent many decades ago. ESP is but a shadow of its former self, and Space Aliens are a cliche.
Anti-vaccine lunacy is an entirely different process. In my view, intellectuals are very loath to admit that widespread political lying *works*. Granting that opens up all sorts of problems, to which I have no solution.
But that’s different from the kind of grouchy complaint “The kids have stopped believing in God because of the liberals, and once there’s no eternal truth, if morality is all relative, that leads to *COMMUNISM*!”
Seth:
Unfortunately, space aliens are a thing now in the elite media of the left, right and center.
As are mind-body healing and related gimmicks such as cold showers and dietary supplements, which, yes, shade right into vaccine denial. On the left these are presented as woo-woo non-corporate solutions, on the center they’re presented as scientifically demonstrated technology, and on the right they represent an end-run around professional expertise. Something for everyone!
So I do see connections, and, yes, I agree that I’m a grouchy old man, but I think I’m making a good point in this rant.
Seth Finkelstein wrote, “Anti-vaccine lunacy is an entirely different process.” It certainly is, and I wonder from when and whence it first infected the planet. Andrew Wakefield is typically cited as the Godfather of vaccine medical misinformation
https://sciencebasedmedicine.org/andrew-wakefield-after-25-years-paving-the-way-for-covid-19-quacks-and-antivaxxers/
However, Wakefield did not, so to speak, “move the needle much” until Covid came along. As always, I failed to predict the allure of this point of view because I and my cohort came into this world before the vaccination regimen we now have. My parents would regale me with stories of how many of their siblings, cousins, etc. who died of the various childhood infections, implying I should be grateful that I am still alive despite having had each and every one of the childhood infections. And, of course, many decades later, I had shingles as well, albeit so mild that there was some debate so as to verify that I was infected.
paul alper: Anti-vaccine lunacy has been around at the fringes for a long time. Mostly from religious fundamentalists who believe in only faith-healing, though also I should note hippie-types who only want “nature” and herbalism (I think the latter is much less relevant, but some people will rush to say both-sides-do-it if I don’t mention them). However, the right-wing lie-machine decided to push it extensively as part of the politicization of Covid and demonizing public health. I’m sure this has been written about at length. This is one reason I believe Andrew is making an error in relating anti-vax to the list of issues above, since what happened was a deliberate, intentional, malicious, political propaganda campaign. This campaign cannot be countered by every single intellectual in the world being absolutely perfect, even if that were possible in practice. It worked because lying works (disclaimer: many times, not all the time for everything). The impulse is to say that if all intellectuals were perfect, then lying wouldn’t work, so the moral solution is then to criticize any bad intellectuals for their failings. Except this reasoning seems false as far as can be tested in the real world, i.e. lie campaigns do not fail for being lies.
We need a generational replication project in biomedicine. Quality control has been progressively more diluted since the introduction of NHST methods ~1950. Ie, standards have been getting worse and worse as those trained before -1950 retired/passed, then the next generation they personally trained retired, etc. There is currently nothing stopping fields from continuing down the wrong path for decades, stringing together meaningless “significant” differences, becoming ever more detached from reality.
Basically, someone with pre-1950 scientific standards (requires direct replication and distinctive theoretical predictions tested on new data), will find only about ~ 1/10,000 biomedical claims have been validated.
Give me a couple billion dollars and I’d start a project disallowing citing anything after a ~1950 date (although it can be used as inspiration, like dreams, etc are treated today). Then let people submit proposals from there, with anything funded also getting a replication, building up a parallel literature. Searching for commonalities (“laws”) would be the focus rather than differences between groups (which are uninteresting because they are expected by default).
Its interesting that we can also now train LLMs up to cutoff dates in the same way: https://github.com/DGoettlich/history-llms
“…Basically, someone with pre-1950 scientific standards (requires direct replication and distinctive theoretical predictions tested on new data), will find only about ~ 1/10,000 biomedical claims have been validated. …”
This. ESP and pop Psychology get all the ‘fame’, but it is much more serious when it comes to Biomed sciences. We all know that research tools in Biomed are limited given the vast complexity of the topics and much is unknown. Why don’t they say so, instead of relying on false over-certainty, pretending we really know how things work. It is quite obvious that they don’t employ whole teams of statisticians, which given the budgets should be mandatory. To be fair, big pharma and other big private Biomed companies do have teams of data analysts. Research that is produced at various universities, on the other hand, relies on a couple of basic stats classes that their Biology grad student and/or faculty may or may not have taken. But hey, it’s ‘statistically significant’, ‘safe and effective’ with balanced ‘risk vs. benefits’ assessment and such (insert your favorite misunderstood phrase). Gigerenzer would never run out of material in Biomed.
Here’s a radical idea. Make open access to the data required. No, it won’t solve everything, but it can make a big dent in the protection against flawed analysis that proprietary data provides. If biomed research falls short, then the army of “methodological terrorists” might provide a mitigating measure.
The problem is more the type of data being collected to begin with. We don’t need any more differences between groups/conditions. We want to know what conditions consistently yield the same results, and theories that allow us to extrapolate.
Anon
Can you clarify your comment? You appear to say the wrong type of data is being collected, but it sounds more like you are objecting to the type of analysis that is done. If too much analysis is being done looking at differences between groups (although I don’t see why that is objectionable – you don’t have to do NHST to examine differences between groups and I don’t see why these are not of interest), I think the same data might be used to “to know what conditions consistently yield the same results.” Exactly what data is being collected that you think is a mistake and exactly what should be collected instead? Since I am having trouble understanding your comment, a concrete example would help me.
Think of the typical experiment that compares group A vs group B and looks for some difference. The data collected is essentially infertile for developing a model of what is going on, which must be dynamic, as all lifeforms are. We can kind of work around this, but why?
The choice of data collection itself is the obstacle to determining what is going on. The decision to collect that type of data was made with the goal of applying NHST.
In light of the title and content of this blog post, I would like to take this opportunity
To simply provide an extra, alternative theory
In light of this all, I wonder whether it might be useful and interesting, not just for me
To point to the influence of a future event on a current response and time-reverse causality
Such things are mentioned in Bem’s 2011 paper concerning extrasensory perception, or ESP
And although the findings presented there contrast with normality
The findings may have been the result of commonly used methodology
So, in light of this paper by Bem, it might be useful to ponder the following possibility
Maybe Bem’s paper is a result, and not a cause, of a possible rooting of seeds of a more evil tree
You see, perhaps a myth-buster has to sometimes be a ghost-buster to provide a key
So others are able to clearly sense, perceive, and see