1. Chatbots
There’s been a lot of talk lately on the routine use of chatbots as a scientific tool (literature review, refactoring code, checking the internal logic of an argument, etc.) as well as the possibility of directing chatbots to perform investigations and write entire papers.
Two big ideas come up:
(a) Chatbots being used to create “deepfakes” of scientific research: papers that are professionally formatted, readable, and are written in the style of top research papers—simulacra of research that is convincing enough that it can get published in serious journals. Such papers could include plausible computer-generated images and data, along with chatbot-generated text describing nonexistent experiments.
(b) Chatbots being used to make real discoveries in pure math or computer science, or doing automated analysis of real data in biology to make real scientific discoveries.
Both these ideas are versions of things that chatbots, generative AI, and machine learning can do well: (a) producing text that looks like human-generated text and producing images and videos that look like real photographs and movies, and (b) finding patterns in big data.
2. Deepfake science conducted by humans
But today I want to talk about something slightly different, which is the pre-AI history of deepfake science, that is, deepfake science conducted by humans.
Let’s go through some history. I don’t claim this to be a systematic history; it’s just some examples I’ve heard about over the years:
– Old-style hoaxes: physical deepfakes such as Piltdown Man, fake data such as from the psychologist Cyril Burt, the doctored fairy photos that fooled Arthur Conan Doyle, the Bell Labs superconductor guy, etc.
– Fraud from the recent golden age of NPR/Ted-style fake science: the Harvard primatologist, the Cornell food-behavior guy, the election-denying gun guy, the experiment-faking UCLA poli sci student, the Surgisphere team, the UNR dean of engineering, etc.
– Data misrepresentation or plagiarism: the University of California sleep guy, the USC giraffe guy, the Stanford president, whoever at Columbia faked the U.S. News report, a few zillion p-hackers, and various researchers who, when confronted with problems with their data, acted as if there was nothing wrong.
– Bad science, not involving fraud but being “deepfake” in the sense of following the detailed forms of a scientific investigation without actually making scientific progress, because the data and measurements are too noisy to learn anything about the target phenomenon (what we call the kangaroo problem): the beauty-and-sex-ratio guy, the ESP guy, the air-pollution-in-China guy, the ovulation-and-clothing team, the “critical positivity ratio” team, the climate-change “gremlins” guy, the mind-body-healing team, etc.
I’d call all of these “deepfakes” in that they are simulating the process of science in a generative manner, not constructing a scientific paper by itself but performing a (human-operated) process that yields a convincing-looking result. As we’ve learned with computer-generated text and images, it can be much more effective to run such a process than to just try to jump to the end.
Depending on the researcher—or, maybe I should say, the operator–the generative process can be done in different places: text, data, numerical images, even the theory itself.
You can go pretty far with a deepfake but eventually you’re likely to be caught out, either because your fake data have internal flaws, because your results can’t be replicated (as with the superconductor), or because there’s a problem with the “meta-data” or cover story. Consider for example the surveys purportedly conducted by Mary Rosh and Michael Lacour: in these two (separate) incidents, no documentation was produced. Or Wansink’s refilling soup bowl and Ariely’s paper shredder: the claimed physical evidence for these devices is dubious.
As the saying goes, reality is overdetermined, and ultimately the only convincing way to make fake data is to collect real data, in the same sense that the only realistic map of a territory is to use a Borgesian 1-to-1-scale.
3. Algorithmic deepfake science
Stepping back, we can think of both the human-generated and chatbot-generated forms of deepfake science as being algorithmic, in the sense that they are created using some sort of general process that is being fed by domain-specific information (not so much information about the science; more information about the conventional forms of data and writing used in the particular scientific subfield being faked).
To put it another way, this work is “deepfake,” rather than just “fake,” in that it’s generating a simulation of the entire scientific process, creating a thick brick of fakeness, as it were, rather than a brittle fake crust. Bem’s ESP edifice, for example, isn’t just a convincing-looking paper; it also includes, at an internal level, experiments, data and a research team, and, externally, a scientific literature and repeated exposure in the news and social media.
Last year I posted 7 steps to junk science that can achieve worldly success:
1. Bad statistical analysis
2. Researcher degrees of freedom
3. Weak or open-ended substantive theory
4. Inaccurate summaries of the literature
5. Institutional support
6. External promotion
7. Celebrity culture.
You don’t need all these for a deepfake. In particular, you don’t need bad statistical analysis if you take the trouble to create fake data that shows what you want to see, and you don’t need celebrity culture if you can get your message out in some other way. The real point here is that the deepfake occurs within a social environment.
In an era where we’re worried about algorithmic deepfake science from chatbots, it could be helpful to think about what we’ve learned from algorithmic deepfake science of the human variety.