A few months ago I read this book, Plastic Fantastic: How the Biggest Fraud in Physics Shook the Scientific World, by Eugenie Reich. The book was from 2009, and I’d never heard of it, or of the case of fraud it discusses; indeed I don’t even remember who recommended the book to me, maybe it was a blog commenter?
Anyway, the book was interesting. Short story is that there was a mediocre young physics Ph.D. who achieved some success by faking data, he got a research job at Bell Labs, faked more data, got more success, eventually got too much success in that people were interested enough in his findings that they tried to use his methods themselves, but the methods never worked. After a bit more struggle he then got fired.
Compared to the usual glacial pace of such scandals in academia, the whole thing was pretty quick and clean. From receipt of Ph.D. to getting fired was just a bit over five years. That might seem like a long time, but it’s quick compared to the careers of Wansink, Ariely, Wegman, Hauser, etc.–not to mention the many bullshitters in social and medical science who are still out there.
I think the key difference between this physics case and fraudulent science in cognitive and social science is that when a key part of any physics paper is its methods. If a finding is interesting, and other researchers want to get into the game, they’ll want to start by replicating the experiment. If they can’t do that, they’ll get upset. In contrast, if you want to follow up on a political science or economics study, you’ll look for new dat, and if you want to follow up on a psychology experiment on embodied cognition or mind-body healing or whatever, you can just do your own experiment from scratch. There’s no particular method that you have to use.
So in social science junk papers like those discussed here or here can stay around forever–even questionable papers written by Nobel prize winners–because people only care about the results, not the method.
Here’s another interesting difference. Those three papers discussed in the link in the previous paragraph are not fraudulent, they’re just bad science, some combination of noisy data, bad theory, bad measurement, and statistical misunderstanding. Here’s the relevant principle: in the social and behavioral sciences, you can get prominent bad results by accident. In physics, you pretty much have to cheat. Sure, experimental physicists get fluke results all the time, but then they don’t replicate and the field moves on. To really stick the landing on a non-result in physics you have to cheat.
Social scientists who do junk research or even fraud can stay afloat forever, but that cheating physicist was headed for a fall, once his research got some attention.
Also, it seems that this guy was a real asshole. Not only did he lie, and then lie again to cover up, then later when it all came out and his university revoked his Ph.D., he sued them. What kind of person would do that: defraud the education system and then sue them?
At this point I should step back and think like a statistician. Some percentage of people are unscrupulous assholes, and some percentage of people are physicists. Draw the Venn diagram: you’ll expect to see some overlap.
In any case, this is all so clean compared to what happens in social science.
Quote from the blog post: “Not only did he lie, and then lie again to cover up, then later when it all came out and his university revoked his Ph.D., he sued them. What kind of person would do that: defraud the education system and then sue them?”
Perhaps he was what Francesca Gino might call a “rebel talent”, albeit a possibly overenthusiastic one. Here is an excerpt from the rebel talents website concerning the book by Gino:
“We think of them as troublemakers, outcasts, contrarians: those colleagues, friends, and family members who complicate seemingly straightforward decisions, create chaos, and disagree when everyone else is in agreement. But in truth, rebels are also those among us who change the world for the better with their unconventional outlooks. Instead of clinging to what is safe and familiar, and falling back on routines and tradition, rebels defy the status quo. They are masters of innovation and reinvention, and they have a lot to teach us.”
Side note: I am happy to see the following sentence in the blog post: “At this point I should step back and think like a statistician. Some percentage of people are unscrupulous assholes, and some percentage of people are physicists. Draw the Venn diagram: you’ll expect to see some overlap.”.
It might further be the case that not all scientists behave similarly when confronted with possible “incentives” or “the system” or “publication pressure” or whatever variable or term one might want to use when talking about research misconduct. I’ve wrote a manuscript which includes several references that together might make this reasoning or suggestion or hypothesis appropriate, or at least something to consider. It might be useful to pay way more attention to this all, which is part of why I wrote the manuscript (titled “Why psychopathy might be present and prosperous in present-day psychological science” which can be found on SSRN).
Aaa:
My problem with Schoen, and Gino, is not that they were critical of the education system that had nurtured them. I’m critical of the education system too! Indeed, I keep criticizing my own employer for their terrible responses to the medical school sexual assaulter, the medical school professors who have conducted or promoted fraudulent research, and whoever faked their U.S. News numbers and covered it up.
I think it’s fine to be a rebel.
The thing that annoys me about Schoen and Gino is not that they criticized their universities, but that they did so after defrauding them.
It took 15 years to find out that Alzheimer’s was not caused by amyloids. It took 1 month to figure out that LK-99 was not a high temperature superconductor.
https://arstechnica.com/science/2026/04/whats-the-deal-with-alzheimers-disease-and-amyloid/
https://www.cnet.com/tech/computing/lk-99-superconductor-from-breakthrough-hope-to-more-humble-reality/
I submit that this is because physics theory makes unambiguous predictions that are curves with very few parameters and these curves must agree with the experimentally measured ones. That is on top of, and secondary to, experiments being replicable (same measured curve with the same parameters) with the same methods, and independently confirmable with different methods.
This has been the approach for 4000 years, since the ancient Babylonians invented astronomy.
I dont think those retractions have much to do with the validity of the the amyloid hypothesis, it was already on thin ice ~2005. By then we knew amyloids were the lowest energy state for proteins, and found in every kind of diseased tissue. Like yea if youre too sick to clean and trash starts accumulating thats not good, but cleaning up the trash isn’t going to fix the root problem.
It was never clear to me why such a dogma grew up around this when the previous “cholinergic” hypothesis seemed to work better.
Typo:
“you’ll look for new dat, and if you”
True dat
It did work out ok but it’s a bit embarrassing how many people were not skeptical. Some of the results casually violated pretty well-established theoretical limits on the breakdown fields of some dielectrics. Some colleagues were still urging others to try to reach Schoen’s level of productivity.
https://doi.org/10.1177/10596011261431423
Jukka:
Thanks for the link. I agree with the authors on almost everything. The only bit that I would add is that many of these problems are politicized, with both “left” and “right” elements within and outside academia pushing consumerization, administrative bloat, overzealous ethics committees, and the requirement for absolute conformity. Try to do anything about any of these and you get ideological pushback from the left and the right.
It’s a little different to much of science fraud in that it was perpetrated mostly in an industrial setting (Bell Labs). What gets me about this (and the Jonathan Pruitt spider biology episode) is the apparent disinterest in collaborators in looking at the experiments. It boggles the mind that Schon would produce earth-shattering “data” over and over and none of his collaborators (and bosses since it was the Bell Labs) thought to say – “Hey Jan show me how you set up and did that experiment; maybe we can do one together”.
Likewise with Jonathan Pruitt’s (JP) collaborators. They designed an experiment, JP “did” the experiments and sent the “data” to a collaborator who analyzed it and they wrote the paper (or several papers). Wasn’t anyone interested in watching some measurements on the puff-response of spiders?
What’s interesting about both those cases is that the fraud could only be perpetrated in a setting in which collaborators showed no interest in looking at the experiments.
From R. G. Steen (2010) “Retractions in the scientific literature: do authors deliberately commit research fraud?”
“It has been postulated that retracted papers involve relatively few co-authors5 since having many co-authors would seem to provide a backstop against both error and fraud. In contrast, we hypothesise that fraud is easier to perpetrate when personal responsibility is diffused. This study reports evidence that having many authors does not protect from fraud, since the number of authors per retracted paper ranged up to 26 (figure 1). The mean6SD number of authors per retracted paper was 5.163.3; the median number of authors per retracted paper was 4. Roughly 7% of retracted papers were written by a single author, but 18% of retracted papers had $8 authors and six retracted papers had >20 authors.” (p. 114)
Interesting perhaps in view of certain recent proposals for large-scale “collaborative” work, where there has been explicit mentioning of clear task devision by some if I am not mistaken. I put collaborative in quotation marks because I reason it might be the case that this “collaborative” work involves a small group of people benefiting disproportionally from all this collaboration. For instance, by managing the efforts and maybe gettting a nice share of all the grant money to possibly pay for free lunches at their newly founded “collaborative center”. I don’t know how this all works, but perhaps it might go like that. Anyway, that’s just one reason why I put “collaborative” in quotation marks.
I feel that misses the point of collaborations. In most cases, if you’re collaborating with someone, it’s because they have skills/expertise that you don’t. If you had the ability to do a specific analysis or experiment, you would probably just do it yourself. If you have a collaborator, you have to trust them to do something that is beyond your ability. It would be very unusual for a bioinformatician to ask to see the lab procedure because it’s completely out of their field of expertise. (There are obviously people have have experience in multiple areas of a study.)
Yes that’s a good point. I was referring to the Schon (superconductivity) and Pruitt (spider behaviour) instances. Schon’s main collaborator was (at least initially) Schon’s boss I think and worked in the same institute and had similar research interests. So the collaboration should have been one where the collaborator could participate in the experimental work or at least observe what was being done. Likewise Pruitt and his main collaborator at the time had similar backgrounds – both behavioural ecologists.
There are lots of reasons why collaborations develop in the way that they do and no doubt many reasons why they occasionally go wrong – in this case I understand that Pruitt had a lab with spiders and the main collaborator at the time was a graduate student elsewhere – so only Pruitt had the facilities to do the experiments. Still, it’s surprising that the graduate student didn’t visit Pruitt’s lab or at least ask to see some videos of the spider experiments which were extremely basic to perform. I don’t ascribe any blame other than to Pruitt, but it does seem odd – perhaps Pruitt was just very successful in finding reasons to keep things hidden.
I do agree though that in general collaborations develop as an application of complementary skills to a particular scientific problem – so one may not feel a need to observe experiments of one’s collaborators (who one has to trust!).
I assume he had the kind of personality that people in the field stereotypically associate with “brilliance” in that field, which in some fields includes being the kind of arrogant and obnoxious person who at an extreme would sue his university for saying true things about him that rightfully hurt his reputation.
These are in one way rebelling against what everyone else thinks, but in another way are doing exactly what everyone else expects. Possibly because the field is inherently interested in overturning established wisdom. People might enjoy having them around because they’re doing what others wish they could do.
In other fields that don’t have the same norms, breaking rules might code differently.
Quote from above: “I assume he had the kind of personality that people in the field stereotypically associate with “brilliance” in that field, which in some fields includes being the kind of arrogant and obnoxious person (…)”
This reminds me of something I wrote in my manuscript mentioned elsewhere in the comment section about the prototypical psychopathic psychological scientist:
“Their meanness, and boldness (cf. Patrick et al., 2009, p. 926-927), and superficial charm make them more likely to engage and/or succeed in “game-playing” (cf. Anderson et al, 2007), and manipulation, and exploitation of other people. Their venturesomeness, risk-taking, assertiveness, persuasiveness, and superficial charm can be mistaken for competence, or seen as desirable qualities (cf. James, 1995; Patrick et al., 2009; Wallace et al., 2022), which might (in-)directly help and facilitate psychopathic psychological scientists regarding their manipulation and exploitation.”
My memory, somewhat fuzzy, is that your assumption is not correct. I think his reputation was as a good listener. (Again, maybe I’m wrong.) The one scientific fraudster of whom I had a direct impression was definitely a good listener type, not at all abrasive.
Michael:
Yes, in the book he was described as being a quiet person who became uncomfortable with all the attention he was getting. Although I guess that part of him must have enjoyed it.
Hm, I was going to say that I’ve known two types along those lines in my career, and one of them is a very quiet person who ends up with a group of people admiring him, as opposed to the more obvious type which is the really aggressive person who doesn’t listen to anyone else. But it sounds like he wasn’t either of these, and maybe people missed that his work didn’t check out for a different reason.
Hi Michael,
I agree with your memory. He came across as quite modest, just shrugging and pretending to follow the data where it led. As we know from some of the statistical anomalies that you helped analyze, that had to be a flat-out lie.
Although we can call him mediocre now, he got some big prizes and was being eyed for other major prizes and jobs, so many people were successfully fooled by him.
Andrew, I’m surprised this is new to you; it was a very big deal at the time, when people imagined fraud couldn’t happen in physics. But in my biased view (as a member of the investigative committee), the (relative) speed and the definitiveness of the report really helped to limit the damage to condensed-matter physics. Nonetheless, there were graduate students and postdocs who wasted critical parts of the careers trying to follow up. No one died from it, though.
I did not know Bell Labs still existed in 1997. I assumed the case must have been at least ten years earlier.
It still exists. See https://www.nokia.com/bell-labs/.
A historical perspective has helped me to think through the implications of the questions you are addressing. The short answer is related to at least two major points: the complexity of the symmetries of the underlying systems to be modeled and the historical eras of the science being prosecuted. As a physical chemist I enjoy looking at the spectacle of the evolution of my science from the alchemical days to the pchem days I enjoyed when I was a student in the 1960s. Thomas Kuhn in “The Structure of Scientific Revolutions” gives a really nice perspective on the early years. Section VI as he addresses the anomalies that arise during a period of real discovery – as in the discovery oxygen and the disproving of phlogiston theory.
Alchemy was about seeking effective solutions for the transformation of substances into gold and elixirs for eternal life. It certainly enjoyed a lot of study by people with money and time on their hands. There was a lot of interest. The thing we can see with the perspective of centuries is the gradual sifting out of baloney, fraudulent results and tidbits of real effects (not related to gold transmutation or life elixirs) from the records left behind by the researchers. The thing to note is that it took years, decades, centuries to sift out the real chemical gold. By 1810 Dalton was taking about atoms, 1869 Mendeleev about the periodic table, 1860-1870 Maxwell & Boltzmann about statistical distributions, 1900 Planck about quanta, 1927 Dirac about electron symmetry, etc until I studied all this in the 60s. It’s amazing – gold from dross.
One takeaway from all this is that it took a long time to get it right, with plenty of divergences and meanderings along the way.
Another is that the underlying components being modeled in the overlying systems: atoms, electrons, electromagnetic fields all have three characteristics: identical natures by category, simple (sort of) aufbau constructions of more complicated pieces based on categorical symmetries, and relatively few degrees of freedom but which scale on building up bigger systems. The final binding in the modeling of large systems is that you can use lagrangians to describe the time evolution of the components. All neat and tidy.
Escaping with using fraudulent methods in modern physics is pretty hard these days.
The social sciences are a different matter. The model in my head is that the fundamental units of construction in human cultures and settings are missing the elements of simple symmetries and are not identical. Evolution has made sure that the fundamental units have a wide distributions of characteristics that are not readily orthogonalized. Evolution has resulted in this situation to make it more likely that the reproductive success of fundamental units is increased when external conditions change over time intervals that are significantly longer than the reproductive/child raising cycle.
The upshot in my view is that there is enough symmetry in the fundamental units and their interactions to make a go with robust statistical methods and workflow to build useful models. That said it takes a lot of work and time to sift the gold from the dross in this era. And, as in the alchemical era, there is money to be made and glory to be gained in the use of fraudulent methods.
Recommendation: might have been this, I’ve often recommended Plastic Fantastic:
https://statmodeling.stat.columbia.edu/2022/12/10/ppnas/#comment-2142883
Makes me think of an experiment from the late 1980’s (Eckhardt et al) that showed that gravity did not follow the inverse square law exactly. Well done experiment, etc. This was pretty startling so people investigated further and found that they should have taken into account the geology of the area–a large mass of dense material was below the surface and influenced the measurements.
If I remember, things proceeded very sanely. The researchers were cooperative and eventually accepted this explanation. They didn’t try to besmirch anyone or come up with excuses or crazy alternatives (as opposed to cold fusion). It has always given me more confidence in our field (physics).
I was stunned by the Eccles thing… yikes. In my entrepreneurship/innovation, inding that a huge result by Moretti is actually crap… and nobody cares. Extra yikes? It make me wonder how many key papers in my area are methodologically bogus. Blindly using structural equation modeling while never asking whether it works on that data set…
In grad school at Ohio State they really pushed us on doing the right methods the right way. That was an era when people were publishing articles on how many key studies were grossly underpowered. I had a proessor who published on how many PCA/FA studies in the A & A+ ournals were terribly flawed (not a route to popularity. lol) . The econ faculty loved to talk about how many regression analyses in the top econ ournals were done poorly. A bad regression in Econometrica? No reviewer or editor noticed? Does not inspire confidence in entrepreneurship research. Nobody gives a hoot about the flawed Moretti study.
One of the top entrepreneurship journals is doing a special issue on replication and I’m not sure how much this issue is top of mind. Anyway, thanks or ruining my week, lol… I am happy to keep you posted on the replication issue.
“had a professor who published on how many PCA/FA studies in the A & A+ ournals were terribly flawed”
Do you have a link to this paper?
Here’s the pdf. https://www.researchgate.net/profile/J-Ford/publication/227656338_The_Application_of_Exploratory_Factor_Analysis_in_Applied_Psychology_A_Critical_Review_and_Analysis/links/59db66d7aca272ab722b6f8b/The-Application-of-Exploratory-Factor-Analysis-in-Applied-Psychology-A-Critical-Review-and-Analysis.pdf
[To be honest, I think I’d terrified to look at landmark studies in any management discipline (I did immediately check my own papers, lol ]
The misuse of stat power is a doozy too…
Stat Power Audits
Mazen, M. M., Graf, L. A., Kellogg, C. E., & Hemmasi, M. (1987). “Statistical power in contemporary management research.” Academy of Management Journal, 30(2).
What they did: This study meticulously reviewed empirical research published in AMJ, the Journal of Management, and regional Academy proceedings.
What they found: Utilizing Cohen’s frameworks, they discovered that statistical power was heavily neglected. Small and medium effect sizes dominated the literature, yet the statistical tests utilized lacked the necessary sample sizes/power to detect them reliably, meaning researchers were frequently at risk of committing Type II errors.
Mazen, M. M., Hemmasi, M., & Lewis, K. E. (1987).
What they did: A sister study published the same year that expanded this empirical auditing lens specifically to the Strategic Management Journal (SMJ), assessing articles published between 1982 and 1984.
These 1980s baseline audits triggered several notable follow-ups that continued to look at the persistence of low statistical power in top management literature:
Mone, M. A., Mueller, G. C., & Mauland, W. (1996). “The perceptions and usage of statistical power in applied psychology and management research.” Personnel Psychology (Mone et al., 1996).
The Scoop: They updated Mazen et al.’s work by examining 26,471 statistical tests across 7 leading management journals (including AMJ, Academy of Management Review, and Administrative Science Quarterly). They found that despite Cohen’s and Mazen’s warnings, power levels remained low, and a survey of authors revealed that two-thirds never conducted a priori power analyses because journal reviewers rarely asked for them.
Combs, J. G. (2010). “Big Samples and Small Effects: Let’s Not Trade Relevance and Rigor for Power.” Academy of Management Journal (Combs, 2010).
The Scoop: Writing from an editorial perspective for AMJ, Combs noted that the pendulum eventually swung the other way. By 2010, researchers were using massive sample sizes that yielded phenomenal statistical power, but it introduced a new problem: studies were claiming “highly significant” results for tiny, managerially irrelevant effect sizes simply because the sample size was large enough to shrink random error to zero.