Zoe Ziani, a psychology researcher who had the misfortune several years ago as a Ph.D. student be tasked with following up on some unrelicable published psychology research, tells the story of how her department pulled the chair out from under her:
I [Ziani] started having doubts about . . . (Casciaro, Gino, and Kouchaki, “The Contaminating Effect of Building Instrumental Ties: How Networking Can Make Us Feel Dirty”, ASQ, 2014; hereafter abbreviated as “CGK 2014”) during my PhD. At the time, I was working on the topic of networking behaviors, and this paper is a cornerstone of the literature.
I formed the opinion that I shouldn’t use this paper as a building block in my research. Indeed, the idea that people would feel “physically dirty” when networking did not seem very plausible, and I knew that many results in Management and Psychology published around this time had been obtained through researchers’ degrees of freedom. However, my advisor had a different view: The paper had been published in a top management journal by three prominent scholars . . . To her, it was inconceivable to simply disregard this paper. . . .
We’ve called this the research incumbency rule: once a paper has been published, people act like it’s correct. There’s a high threshold for post-publication criticsm.
Ziani continues:
At the end of my third year into the program . . . I finally decided to openly share with her my concerns about the paper. . . . Her reaction was to vehemently dismiss my concerns, and to imply that I was making very serious accusations.
This sort of thing annoys me a lot: the personalization of scientific discourse. I don’t like the “scientist as hero” narrative I also don’t like the assumption that, just because someone does bad science, that they’re a cheater (equivalently the assumption that, just because someone doesn’t cheat in science, that their work is solid). Remember, honesty and transparency are not enuf.
By implying that Ziani “was making very serious accusations,” the adviser was doing a classic judo move, taking a technical, scientific criticism and turning it into something personal. It’s a sort of reverse ad-hominem argument that I think has no place in scientific discussion. It’s the last refuge of the scientific scoundrel.
Ziani continues:
It was at this point that I started suspecting that part of the evidence presented in CGK 2014 was not just p-hacked but based on fabricated data. At the time, I wasn’t clear how warranted these suspicions were, or about the best way to share them . . .
What I knew, however, was that I had accumulated enough theoretical and empirical arguments to seriously question the conclusions of CGK 2014, and that these arguments might be of interest to the scientific community. Indeed, CGK 2014 is an unavoidable building block for anyone studying networking behavior: It is authored by influential scholars, published in a prestigious journal, received the Outstanding Publication Award in OB at the 2015 Academy of Management annual meeting for its “significant contribution to the advancement of the field of organizational behavior”.
Ding ding ding! Harvard! Awards! Prestige! Next step, Ted and NPR.
Ziani continues:
I, a (very) early-career researcher, took a deep dive into a famous paper and discovered inconsistencies. . . . The three members of my committee (who oversaw the content of my dissertation) were very upset by this criticism. They never engaged with the content: Instead, they repeatedly suggested that a scientific criticism of a published paper had no place in a dissertation. . . . After the defense, two members of the committee made it clear they would not sign off on my dissertation until I removed all traces of my criticism of CGK 2014. Neither commented on the content of my criticism. Instead, one committee member implied that a criticism is fundamentally incompatible with the professional norms of academic research. . . . adopting what he called a “self-righteous posture” that was “not appropriate.”
This rings a few bells. Opposition to criticism, tone policing, and a go-along-get-along attitude that encourages cynical complacency and discourages “self-righteousness.”
Ziani tells more of the story:
I ran a replication of Study 1 of CGK 2014 using the authors’ original materials. Not only did I fail to replicate the original result, but I also found serious anomalies when comparing the data of my replication to the data of the original.
This often happens, including with legit studies. Look into almost any report carefully and you’ll find some data-related errors, sometimes fairly minor and sometimes in a way that’s fatal to the main conclusions being made (or, notoriously, here). Again, though, it’s not just the extreme Wansink or Lacour-level examples. Even solid studies typically have enough data rattling around that mistakes creep in, and the more carefully you look, the more mistakes you’ll typically find. This is not to excuse any misconduct in CKG 2014; it’s just a more general statement that data problems are to be expected.
And now on to Ziani’s conclusions about science:
From a truth-finding perspective, p-hacking is as damaging as fraud. . . . In a world in which ridiculous effects can be shown to “exist” thanks to p-hacking . . . how does one identify fraudulent findings? P-hacked effects also provide the implausible theoretical foundations on which fraudulent findings are built. . . .
Think about all the people who try to replicate, extend, or build upon these false positives. [I wouldn’t use the word “false positives,” as I don’t buy into the framework that effects are real or not, but I agree with the general point here. — ed.] Any resource spent trying to extend or replicate fake research is a resource that isn’t spent discovering real findings. . . . When a subset of scientists can reliably produce incredible effects (because they cut corners), and publish hundreds of papers, they set a bar that serious, careful researchers can never hope to meet.
I agree; this is similar to points we’ve made about about Clarke’s and Gresham’s Laws as applied to science.
Finally, Ziani has some specific comments regarding business schools:
The incentives for fraud in business academia are significant. If you can meet the standards for hiring, promotion, and tenure at an R1 university (something that is much easier once you fabricate your data), you will get:
– A 6-figure salary with full benefits until you retire
– Complete job security
– A flexible work environment (no boss, remote work…)
– The social status and reputational benefits that go with the “Professor/Dr.” title
– Opportunities to do book deals, TED talks, to teach in executive education, to conduct corporate workshops…
The benefits of fraud must be balanced with the risks of course. Are the risks of being caught for faking data high enough? I [Ziani] don’t think so:The peer review process, as it exists today, makes it extremely difficult to catch fraud. . . .
The bar to accuse someone of fraud is extremely high. Failing to replicate the effect? Not enough. Non-sensical effect sizes? Not enough. Anomalies in data? Not enough. Unless you can invest the resources to identify anomalous patterns of fraud across multiple papers, THEN drum up enough support from journals or universities to consider your suspicions, THEN hold their feet to the fire when they are unwilling to act… the probability that the person will never face consequences for fabricating data is very high.
The incentives to investigate and call out fraud are non-existent. In fact, the opposite is true: If you find something fishy in a paper, your mentor, colleagues, and friends will most likely suggest that you keep quiet and move on (or as I have learned the hard way, they might even try to bully you into silence). If you are crazy enough to ignore this advice, you are facing a Sisyphean task: Emailing authors to share data (which they do want not to), begging universities to investigate (which they do not want to), convincing journals to retract (which they do not want to), waiting months or years for them to share their findings with the public (if it ever happens)…
In summary:
Business academia needs to reckon with this inconvenient truth: Committing fraud is, right now, a viable career strategy that can propel you at the top of the academic world.
Not just business, and not just academia.
I just finished writing a paper, and read some stuff about fraud for it as well. Here is a quote from a correspondence by Flenning published in Nature 2004 if I am not mistaken that came to mind when reading this blogpost just now:
“For example, I asked a colleague who was familiar with the details of a recent case of scientific fraud why — given the risks — he thought the perpetrators had done it. He drolly observed that the real question is not why a few scientists commit fraud, but why more don’t do it. He went on to say that since the maximum penalty for getting caught (dismissal) was no worse than the routine penalty for not producing enough high-profile papers (no job), most junior scientists, at least, have nothing to lose by committing fraud.”
The same can perhaps be said for many questionable behaviors in psychological science, which is part of why they might occur. If committing fraud is a viable career option, building a career using questionable research practices might be an even more viable career option…
The saying used to be “crime doesn’t pay.” I’m not sure that is really accurate, since most nonviolent crimes are the result of rational behavior. But the modern version seems to be “truth doesn’t pay” and evidence seems to be piling up that this is true. And falsehoods also seems to be the result of rational behavior. So, what does that say about rational behavior?
I think we can still be confident that “truth does pay” in the long term. At least in the physical/biol sciences it’s likely extremely rare to be able to pursue a career based on mistruth and fraud unless one occupies a research backwater in which no one cares very much about the subject of the research.
So thinking of a few well known examples, Jan Hendrik Schon of semiconductor fame had a stellar fraud-based career of around 5 years (1997ish- to 2002ish) before he crashed and burned; Jonathan Pruitt’s (spiders) stellar career crashed after around 6 years (2014-2020); Elizabeth Holmes (Theranos) managed around 11 years (2004ish-2015ish) before self-destructing; the room temp superconductivity guy (Dias) lasted around 5 years…and so on.
I suspect a lot of these guys and gals committing fraud in highish impact fields may have had some sort of extreme risk-seeking or other sort of personality disorder that compelled them to commit fraudulent “research” that was bound to be uncovered.
Of course in politics serial lying and misrepresentation is quite rational but that’s largely because there isn’t a “long term” in politics (unless one happens to be a dictator). If you wish to make a mark and have a fulfilling career in science tho, it still does pay in the long run to be honest and truthful.
Chris:
Some people have been able to build successful careers with what look to me like junk science, for example Brad Bushman and Ellen Langer in psychology. I think that part of their success comes in their telling people what they want to hear, and the phenomena they study are slippery enough that they don’t need research success to achieve academic and popular success.
Maybe… that’s why I referred to mistruth and fraud in the physical/biol sciences where there is a fairly “strict” external reality and attempting to cheat that simply ends up cheating oneself in a manner that becomes very apparent eventually. I don’t know anything about Brad Bushman but if Ellen Langer is the Mindfulness woman (?) is there anything mistruthful or fraudulent in her work? I agree it’s possible that she may be telling people what they want to hear, but it seems a positive thing to pursue understanding of the influence of the mind on human wellbeing as a way of improving the latter. Perhaps there is a “dealing with first world problems” aspect to her work in the sense that someone in dire straits and unable to pay for their insulin medication (say) is not going to take too kindly to a suggestion that everyday they think of three things they feel certain about and to question these!
Chris:
You can search on Bushman in this blog for details on his work. I have no reason to think that either Bushman or Langer has any fraud in their work. But their work is mistruthful in that they make strong claims that are not supported by the data; see here, for example. I’m not super interested in the moral question of whether they know they are doing bad work, whether they are doing bad work on purpose, etc. My point is just that they have built highly successful academic and scientific careers based on a foundation of what seems to me to be junk science. It may be that the work of both Bushman and Langer have been, as you say, a positive thing for society. It’s not good science, but there are lots of ways to contribute to the world without doing science. My problem is not necessarily with the advice they are giving; rather, it’s with their claims of scientific evidence where there is none.
fair enough; since the topic of the top post is about “Committing fraud … as viable career strategy” that was where I was directing my posts.
I just noticed that Professor Langer did that “time perception influences wound healing” study which seems like something crying out for a “replication”!
Chris:
Yes, good point. I should not have thrownBushman and Langer into a discussion of a post about fraud.
Regarding the study claiming that time perception influences wound healing: if someone wants to replicate it, they can, but I wouldn’t recommend doing so, as it seems like it would be a waste of time!
On the other hand, if no one ever bothered to figure out what was required to replicate the crazy-sounding “Leyden jar” experiment we wouldn’t be using electricity today:
https://statmodeling.stat.columbia.edu/2024/10/20/what-is-the-purpose-of-a-methods-section/#comment-2381778
From the fact that some frauds were quickly found out in physical/biol sciences you cannot logically deduce that that will always be the case, or has always been the case. You simply do not know how many frauds are still working in science, undetected.
Yes that’s possible of course. One of the things that seems a massive red flag to me in the examples (Schon, Pruitt and Dias – this also applies to Diederik Stapel and probably many others), is the manner in which they contrived to conceal their data (or “data”) even from their coworkers. I guess if you’re going to do audacious fraud (fabricating data) that has to be the case but it’s concerning that students and collaborators were able to be isolated from the data that they were given to analyze or might even have collected (Pruitt) … maybe highlighting these examples will help in recognising this sort of ref-flag behaviour.
Matthew Walker has made a lot of money out of being a fraud in an important biological subject and has garnered a lot of attention, without facing any serious consequences. Although it’s clearly lower profile than Schon and didn’t defraud investors like Theranos, I’m not exactly sure why this case went unpunished. Luck?
Joseph:
For the reply of the relevant University of California official when asked about Walker’s research misconduct, see here: https://statmodeling.stat.columbia.edu/2020/03/24/why-we-sleep-a-tale-of-institutional-failure/
My conclusion was that they didn’t want tol admit that they were paying $218,749.00 a year to someone who was doing research misconduct.
Harvard and Yale similarly did nothing about the famous plagiarists in their law school, Stanford did nothing about its podcaster who promotes junk science, USC did nothing about the plagiarist in its medical school. Or, to get closer to home, Columbia seems to have done nothing about its scandal-clouded cancer researcher.
So, when you say you’re not exactly sure why that case went unpunished, my response is that it’s my impression that most such cases go nowhere. There are some high-profile examples of faculty getting kicked out for research fraud (Marc Hauser and Francesca Gino at Harvard, Brian Wansink at Cornell, Erick Jones at the University of Nevada, Diederik Stapel at Tilburg University), but most of the time they just stay around. When a particular case goes unpunished, I’d just say that’s standard policy.
What I am interested in is 1) how much of that $218,749.00 a year possibly came from grants, and 2) how much of the grant money did the university receive and subsequently used for things like “overhead costs” or whatever the appropriate term.
Maybe such answers can help me try and make sense of this all.
Although it’s clearly lower profile than Schon and didn’t defraud investors like Theranos, I’m not exactly sure why this case went unpunished.
Good question Joseph. In addition to what Andrew just posted, I’m guessing Matthew Walker is considered to be an example of mediocre/second-rate research and dubious self-promotion rather than fraud; and maybe it’s considered OK to put anything you like in a pop-sci book.
But I’m not sure the case has gone unpunished since he is now known to engage in dubious research practices; he hasn’t escaped reputational damage. He’s had an article retracted for massive self-plagairism.
““truth doesn’t pay” and evidence seems to be piling up that this is true”
Dale: I’ll give you a dollar for saying this.
But the full saying is “crime doesn’t pay, because if it did, it wouldn’t be a crime”!
I had a similar experience, though quite a bit less intense and not personally damaging, when writing a paper for a graduate course. My paper was on a niche topic. Most of the papers I cited were by the same person. Trying to follow through the various plots in the different papers by that person revealed major inconsistencies. My discussion in my paper of these inconsistencies yielded a gentle chiding from the prof, on the order of “you’re making way too much of this”.
Back then most students would have avoided that discussion, but I had previously found errors in text books and other papers that I had confirmed. Those early experiences – where a scientist had said “oh, wow, that is a mistake!” – taught me the wrong lesson. I guess when you’re a nobody undergrad pointing out errors, its kinda cute. But once you’re a grad student aspiring to a professional position, everyone’s Ike Clanton: “Listen, Mr. Kansas Law Dog. Law don’t go around here. Savvy?”
This research-whistleblower manual seems relevant and deserves more attention: https://zenodo.org/records/8192478
Yes, there seems little meaningful reason to doubt that investigation of fraud and mis-conduct is negatively incentivized. And it’s not a hot-take to assume that changing people’s attitudes and the sociology of research to make the investigation of fraud and mis-conduct to be perilous, with a low probability of success.
So then, well-considered and ‘official’ methods for people to safely and effectively call-out fraud (such as the linked, pubpeer, and social media) are an important and perhaps the only alternative.
Maybe I should stop reading this blog after 17 years. I keep learning things about research and publishing practices that make me sound like a raving crank if I try to explain them to my normal friends.
Really. As I’ve mentioned before, I was following a blog by a couple of Canadian grad students that said something nice about Wansink. I warned them that his research was probably problematic and they should check out the references I gave. I got royally flamed for impuning the honor of a respected researcher. I had never, ever, run into such closed minds in my life. The complete lack of intellectual curiosity was shocking.
David:
That’s kind of horrible to hear. On the other hand, it’s kind of heartwarming that the world is big enough that there are pockets of Wansink fans out there. What’s troubling is when these things go mainstream, as with the mindless promotion of mind-body healing (as here and here), the anti-vax movement which seems to have taken over the U.S. government, and the push from the elite media on UFO’s as space aliens.
The root “fraud” is failure to regularly perform the independent replications. Use of the term “science” leads the public to believe that is being done, but it is extremely rare in all these problem areas of research.
The trick is to do so-called conceptual replications where a key element has been changed (eg, use males instead of females). Thus, either you get a successful replication or the failure can be attributed to the different design. In this way no prior research need ever get questioned.
Very convenient for the researchers but unfortunately thats something other than science, and it isn’t capable of yielding the promised benefits.
“Very convenient for the researchers but unfortunately thats something other than science, and it isn’t capable of yielding the promised benefits.”
Maybe this depends on what the “promised benefits” exactly are.
I mean, journals can publish lots of this kind of research and maybe even ask money from the members of the general public to pay to have a look at the published research.
Scientists can make a career out of doing things sub-optimally perhaps, so that might benefit a few people in that regard.
Universities receive tuition costs if I am not mistaken to “teach” students that kind of research that may have employed sub-optimal practices. So the universities kind of indirectly benefit.
Etc.
Stuff like electricity and cures for disease. Making nontrivial predictions about the future.
It does function as a jobs program, but itd be cheaper to go back to monasteries for that.
Also, suboptimal isn’t really accurate term for it. These are more like bizarro practices that are the opposite of science. Performing replications is considered “not novel” and thus unworthy of funding.
I have been following along with the Many Coauthors Project, which has pretty much run out of gas. In the first link above, Ziani listed three conclusions, here are Nos. 2 and 3:
“2. The strength of evidence against the null presented in study 1 of the paper made it extremely unlikely that the result was p-hacked: It is statistically implausible to obtain such a low p-value under the null, even when using researchers’ degrees of freedom.
3. Francesca Gino had many other papers that appeared equally implausible (i.e., untrustworthy psychological mechanisms leading to large effects with very low p-values).”
Gino became a superstar for one reason and one reason alone, and it can be written very succinctly, p < .001 for the main effect. That number appears in paper after paper, enough that I lost count when I was skimming through. No matter how wacky the intervention, the effect was always huge. A preternatural marvel!
Ziani goes on to say:
"I am very disappointed with Gino’s more senior co-authors. It is shocking to me that, Juliana Schroeder excepted (who has sent a clear signal to the scientific community that she is taking this scandal seriously, and shared transparent guidelines on how she is evaluating the papers she has co-authored with Gino), none of these tenured co-authors have made any sort of public statement. Do they not think that the scientific community deserves more transparency? Do they not feel the need to signal whether and why people should still trust their work? I still have some hope that the repeatedly delayed Many Co-Authors project will shed some light on these questions… but I’m not holding my breath."
I think we can close the book on this now, and the answers to the questions at the end are "no" and "no." Other than Juliana Schroeder, who moved as quickly as possible to retract any papers she shared with Gino, the predominant response in the Many Coauthors Project was to circle the wagons. Again and again, the coauthors responded with silence about Gino accompanied by insistences that "these findings have been replicated in [insert paper title here]." And then you dig into that paper and find a lot of bait and switch until you are not really sure what exactly replicated, because it certainly wasn't the study Gino performed.
The field of social psychology got a nudge, but barely budged.
Quote from above: “The field of social psychology got a nudge, but barely budged.”
This reminded me of a paper I came across recently when writing my recent paper. I think it explained a few things I wondered about for over 10 years now ever since the fraud by Diederik Stapel was investigated and mentioned back in 2012 or thereabouts. It also ties in with this blogpost concerning a few things mentioned by Ziani concerning a more recent case of whateveryouwanttocallit.
From a paper by Martin (1992) “Scientific fraud and the power structure of science”
“The main opposition comes from those who lose out or prefer to play the game a different way. By and large, these critics are not influential; they have been unable to do more than occasionally voice concern about the practices, which continue unabated”
Gino may still skate. Defamation was a laugher, and she will most likely lose on her breach-of-contract claims as well (courts have repeatedly held that “employee handbook” type information does not constitute a contract).
But she may very well be found to be protected by the Statute of Limitations. Harvard’s argument is that her continued use of her scientific findings in Ted Talks, etc., constitutes renewal of responsibility, but that claim is not responsive to the reasons that Statute of Limitations exist in the first place.
Can you imagine Gino being re-instated at Harvard with full tenure and back pay, raging furiously against the injustice of the accusations?
“Can you imagine Gino being re-instated at Harvard with full tenure and back pay, raging furiously against the injustice of the accusations?”
I can totally see that happening.
It also reminded me of the Stapel case, where he wrote in his book that a colleague wrote an e-mail to him in which she said something like:
“She’d rather be in that elevator with somebody who’s made professional mistakes, but who’s still an OK person, than with somebody whose work is just fine, but whose personal life is an immoral mess.” (Stapel, 2014, p. 193. Side note: Thank you for the translation Mr. Brown! It was useful again recently when writing).
If I am not mistaken, I also remember some folks saying that Stapel deserved a 2nd chance at a university or something like that in some special role or I don’t know what they thought of. That was shocking to hear. Like, your primary job as a scientist regards facts/truth/etc., and when you screw that up for a decade or so there are apparently still people who think you can remain employed at a university after messing up that badly.
“whose personal life is an immoral mess.” ”
A very famous married male computer scientist whom I used to play Go with, “fell in love” with one of his female students. Her only option was to run like hell.
Immoral messes in academia can be devastating. Please. Don’t make light of them.
Here’s an interesting bit of recent news on the case:
Law Firm Withdraws From Representing HBS Prof. Gino in Suit Against Harvard
https://www.thecrimson.com/article/2025/2/15/nesenoff-miltenberg-withdraws-gino/
“Harvard Business School professor Francesca Gino is no longer represented by Title IX law firm Nesenoff & Miltenberg after four attorneys from the firm withdrew from her discrimination case against Harvard on Thursday. …
After Joun’s partial dismissal, Gino revised her complaint to incorporate additional sex-based discrimination claims under Title VII of the 1964 Civil Rights Act, which governs discrimination in employment decisions.
Gino is now represented by Patrick J. Hannon and Barbara A. Robb, lawyers from Hartley Michon Robb Hannon. Both joined Gino’s legal team on Jan. 27, a month after Nesenoff & Miltenberg lawyer Julie A. Sacks withdrew from the case on Dec. 18.”
During 50 years in academia, including 16 years as an administrator, I heard many odd claims, but I found the reported view of three members of Ziani’s committee – “that a scientific criticism of a published paper had no place in a dissertation” – utterly bizarre. I don’t doubt the accuracy of Ziani’s report. I can’t guess what the three members had in mind (other than, e.g.,, “Don’t get us into trouble with a powerful person at an even more powerful institution.”).
I served on PhD dissertation committees in philosophy, math, statistics, psychology and education, but I’ve read PhD dissertations from other disciplines (e.g., physics, religious studies, English, linguistics, history, economics, political science, neuroscience) and almost all of them contained criticisms of published papers. The exceptions were math dissertations in which a theorem was stated and proved, and physics dissertations in experimental high energy physics reporting experimental methods and results. But I know of no reason to prohibit math or physics dissertation criticism of published papers.
I can imagine dissertations in other disciplines that did not engage in some criticisms of a published paper, but that would not help to justify prohibiting all dissertation criticism of a published paper.
So far as I know, none of the many meanings attached to “scientific” would make the three committee members’ view plausible. Stipulating an unusual meaning for “scientific” (e.g., “whatever in a published paper must never be criticized in a dissertation”) would result in the unhelpful claim, “Thou shalt not criticize that which should not be criticized ….”
Business schools are avowedly more ‘ideological’ than many others, but not even stalwart devotion to promoting capitalism would justify the the three committee members’ view.
” Instead, they repeatedly suggested that a scientific criticism of a published paper had no place in a dissertation. . . .”
Um, what?
Surely that can’t be a norm in psychology.
It is not a norm in psychology and seems very bizarre to me (as someone who has defended a psychology dissertation)
Jamie:
There’s lots of variation within academic psychologists. Some psychology professors are very much open to criticism. Others, not so much.
Here’s a story for you. A few years ago I was invited to speak at a psychology department at a prominent university. Before my talk, I was instructed to avoid talking about a certain topic that might embarrass one of the professors in that department. And then that person didn’t even show up to my talk! I guess the important thing was that nobody at the university be exposed to any subversive ideas.
I think that’s right that there’s a lot of variety, but my response was more specifically geared towards: “Surely that – (scientific criticism of a published paper had no place in a dissertation) – can’t be a norm in psychology”.
I really don’t think it is a norm in psychology for it to be considered improper to criticise published work in a dissertation.
I can imagine, though would not endorse, many departments (not just in psych) having some norms where they’d ask speakers not to say something that they think their faculty will get embarrassed by.
Jamie:
Regarding your last paragraph: I don’t think it’s a norm. I’ve given hundreds of talks, and this was the only time I’ve been asked not to speak on a specific topic.
Something similar happened to me. Several years before the Wansink scandal broke, I was scheduled to give a talk at a center on food policy and obesity. My talk wasn’t going to be mainly about Wansink, but I did plan to criticize the mediation analyses in his work.
A member of the center politely urged me to tread carefully. She told me that Wansink had a “personal/professional relationship” with many members of the center, and she worried that I might upset those other members by being too critical.
The norms and training in management make it nearly impossible to challenge fraud. Management research obsesses over “framing” and theoretical novelty, to the point where researchers in other disciplines have a hard time publishing in management journals even when they’re doing very relevant research. Simply showing evidence that X causes Y isn’t enough, even when X and Y are objectively important, you have solid identification, etc…. You must explain WHY X causes Y in a way that makes a compelling story. Black boxes need not apply; bonus points if your proposed explanation is a bit counterintuitive and lends itself well to the Freakonomics treatment. Oh, and don’t worry too much if the explanation is actually robust. Causal mediation analysis? Sounds great!
This fixation on the “why” means that every paper has to have its own original theoretical contribution or else its not seen as significant enough for a top journal. You know what kind of paper is really difficult to frame around an original theory? A paper which challenges the empirics behind an existing theory. Even if you empirically demonstrate that prior research doesn’t hold up, top management journals won’t accept it without your own alternative theory. OK, so the prior work says X causes Y. Now you’re arguing that X doesn’t cause Y, but unless you can come up with your own theory about what you think is causing Y (or what the effect of X is), you don’t have a theoretical contribution and therefore you don’t have a publishable paper.
Maybe I’m getting old, but I have mixed feelings about Ziani and her position. I’m around the same age as Gino and in an area tangentially connected to her. Even when I was a grad student, there was doubt about the robustness of this type of social psych, so this goes way back before Gino.
Now, if you incredulously believe in this literature and devote your life into it, at what point should you bear some of the burden for not doing your own due diligence. It reminds me of ex-Scientologists, some of whom really do seem quite sympathetic, but others seem like they were perfectly happy to ride the gravy train until it’s no longer in their benefit. I don’t know which type she is, but I’m bothered at lack of reflection in her writing on how she (and other students) can improve her critical thinking, which seems so lacking in their field.
Anon:
I’m getting old too, but I have more sympathy than you do toward Ziani. I don’t know any of the people involved in this story, but, to speak in general terms, students are typically strongly influenced by the ideas around them. I know I was! It’s not so simple as all that, because some selection is involved (yes, I was influenced by my teachers, but I had some leeway in picking which teachers to follow); still, I think it’s natural for students to go with what they’ve been taught, and it can take a lot for someone to break away. I don’t see Ziani’s story as lacking in reflection on how students can improve their critical thinking; it’s just that much harder for students to think critically if their attempts to engage in critical thinking are suppressed.
I encourage my students to think critically and express their disagreements with me, but not all teachers do that.
Did you even read her post? She said she had doubts about the paper from the start, but that she was forced to build upon it by her advisors.
Way to miss the point lol. Ziani was forced to work on the topic, and it’s only through her effort (and Data Colada’s) that Gino was exposed.
I did read her post and I understood her point. But make no mistake this is HER account and her narrative. I simply choose to take her account with a grain of salt. As we have seen this from this episode, uncritically accepting the narrative and identifying heroes, villains, and victims is part of the problem. Hence I said I do feel cynical when I wrote my comment, and I’ll add that I don’t derive any joy from it.
OTOH, your reaction is more evidence for me that there does need to be more questioning of people’s narratives. Questioning people claims of victimhood is not cool, I get it. I would never do this if I had to put my name to it, but the answer is not to uncritically berate others to accept whatever narrative people put out. You believe her narrative, I get it and I respect your position. But voicing doubt is not equivalent to laziness, malice, or lack of intelligence, I hope you can accept that as well.
I’m all for informed skepticism. The reasons why I think her story can be trusted are:
A New Yorker interviewed her, and apparently saw enough receipts to print quotes from the pressure she received from her advisors (https://www.newyorker.com/magazine/2023/10/09/they-studied-dishonesty-was-their-work-a-lie).
She was the whistleblower on the Gino case. I think it signals a skepticism/doggedness from her that is hardly compatible with the “gullible grad student incredulously believing zanny findings”.
Just to give an example of how the world is not entirely corrupt, I had the opposite experience in my dissertation. It was all theoretical (game theory), but in a late stage I got a letter from my advisor saying that my result contradicted what was found in another paper, by a Nobel-winner no less. I stayed up a few nights and found a flaw in the Nobelist’s model. (Pure luck on my part, I make no claims.) I sent it to my adviser, and he said, looks like you’re right this time, go ahead and finish. One of the scarier weeks of my life, but it turned out well. (This was in econ.) My take: a *lot* depends on your advisor.
Quote from above: “My take: a *lot* depends on your advisor.”
Even a fraudulent psychological scientist wrote that “People never all react the same way to a situational stimulus; there’s always some variation. People are different from each other (duh!); sometimes just a little, sometimes a lot.” (Stapel, 2014, p. 148-149)”.
I think much more attention should be given to individual differences, personalities, choices, responsibilities, etc. in light of questionable and unethical research behavior (in its many possible forms) of scientists. Not doing so might also result in something similar that might have happened concerning ambiguity in research practices. For instance, as Simmons et al (2011) note: “Ambiguity is rampant in empirical research.” which may facilitate a particular type of “serial abuser” who “(…) uses his experience and deviousness to exploit uncertainties or ambiguities in research guidelines and prospers in poorly regulated, grey areas.” (Kwok, 2005, p. 554).
Perhaps assuming that all or most scientists have good intentions, and not thinking and talking about how scientists might have different personalities that may influence their behavior, and mostly thinking and talking about “the publish or perish system” and “the incentives” results in a situation where questionable and unethical research practices are almost seen as inevitable or defensible and something that everyone does. This in turn might facilitate the type of unethical and bad scientist, which further reinforces the idea, and reinforces the process. Perhaps that’s part of why academia is in the mess it is in.
My experience in plant physiology has been very different. In 1990 I attended a conference as second year grad student and realized that the mechanism assumed to be underlying the observed physiology was almost certainly not in play. During a break I ran my idea past my advisor. She said that my logic held and that I should work it up into a mini presentation for the conference’s final discussion.
I made my presentation and started a heated discussion that ended with the senior silverback of the field declaring “I’m sure you’re wrong, but I can’t say why. Come up with the correct experiment, and I’ll help you as much as I can.” It took me almost a year of modeling and preliminary work, but I went to him with a proposed experiment, one that required equipment he had that my lab lacked. He said, “I still think you’re wrong, but it is the right experiment. Come spend next year in my lab.”
I did. He was very supportive throughout. The experiment proved me mostly correct and his earlier work somewhat flawed. He encouraged me to publish.
In other words, sometimes the system works as advertised
Sometimes we need to hear stories like this to counter the constant influx of stories about rent seeking academics. Thank you for sharing (and glad it worked out so splendidly).
Economics does not even consider this stuff fraud. In most fields, it is not possible to get tenure at a research university without doing things everyone knows are wrong
Alfred:
That last sentence of yours cannot be correct. I know tenured faculty in research universities all over who have not done any sort of research misconduct or misrepresentation of data or research incompetence or whatever you want to call it.
OK, so I exaggerated. Apologies.
But this is definitely true: in large parts of empirical economics (called “applied micro”), the Top 5 journals insist on results that can only reflect p-hacking and other bad behavior. Just take a look at a few published articles. Typically there are 5-6 “outcome variables” and the results for every single one neatly support the authors’ case. People use data that nobody should take seriously, or the authors badly misrepresent the data. And in some fields, people rely on approaches that have been shown not to work.
This situation exists largely because of the hierarchical nature of the economics profession. A handful of people and departments control access to the Top 5, and they are not going to ask questions about what has delivered professional benefits to them and their students.
The top journals generally refuse to publish criticism. So do most of the lower-ranked journals. The profession has a lot of gossip about this problem, but the only place to find serious discussion of specific work is in online working paper series. There are some projects in which people download code and data for published papers are try to “replicate” them. This is a minimal exercise: it amounts to checking that the code works and, in some cases, asking whether the latest standard errors razzle-dazzle makes any difference. This “replication” does not ask the more serious questions about whether the models are well-specified, whether the data are what the author claims, etc. These efforts do not really police bad research conduct.
Young economists discussing the Gino case note how similar their world is to what she did. Economics deals with different questions But the basic notion that to succeed, all work must have clean, dramatic, flashy results… that nowhere in an article can the author concede that something is not 100 percent perfect… that’s increasingly the same in applied economics, with bad consequences for the research ethics of those involved. Someone who won a “Nobel” a few years back likes to say that the best benefit of his status is the right to be honest.
That’s a formula for Ginos.
Though harsh, I mostly agree with what you say. But I would highlight what I think is the chief reason why this situation persists, indeed flourishes. That is that the vast majority of applied micro papers use data that is not publicly available. Even when journals have open data policies, they are always open to exceptions – such as when the data is proprietary. Which it often is for applied micro. Regardless of journal “requirements” for significant results, and regardless of authors’ insistence on confirmatory analyses, without the ability to reanalyze the data nobody (not peer reviewers, nor editors, nor “methodological terrorists”) can prevent these practices. The replication packages you mention rarely contain the raw data, so what is replicated is that the code works as stated, but without the ability to see what happened between the raw data and the analysis. The myriad forking paths remain largely un-mapped.
90% of research , across all fields is unreliable . Be there no doubt that
the dishonesty is rife . However, a whistleblower will perish long before justice can be served .
From yesterday’s Wall Street Journal
Harvard Revokes Tenure of Star Professor Accused of Research Misconduct
Francesca Gino, a business professor whose research focused on why people lie and cheat, co-authored papers that included manipulated data
Lessig recently published some new podcasts on the Gino case. I found his discussion in final episode /S3E9 rather interesting in almost unwittingly touching on these incentives: https://theginocase.info/
“In a certain way, these scholars run a kind of academic paper factory. Across the period Francesca was working and the papers she was working on that are the subject of this prosecution, she employed, in total, more than 60 research assistants. …”
“As I’ve said throughout these podcasts, this is a weird field of science where the work is almost factory produced, where there are many people involved in the production of the science, and where, frankly, not much actually hangs on the results. This is not like Boeing screwing around with safety standards. No plane is going to fall out of the sky if there’s an error copying data in an excel sheet. So, it’s not hard for me to understand how people would not take seriously enough the need for precision in the work that they do pas they prepare these data. Especially when you understand who’s preparing the data, undergraduate research assistants, people with a million other things going on, people for whom absolute accuracy may not seem as important as answering that next snap.”
This is from the Levelt et al. (2012) investigative report concerning a fraud case involving a social scientist. Similar perhaps to the Gino case in some ways, but a little over a decade earlier. Here are some quotes from that report to underline the “factory” aspect, and the “many people involved in the production of the science”:
“In most cases, the data came from studies involving first or junior year students (Amsterdam, Eindhoven), or undergraduate students (University of Michigan, University of Georgia), see for example Chapter 5. First-year psychology students at the University of Amsterdam fulfilled an obligation to act as experimental subjects for a number of hours.” (p. 39)
“Mr Stapel’s group held an honours class for motivated social psychology students, with regular sessions to discuss research. This little club was an important source for recruiting PhD students.” (p. 41)
And taking an even bigger picture perspective concerning the “academic paper factory” view, the following might be interesting. I recently came across the following by Fyfe et al. (2017) titled “Untangling Academic Publishing”:
“Just four large commercial firms each publish more than 2,000 journals: Springer Nature, Elsevier, Wiley-Blackwell, and Taylor & Francis (Ware & Mabe, 2015, p 45) Their profitability has become so reliable that they are regarded as sound investments (Morgan Stanley Equity Research, 2002; Aspesi & Luong, 2014) This ‘oligopoly’ of big commercial firms has most influence in the social sciences, where they publish 70% of articles globally.” (p. 10)
And then one could think about the possible role in this all of social studies at universities and student loans, tuition, possibly largely useless diploma’s and poor education, and who knows what else. Viewing certain things from the perspective of some sort of “factory” might be appropriate from a bigger perspective as well. It’s no surprise perhaps that the fraudulent social psychological scientist co-wrote a book after his fraud became known titled “The Fiction Factory. A Liberating Epistolary Novel”.
Also see Binswanger (2014) “Excellence by nonsense” -paper:
“Why did successful and independent universities forget about their noble purpose of increasing knowledge and instead degenerated into ‘‘publication factories’’ and ‘‘project mills’’ which are only interested in their rankings?” (p. 50)
“Today’s universities are, on the one hand, fundraising institutions, determined to receive as many research funds as possible. On the other hand, they are publication factories, trying to maximize their publication output.” (p. 53)
A paper factory
That’s what it is, that’s what he told me
But it’s likely more than than as far as I can see
For instance, the factory likely has a door, and that door likely has a key
Papers, journals, subscriptions and paywalls, who stands to gain?
Students, universities, loans and tuition, is it that hard to explain?
Is that part of the answer to why some restrain?
Is that part of the answer to why things remain?
A publication full of non-replicable findings, nonsense, or even lies
A diploma after which you go on and sell fries
But in the meantime someone gets the prize
It pays when someone sells and when someone buys