Dale Lehman writes:
You are undoubtedly aware of the controversy surrounding the (now former) president of Stanford University. Here is the announcement of his resignation (Stanford University president Marc Tessier-Lavigne announces his resignation after flaws were found in his research (nbcnews.com)) and here is his statement (Issues with Five Papers and Planned Actions | Tessier-Lavigne Laboratory (stanford.edu)). I certainly have no direct knowledge of the situation and I suspect you don’t either. He may well be innocent of any fraud or misconduct. But his high profile and blaming unnamed other researchers working in his lab sound like a classic whitewashing. He admits to trusting these people and not acting quickly or thoroughly enough or having enough controls to protect the integrity of the research. But it strikes me as odd that somebody would expose him to such potential damage – unless their careers made them feel that they needed to do this – and, if that is the case, doesn’t he bear responsibility for establishing the climate in which has labs operate? I’m just wondering what your take is on the situation.
My reply: Yeah, I do kinda feel that way, but just on general principles, without any actual knowledge of the case. Often what seems like the most damning evidence comes after misconduct is revealed. One of the things that particularly bugged me about Columbia cheating on its U.S. News numbers was that the administration didn’t seem to care. I didn’t hear of anyone getting fired, I don’t recall any firm pronouncements from university officials about how such behavior is never tolerated here, etc. And I guess their stop-mentioning-the-scandal-and-maybe-it-will-go-away strategy worked, as there doesn’t seem to have been much of any followup on the story.
Regarding your suspicion about employees who cheat on behalf of their boss, even if the boss might not know about the details . . . I guess it depends. We’re used to thinking of the lab director as the bad guy, but unethical people can be found at all levels of an organization. I guess the problem is with the laboratory workflow, that they were publishing claims without checking them first. It’s tough, though, as a lot of science does run on trust.
It will never be fixed if the blame is placed at the lab level. The root cause is that *no one is doing independent direct replications*. This is 100% a funding problem.
As long as people know theres little chance of anyone checking their work youll have rampant incompetence, fraud, and kind of grey area “sanctioned incompetence” (eg p-hacking, or the ML version of reporting predictive skill after repeatedly running cross validations).
The fraud is a tiny part of it. Eg, if Id cynically stayed in medical research Id have no problem finding “significant” results and weaving whatever narrative I want. And that requires nothing “against the rules”.
Thats more common than outright fraud, but even that is minor. The biggest issue is no one knows enough about these phenomenon to even design a repeatable an experiment.
In that case its kind of like you may as well just make it up, because whats the difference?
My wife did her postdoc at UCSF and for the first few years she was in the lab across the hall from Mark Tessier-Lavigne (MTL). I asked her what she knew about this situation and she said that her understanding is that all of the problematic papers were authored by a woman named Elke Stein who worked with MTL over many years. She characterized the situation as Ms Stein having “mental health issues” that led her to deceive others out of her own stress or problematic behavior rather than any direct encouragement externally (systemic encouragement of the bio-research-industry, sure. Direct culpability for the issues from MTL? No). She said that MTL’s main “fault” was that he did what that field considers to be “success” which is to pull in many grants and grow the lab to a very large size and also get involved in business ventures and etc, spreading himself too thin. She said his lab covered essentially the entire floor of the building she was in, and her PI had a small corner where she ran a lab with 8 full time scientists, so I’m assuming based on that MTL had 25 or 30 members or something like that.
The problem is that science has become “capitalism”. Which is to say that the people who accumulate power and wealth more easily accumulate more power and wealth by wielding that power and wealth. A lab that has 2 grants is much more likely to get another one than a lab that has 1 grant. A lab that has 4 grants can very easily get a 5th compared to a start-up researcher with nothing but a university start up package… etc. The problem with this system is that quality of research is only ancillary to the evaluation process, similar to the situation with say Private Equity firms strip mining Red Lobster. The connection between providing a quality service and getting money and power is very questionable. In fact in science my experience has been the more careful and quality oriented the researcher is the higher the chance they will be driven out of the industry. Katalin Karikó was a prime example of this.
So, labs get big, the Universities absolutely promote that (because they get 50-60% overheads on top of every dollar granted to the lab) and people in biology internalize the experience that success demands getting multiple grants and running a lab that you’re trying to grow in size.
Now, my wife also says that rather than the MTL controversy, which she characterizes as basically a good scientist spread too thin to detect problematic behavior by an employee, she thinks the much more problematic situation is Berislav Zlokovic at the Zilkha center at USC https://www.insidehighered.com/news/faculty-issues/research/2023/11/22/usc-probes-star-neuroscientist-research-fraud-allegations
In that situation, the PI himself Zlokovic appears to have created a toxic environment of “get me the answer I want at any cost” encouraging underlings to create fraudulent data via “nudge nudge wink wink” type methods, or even possibly taking their notebooks and changing information in them.
To me, the system is broken, and MTL represents one way in which that can break, loss of integrity un-detected because of being spread too thin, and Zlokovic represents another way that can break, loss of integrity at the top of the lab pushing others to commit fraud. Both are problematic. But good PIs are much more afraid of the MTL type event, because they know they themselves are not cheating, but loss of trust in their employees leads to inability to do anything. From an information-security type perspective, if the “attacker” is in the house making the data and lying to your face it’s hard to detect without having such low levels of trust that it damages all your relationships with all your employees.
Daniel:
Interesting. I could imagine being fooled by a colleague or student who fabricates results. It’s hard to be on top of everything. I don’t think this has happened to me, except in small ways: a few times I’ve been involved in projects where the collaborator or student doesn’t seem serious about getting things right, or who seems willing to make strong conclusions from weak data, and I withdraw from such collaborations. I bet I could get scammed by a real manipulator.
We don’t have tons of money floating around, though, so . . . if there are any would-be manipulators reading this comment, I recommend you take your talents to a higher-cash-flow laboratory!
While I was there, 10 people reported to Andrew either directly or indirectly through ISERP, including me and the postdocs and research engineers, but excluding students. That required about $1.5M in grant money per year on top of a generous slush fund Andrew had through statistics. That’s well over a $2M/year burn rate (though around $550K of that went to overhead—universities are ridiculously inefficient at converting grant money to research). We were putting in north of a dozen proposals/year (we applied to NSF, NIH, ONR, IES, Schmidt Futures, Sloan, DARPA, and I was about to apply to Moore) among me, Andrew, and Ben Goodrich.
Definitely a rich-get-richer scenario. ONR literally showed up at Andrew’s door at Columbia and asked how they could give him money. Sloan helped us write our proposal. We had inside connections at Schmidt Futures. None of this is “objective” the way you might suspect it is when you’re a student. We weren’t insiders enough at DARPA, so we never got funding through them.
Always good to hear about specific observations that either fit, or don’t fit the theory. In this case, seems to fit well. I’ll keep harping on it ;-)
Bob:
I don’t know if I’d call it rich-get-richer. It was more of a rich-if-you-stay-on-the-hamster-wheel scenario. We still have grant funding, but a lot less, and not nearly as many people working on Stan-related projects here at Columbia. But that’s good, because lots of people are working on relevant projects at Aalto, Flatiron, and elsewhere. In that sense, you could say the system worked, in that it’s hard enough for me to raise money that I raise less of it, and lots of work still gets done. And in the meantime I think all that money was well spent. Bayesian workflow and Stan are making a difference in many different areas of science and engineering, in business, academia, and government.
My experience in reviewing grants and reading reviewer reports on the 50+ grants we submitted while I was at Columbia (we had about a 30% hit rate, which is well above average, especially as we always asked for max budget on the theory that the expected reward is higher), is that there’s a strong bias toward funding those who are already successful and run big labs. It’s why I don’t say “yes” any more to grant reviewing—they don’t need me to just keep giving money to the same “famous” people.
Even when we were starting, it would have been harder if we didn’t have someone as “rich” in fame as Andrew as a PI. In fact, Andrew was PI on almost all of our grants, even the ones I mainly wrote, for exactly this reason. It’s not something an unknown can lean into (well, maybe they can if they’re at Stanford or Berkeley and have the institutional rep).
Bob:
You write, “Andrew literally argued that we could write relatively shoddy grants . . .”
I don’t recall arguing this. What I said was that I’d rather allocate more time to research and less time to grant writing. I don’t think our grants were shoddy, either in relative or absolute terms!
Daniel’s assessment is generally correct.
Andrew writes, “I didn’t hear of anyone getting fired, I don’t recall any firm pronouncements from university officials about how such behavior is never tolerated here, etc.” I wonder if, at least in the sciences, one of the problems is that no one agrees on who is responsible for policing / punishing ethical violations. Is it the university? The funding agency? Each can think it’s the other’s job, and neither of them has an incentive to uncover wrongdoing. I have long been amazed at how little funding agencies care about the quality of science, or the structure and incentives of research groups.
Raghu:
You refer to the sciences, but when I wrote, “I didn’t hear of anyone getting fired, I don’t recall any firm pronouncements from university officials about how such behavior is never tolerated here, etc.”, I was referring to Columbia faking its US News statistics. That wasn’t done by the sciences. It was done somewhere in central administration, I guess. The statistics department wasn’t involved, that’s for sure!
Yes, certainly. I was applying your statement to the sciences because of Daniel’s comment. For Columbia faking its US News College Ranking statistics, it’s clear where the responsibility lies!
“unethical people can be found at all levels of an organization”
The article points out that these many fraudulent papers came from his work across at least three different organizations: Stanford, Genentech, and UCSF. It does seem unlikely that he played no part in the problem. Sometimes I wonder if you withhold criticism as a rhetorical strategy to get the reader to adopt those criticisms independently!
Peter:
Yeah, sometimes I do follow that strategy! In this case, though, I have not been following any of the details and so I really do want to leave it to others to form their own judgments.
I am surprised that no one has stressed that while Marc Tessier-Lavigne stepped down from the presidency, he still kept his faculty position at Stanford, and he remains on the Board of Directors of Regeneron.
https://www.youtube.com/watch?v=OHfVZ5rvxqA
More to the point, whatever transgressions Francesca Gino is accused of, her research endeavors (in behavioral science) are unlikely to cause death. The situation is quite different when it comes to data fabrication in neuroscience because lives depend on research in this field. In just the past couple of years, allegations of photoshop manipulations have turned up not just at Stanford, but at Johns Hopkins, Ohio State and USC. So to speak, it is too easy to cheat in this field.
Two individuals, Sholto David and Elisabeth Bik, are well-known sleuths for spotting manipulated/altered photos.
Aren’t there pretty solid tools already (I think adobe for example might have some) that can detect hallmarks of altered photos? Maybe journals should collaborate with some of these companies to pass neuro images through such a system as part of review?
Doesn’t that just mean the active liars will be passing their images through it too until they’re hard to detect? I don’t think an arms-race is a good plan here.