A correspondent who wishes to remain anonymous points us to this article in Technology Review, “Why Meta’s latest large language model survived only three days online. Galactica was supposed to help scientists. Instead, it mindlessly spat out biased and incorrect nonsense.” Here’s the story:
On November 15 Meta unveiled a new large language model called Galactica, designed to assist scientists. But instead of landing with the big bang Meta hoped for, Galactica has died with a whimper after three days of intense criticism. Yesterday the company took down the public demo that it had encouraged everyone to try out.
Meta’s misstep—and its hubris—show once again that Big Tech has a blind spot about the severe limitations of large language models. There is a large body of research that highlights the flaws of this technology, including its tendencies to reproduce prejudice and assert falsehoods as facts.
However, Meta and other companies working on large language models, including Google, have failed to take it seriously. . . .
There was some hype:
Meta promoted its model as a shortcut for researchers and students. In the company’s words, Galactica “can summarize academic papers, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more.”
Like all language models, Galactica is a mindless bot that cannot tell fact from fiction. Within hours, scientists were sharing its biased and incorrect results on social media. . . . A fundamental problem with Galactica is that it is not able to distinguish truth from falsehood, a basic requirement for a language model designed to generate scientific text. People found that it made up fake papers (sometimes attributing them to real authors), and generated wiki articles about the history of bears in space as readily as ones about protein complexes and the speed of light. It’s easy to spot fiction when it involves space bears, but harder with a subject users may not know much about.
I’d not heard about this Galactica thing at all, but the article connected to some things I had heard about:
For the last couple of years, Google has been promoting language models, such as LaMDA, as a way to look up information.
A few months ago we discussed that Google chatbot. I was disappointed that the Google engineer was willing to hype it but not to respond to reasoned criticisms of his argument.
The Technology Review article continues:
And it wasn’t just the fault of Meta’s marketing team. Yann LeCun, a Turing Award winner and Meta’s chief scientist, defended Galactica to the end. On the day the model was released, LeCun tweeted: “Type a text and Galactica will generate a paper with relevant references, formulas, and everything.” Three days later, he tweeted: “Galactica demo is off line for now. It’s no longer possible to have some fun by casually misusing it. Happy?”
I hate twitter. LeCun also approvingly links to someone else who writes, in response to AI critic Gary Marcus:
or maybe it [Galactica] was removed because people like you [Marcus] abused the model and misrepresented it. Thanks for getting a useful and interesting public demo removed, this is why we can’t have nice things.
Let’s unpack this last bit for a moment. Private company Meta launched a demo, and then a few days later they decided to remove it. The demo was removed in response to public criticisms, and . . . that’s a problem? “We can’t have nice things” because . . . outsiders are allowed to criticize published material?
This attitude of LeCun is ridiculous on two levels. First, and most obviously, the decision to remove the demo was made by Meta, not by Marcus. Meta is one of the biggest companies in the world; they have some agency, no? Second, what’s the endgame here? What’s LeCun’s ideal? Presumably it’s not a world in which outsiders are not allowed to criticize products. So what is it? I guess the ideal would be that Marcus and others would voluntarily suppress their criticism out of a public-spirited desire not to have Meta take “nice things” away from people? So weird. Marcus doesn’t work for your company, dude.
The funny thing is that the official statement from Meta was much more reasonable! Here it is:
Thank you everyone for trying the Galactica model demo. We appreciate the feedback we have received so far from the community, and have paused the demo for now. Our models are available for researchers who want to learn more about the work and reproduce results in the paper.
I don’t quite understand what it means for the demo to have been paused if the models remain available to researchers, but in any case they’re taking responsibility for what they’re doing with their own code; they’re not blaming critics. This is a case where the corporate marketing team makes much more sense than the company’s chief scientist.
This all relates to Jessica’s recent post on academic fields where criticism is suppressed, where research critique is taken as personal attacks, and where there often seems to be a norm of never saying anything negative. LeCun seems to have that same attitude, not about research papers but about his employer’s products. Either way, it’s the blame-the-critic game, and my take is the same: If you don’t want your work criticized, don’t make it public. It’s disappointing, but all too common, to see scientists who are opposed to criticism, which is essential to the scientific process.
The big picture
Look. I’m not saying LeCun is a bad person. I don’t know the guy at all. Anybody can have a bad day! One of his company’s high-profile products got bad press, so he lashed out. Ultimately no big deal.
It’s just . . . that idea that outside criticism is “why we can’t have nice things” . . . at worst this seems like an authoritarian attitude and at best it seems to reflect an extreme naivety about how science works. I guess that without outside criticism we’d all be driving cars that run on cold fusion, cancer would already have been cured 100 times over, etc.
P.S. I sent the above post to some people, and we got involved in a discussion of whether LeCun in his online discussions is “attacking” Galactica’s critics. I said that, from my perspective, LeCun is disagreeing with the critics but not attacking them. To this, Thomas Basebøll remarked that, whether the critics are on the “attack” or not, LeCun is certainly on the defensive, reacting to the criticism as though it’s an attack. Kind of like calling it “methodological terrorism” or something.
That’s an interesting point regarding LeCun being on the defensive. Indeed, instead of being in the position of arguing how great this product is for humanity, he’s spending his time arguing how it’s not dangerous. I can see how this can feel frustrating from his end.
P.P.S. LeCun responds here in comments.