How horrible. I remember when The Onion started. They were so funny and on point. And now . . . What’s the point of even having The Onion if it’s running plagiarized material? I mean, yeah, sure, everybody’s gotta bring home money to put food on the table. But, really, what’s the goddam point of it all?
Jonathan Bailey has the story:
Back in June, G/O Media, the company that owns A.V. Club, Gizmodo, Quarts and The Onion, announced that they would be experimenting with AI tools as a way to supplement the work of human reporters and editors.
However, just a week later, it was clear that the move wasn’t going smoothly. . . . several months later, it doesn’t appear that things have improved. If anything, they might have gotten worse.
The reason is highlighted in a report by Frank Landymore and Jon Christian at Futurism. They compared the output of A.V. Club’s AI “reporter” against the source material, namely IMDB. What they found were examples of verbatim and near-verbatim copying of that material, without any indication that the text was copied. . . .
The articles in question have a note that reads as follows: “This article is based on data from IMDb. Text was compiled by an AI engine that was then reviewed and edited by the editorial staff.”
However, as noted by the Futurism report, that text does not indicate that any text is copied. Only that “data” is used. The text is supposed to be “compiled” by the AI and then “reviewed and edited” by humans. . . .
In both A.V. Club lists, there is no additional text or framing beyond the movies and the descriptions, which are all based on IMDb descriptions and, as seen in this case, sometimes copied directly or nearly directly from them.
There’s not much doubt that this is plagiarism. Though A.V. Club acknowledges that the “data” came from IMDb, it doesn’t indicate that the language does. There are no quotation marks, no blockquotes, nothing to indicate that portions are copied verbatim or near-verbatim. . . .
Bailey continues:
None of this is a secret. All of this is well known, well-understood and backed up with both hard data and mountains of anecdotal evidence. . . . But we’ve seen this before. Benny Johnson, for example, is an irredeemably unethical reporter with a history of plagiarism, fabrication and other ethical issues that resulted in him being fired from multiple publications.
Yet, he’s never been left wanting for a job. Publications know that, because of his name, he will draw clicks and engagement. . . . From a business perspective, AI is not very different from Benny Johnson. Though the flaws and integrity issues are well known, the allure of a free reporter who can generate countless articles at the push of a button is simply too great to ignore.
Then comes the economic argument:
But in there lies the problem, if you want AI to function like an actual reporter, it has to be edited, fact checked and plagiarism checked just like a real human.
However, when one does those checks, the errors quickly become apparent and fixing them often takes more time and resources than just starting with a human author.
In short, using an AI in a way that helps a company earn/save money means accepting that the factual errors and plagiarism are just part of the deal. It means completely forgoing journalism ethics, just like hiring a reporter like Benny Johnson.
Right now, for a publication, there is no ethical use of AI that is not either unprofitable or extremely limited. These “experiments” in AI are not about testing what the bots can do, but about seeing how much they can still lower their ethical and quality standards and still find an audience.
Ouch.
Very sad to see an Onion-affiliated site doing this.
Here’s how Bailey concludes:
The arc of history has been pulling publications toward larger quantities of lower quality content for some time. AI is just the latest escalation in that trend, and one that publishers are unlikely to ignore.
Even if it destroys their credibility.
No kidding. What next, mathematics professors who copy stories unacknowledged, introduce errors, and then deny they ever did it? Award-winning statistics professors who copy stuff from wikipedia, introducing stupid-ass errors in the process? University presidents? OK, none of those cases were shocking, they’re just sad. But to see The Onion involved . . . that truly is a step further into the abyss.
At least only Onion AFFILIATED.
I’ll decide to leave my fantasy bubble unpopped and continue to enjoy the Onion PROPER.
By the same logic: Columbia stats professor cheating on US rankings
Total:
Columbia cheated on the U.S. News ranking, but this had nothing to do with our statistics department! Had they talked with us, I assume we would’ve recommended that they process their numbers transparently, and that scandal never would’ve happened.
Yes, but you’re affiliated with Columbia. If the Onion is guilty by association…then so are you.
It’s sad, I like this web site, but to have it tarnished by this kind of rampant cheating is just appalling.
Total:
Fair enough. This website is Columbia-affiliated, so take the good with the bad. It really is unfortunate that the university didn’t talk with the statistics department regarding how those numbers were prepared.
Not the same logic, yours is fallacious logic.
Plagiarism and errors can be considered two different problems. Most complaints of stat/predictive models seem to me time-boxed; even if the models right now are not better than some standard, there is often no fundamental reason they won’t be improved in the future to be as accurate (or more accurate) than humans.
I find the plagiarism more interesting to discuss. I’ve gotten a few referrals to my blog from this service
https://www.perplexity.ai/search/Fit-beta-binomial-uQpfz454RBm5MVWxRRcirw?s=c
It is stochastic, so that current result seems to me most would find that an acceptable revision (prior I have done that search and I would have described as more clearly plagiarized, but it always gave a link to my site as the source).
So do folks think that is acceptable? Just a link/warning “I am a robot parroting other things I’ve assimilated from X,Y,Z?”.
Snopes co-founder and then-CEO David Mikkelson was also found to be a serial plagiarist a few years ago https://www.nytimes.com/2021/08/13/business/media/snopes-plagiarism-David-Mikkelson.html
I am surprised that nobody has yet mentioned several famous, real instances of academic plagiarism by presidents of universities. The first involved a stirring commencement address and has to do with making one’s bed in the morning
https://video.search.yahoo.com/yhs/search?fr=yhs-iba-3&ei=UTF-8&hsimp=yhs-3&hspart=iba&p=university+of+texas+and+bed+making&vm=r&type=gyff_9318_FFW_US#id=4&vid=f6c114818634b20c30e7b1a9c668c21b&action=click
which originated in 2014 at the University of Texas. The plagiarism occurred in 2021 at a commencement speech at the University of South Carolina
https://www.cnn.com/2021/05/12/us/usc-president-bob-caslen-resigns-trnd/index.html
The second one is more obscure and involves the interim President of the University of Minnesota and his speech at North Dakota State University.
https://news.google.com/newspapers?nid=1908&dat=19880715&id=BjYrAAAAIBAJ&sjid=eNUEAAAAIBAJ&pg=2891%2c1186637
which, according to my memory, was lifted from an address to a Virginia audience and if memory serves me right, that was taken from some other speech by yet a different person; the interim president didn’t get the job once this came out.
The Chronicle of Higher Education of May 19, 2000 details a third example, the1999 cribbing done by the president of Wesley College who stole the 1991 paper of the president of Connecticut College. Each even had the same title, “In Pursuit of Global Civic Virtues.”
Recent posts and discussions have a lot to say about the practice of speechwriters cribbing from speeches which someone else had already given eg. https://statmodeling.stat.columbia.edu/2023/12/22/plagiarism-means-never-having-to-say-youre-clueless/ I would not call this “academic plagiarism” because speechwriting is a different world than academic research with different norms (eg. nobody expects an elected official to credit his speechwriter before every speech) If a university president sold a property which he did not technically own, we would not call that “academic fraud” even though the fraudster is an academic.
Sean make an interesting distinction regarding academic fraud—speech-writing vs. the written (publishable academic) word. In the three cases I cited, we know about them only because they were so laughably blatant. The “we” in the preceding sentence is me because I wondered how in the world did the perpetrators expect to get away with it. The events, unlike a publication in a journal that few if any follow, were presumably covered in the local media. As to university presidents employing speech writers, is the former group so busy that the latter group can pass along material without fear of consequences?
Recent news
https://www.cnn.com/videos/us/2023/12/29/university-wisconsin-la-crosse-joe-gow-fired-explicit-videos-wife-cnc-vpx.cnn
indicates that university chancellors may indeed be ultra busy.
I don’t think it is just spoken versus written! I think that some areas of life, such as vaguely inspiring speeches or business copywriting, have a norm of borrowing without credit and no serious penalties for doing so, while other areas like academic publications or journalism are strict about credit and occasionally punish people who borrow without acknowledging. I expect that academics borrow other people’s business writing or vaguely inspiring speeches as often as anyone else.
What I find confusing about both Bailey and Frank Landymore and Jon Christian’s articles is that it’s really unclear to me what exactly the AI “reporter” was supposed to be doing in the first place. Bailey seems to think it’s an example of generative AI memorising its training data, and uses it as a cautionary warning about using AI to generate content. But Landymore and Christian mention at one point that A.V. Club might just have been dressing up querying a database and returning verbatim results as an AI system. In other words, yes, it’s an example of AI hype, but not for the reason Bailey seems to think that it is: it’s just how people are calling a lot of things AI not only as cheap marketing device but also a way of avoiding culpability (if the AI “reporter” was never supposed to do more than return verbatim copy from filtered results of a database, then it’s the editorial staff who did the plagiarism, by not editing it properly).
I really don’t have any idea what we’re supposed to conclude from the story.
Joe:
Here’s how I saw the story, as an outsider to the whole process. I used to regularly read The Onion, and one thing I enjoyed was its movie and culture reviews (“The A/V Club”), which were readable, interesting, and, despite appearing as part of The Onion, non-satirical. If they’re returning database queries or otherwise presenting bland, characterless material, that’s not what I was expecting or what I was wanting to read.
I guess the analogy would be if someone were to replace this blogs by computer-generated essays on topics such as multilevel modeling, political polarization, Bayesian computing, etc. The blog would then add nothing to the discourse; indeed, it would pollute the discourse by making it harder for people to find the real stuff.
As a former Onion fan, I experienced this story as a (small) betrayal.
I used to read the AV club all the time — it had great reviews and one of the best commentariats the internet has ever seen. I haven’t read it in years — maybe 2015-ish? — when its quality went sharply downhill; shrill and full of click-bait. It had a lot of staff departures around that time, described at https://en.wikipedia.org/wiki/The_A.V._Club . That Wikipedia article reminded me of something somewhat relevant to the post, that one of their writers wrote a book review of a book that didn’t exist! (The writer was fired.) The next step will be AI-generated reviews of non-existent books movies, plus AI-generated comments.
Happy New Year!
Oh, I agree with you completely in that way. I just found Bailey’s take to be a bit strange if the system was “AI” (whatever that means) search and retrieval of IMBd’s massive database (“compile” is a bit vague, but it’s not incompatible with a verbatim return of relevant results of a search query)
Again, I’m just not sure whether the problem was with the tool or with the uses of the tool. In this case, it doesn’t make any difference, but there are larger implications depending upon if answer is the technology itself or the people who misuse it.
(I’ve sometimes spent a lot of time trying to find blog posts of yours that I vaguely remember so I don’t think an AI system that could query all the posts and effectively filter the results would necessarily pollute the discourse, although, again, I could see how it could be misused).
“How horrible. I remember when The Onion started. They were so funny and on point. And now . . . What’s the point of even having The Onion if it’s running plagiarized material?”
WTF???
This story is not even about The Onion! It’s about The AV Club, do you remember a time when that was good or a time when AI produced writing was good? The good people at The Onion and its contributors have nothing to do with this errant AI. Why tarnish their name??? The Onion is still producing great satire and none of it is written by a bot, at least for now.
The parent company, G/O media does have a problem with their new CEO Jim Spanf***er who has been hard at work torpedoing brand at the expense of his talented writers and you’re helping him to do it with posts like these. Idk why!