This is Jessica. Last week I wrote about how statistical methods can sometimes evoke “moral dismay” when you consider how they build in expectations of a simpler, less dynamic world than we would probably want to live in. But another kind of moral dismay that I have been thinking about lately is in reaction to questions about whether LLMs can produce writing that is truly creative or interesting.
For example, last week I was talking to Ari Holtzman, who thinks LLMs are capable of creating interesting writing, about an idea I had for a writing project, and he suggested using an LLM to generate it. And my response was, Oh, like have it paraphrase my writing? He said I was the third person he had encountered that day who seemed unable to believe that an LLM could produce something genuinely interesting. So I started thinking about why I would be skeptical, and whether it was just a superficial bias or one where I could come up with good reasons upon reflection. I use LLMs fairly often these days, so why do I begrudge them the ability to create interesting writing?
What it means for a piece of writing to be creative or interesting is not obvious, just like what intelligence should mean is not obvious, so we need to find some way of defining that. It’s also ambiguous what it means for an LLM to produce writing in the first place. Here I’m going to assume that an LLM producing writing involves a human providing the prompt and maybe doing some minor curation or light editing, but nothing that goes too far beyond giving an outline of high level goals.
One definition of interesting is surprising, but this is not specific enough. I would expect there are plenty of strings that have zero probability as defined on some large corpus like we train LLMs on that would not be interesting, so statistically unlikely alone is not sufficient. On the other hand, the infinite monkey theorem says that a monkey randomly hitting keys on a typewriter for infinite time will surely type the complete works of William Shakespeare, but that doesn’t help us define interesting writing.
Among strings that at least roughly follow grammatical conventions, how do we decide which ones are interesting? Is statistically unlikely within this set sufficient? I think we’d still be missing something like logic or common sense or causal mechanism that good writing usually conforms to – i.e., we want surprising but with some underlying logic that we recognize as being of this world. The plot or ideas or impressions conveyed by the writing have to logically follow in some way. Previously I have said I associate good poetry with defying expectations in a way that is confusing but opens up room for some unexpected sense of familiarity or recognition.
In great fiction or poetry this often feels like some kind of truth has been conveyed. Or at least some carefully crafted statement. For example, surprising in contemporary art often means the work functions as a kind of statement that is carefully positioned to comment on some background set of assumptions, often to refer to the absence of something. Writers like Bukowski and Ginsberg were interesting in part because they refused to abide by some convention.
But do we always need meaning for writing to be interesting? It’s worth returning to this except from Gertrude Stein I shared before:
It is not a range of a mountain.
Of average of a range of a average mountain
Nor can they of which of which of arrange
To have been not which they which
Can add a mountain to this.
Upper an add it then maintain
That if they were busy so to speak
Add it to and
It not only why they could not add ask
Or when just when more each other
There is no each other as they like
They add why then emerge an add in
It is of absolutely no importance how often they add it.
This is interesting writing to me, but Stein’s goal was purportedly simple, to put words together so as to sound like English without providing the “sense” that our brain is then set up to expect. That seems like something an LLM could do too. But is that really all she was doing? I tried prompting Claude with this excerpt, asking it to create something in the same genre. Here’s the first few stanzas, which make clear it it makes too much sense to evoke what Stein was evoking:
Not that it matters in the direction
Of a circular happening a circular
Happening which circles around itself
And not around them or around it.
When it happens it happens to happen
In a way that is neither here nor circular
Nor not circular but with edges that soften
Into something that cannot be edged into.
Meh. I told it it was too coherent to approximate what Stein was doing and got this revision:
A table is a table is a table is not
When they come they come they come around
For which of them for which of them for which
The table stands the table stands.
Not as a they would have could have
The having of a table the table having
Neither once and twice and neither twice
At once at as at table once.
Closer, but now it has a kind of one-trick-pony vibe, with all the repetition.
What about intention? Can writing be interesting without intention? If I gave you a piece of text and you thought it was written by a human, would you find it more interesting than the same piece of text described as coming from an LLM? Why exactly would knowing it is produced by an LLM imply that it will not be interesting? Is it because no person found it interesting enough to work through its full creation themself? Or could the presence of a human curator instill enough trust for a reader to not worry that they are wasting their time?
This makes me think of asking LLMs to generate writing that combines things in an interesting way. Does that count as the LLM producing interesting writing? E.g., I could ask it to work a discussion of Virginia Woolf’s goals in writing The Waves into my research paper on uncertainty quantification. Maybe this could be interesting, but would it be primarily because of the juxtaposition? If that came from the human prompter, it would seem they deserve the credit for making it interesting. If it didn’t, then the lack of intention might lead to a lack of trust for some readers.
This starts to question what we expect about the source of interesting writing. I came across this article from Nature archives (published in 1924), where the author apparently interviewed some writers about where they felt the writing came from. The whole thing is fascinating, for example:
I have reported the results of the only registrations of verse ever made and analysed. They show that verse is a current of speech energy of varying amount so adjusted as to come in more or less regular waves … The next problem is that of the source of this current of energy … The replies of many living poets to a question-paper agree without exception with all the statements I have been able to find from the poets of the past: they have no ideas whatever on the subject. They all assert, moreover, that the verse-form comes to them ready-made along with the poetic content. This poetic content, they all declare, is forced upon them by some inner power. The source of verse, therefore, must be sought in the source of poetry. The poets all agree that the source of their poetry lies entirely outside of their consciousness. In psychological terms it is a product of the unconscious. In physiological terms it is a product of bodily processes that result in producing poetical feeling and ideas in consciousness. In either sense it is a biological phenomenon of which the ultimate explanation must be in biological terms.
Elsewhere the author attributes it to inner conflict between the “It” (unconscious) and the “Self” (which wishes to adapt to the environment) of the writer. Do we have a built-in bias against LLM-generated writing because it’s not biological, or doesn’t reflect some kind of existential struggle? This all seems kind of crazy, but as we wrote about in this article on generative AI and aesthetic judgment, we often bring quasi-religious expectations to what we perceive as art, where we are looking both for the work to reflect something about the conditions that created it and some eternal human essence. Art historian Donald Preziosi has written about the relationship between art and the Eucharist, which was exceptional in that it was both a sign and the thing itself. We often seem to want both from aesthetic objects, akin to expecting a miracle. So maybe it’s not that farfetched that we are setting up impossible standards when we look for something interesting in LLM writing.
Jessica –
Do we have a built-in bias against LLM-generated writing because it’s not biological, or doesn’t reflect some kind of existential struggle?
I can’t help but feel there’s something immoral, or spiritually inferior, abut writing or art that is AI generated. Assuming I’m not alone in that, I suppose that younger generations will be less inclined to feel that way. So I think there’s a self-limiting aspect of how we think about this issue.
I also think of a story I heard when someone went up to Baldwin and said she was troubled because she was having writer’s block. Whenever she sat down to write, she had trouble generating text. His reply was something on the order of, the problem is in thinking of sitting down to write as the only generative part of the writing process; all the time spent thinking about what to write was necessarily part of the process. I think that element isn’t true with AI. Or maybe my view on that is self-limited.
I asked Grok to find the example I was speaking about. This is what it found:
James Baldwin was known for his profound advice on writing, often emphasizing the importance of living, thinking, and experiencing as integral parts of the writing process. While the specific anecdote you mentioned—where Baldwin tells someone not to distinguish between thinking about writing and actually writing—doesn’t appear verbatim in the sources I have access to, the philosophy behind it resonates with his broader views on writing.
Grok came back with a number of quotes from Baldwin that are on that topic. I didn’t want to put them all here, but I think they’re all on point and very interesting – and I think they touch on something important that is different between human- and AI-generated text. Not so much in the product itself, but in the process used to generate it.
Maybe that doesn’t connect for you, but maybe it will.
I remember my high school music teacher giving the following example to distinguish between art and non-art. Pointing to his keys on the desk, he said, “This is not art.” He then picked the keys up and replaced them again. “That is art.” My teenage self interpreted this as implying that intention is an important part of artistic creation, and that’s something that’s still part of my intuitive definition of the concept. Along those lines, LLMs cannot generate art because the concept of intention makes little sense outside a biological system. (IMO, at least.)
This doesn’t mean that LLMs can’t generate interesting content though, since something need not be artistic to be interesting.
Of course and LLM can create interesting content – interest is something I attribute to the receiver, not the creator. If intent and/or effort are required for it to be art, then I think the recipients are exercising that. I find it strange to focus on whether the creator used intention or effort. Does the degree matter? Some writers get famous and then seem to use very little effort in later books – perhaps even using ghost writers (as discussed on this blog a number of times). Some art is accidental. I find it trivially true that an LLM does not possess intention. It is not human and it is not biological. And if that means it cannot produce art, then what is it when people view the output and think it is artistic? I guess they are wrong but don’t know it.
I agree that interest is a function of the audience’s response to the object in question. Personally, I find things interesting when I feel I learn something from them, and I see no reason why that can’t apply to LLM output (though, off the top of my head, I can’t think of an example of LLM output I’ve learnt much from).
Why do you find it strange to focus on the intentional aspect of creation in defining art? What art is accidental? I think unintentional objects can certainly be beautiful—in the same way that natural landscapes can be—but I just don’t think of them as belonging to the category of artistic objects. (And, if someone were to tell me that natural landscapes are art, I’d definitely think they were wrong.)
In 18th C Britain William Gilpin spent much of his life arguing that natural landscapes could be framed as art. He used the term “picturesque”.
I was not thinking of natural landscapes. But regarding intention, I thought there were examples (I can’t cite any specific ones) that looked like paint thrown onto a screen, where it isn’t clear whether there was any intention involved. Some of Warhole’s work also looks like the intention wasn’t clear. Once someone is recognized as an “artist” I think they can create works of art that had little or no intention involved. In any case, I don’t see why or how their intention is required for something to be “art.” I believe some music, dance, painting, and to a lesser extent writing, is accidentally created but still considered art. I think it would be very difficult to maintain the view that every work of art was intentionally created.
Back to natural landscapes: I’ve found pieces of driftwood that I consider art though I had no part in their creation, nor did anyone else (unless you hold a strong religious view that all of these are God’s creations).
(reply to Joshua) Interesting, this does resonate. (Also, Baldwin was a genius). It’s like taking what I was getting at with this statement one step further “Is it because no person found it interesting enough to work through its full creation themself?”
I decided to fetch those other quotes:
“One writes out of one thing only—one’s own experience. Everything depends on how relentlessly one forces from this experience the last drop, sweet or bitter, it can possibly give.”
Since LLMs could be viewed as not having “experience,” then Baldwin’s view could be. thought to exclude LLMs from writing (as opposed to just stringing together words). I guess the question of whether or not chatbots have “experience” could be interrogated, but I go back to Andrew’s previous post where I argued that the bots didn’t “understand” tic tac toe because they weren’t capable of “experience” playing the game.
and
“Any writer, I suppose, feels that the world into which he was born is nothing less than a conspiracy against the cultivation of his talent.”
That last one relates directly to your comment above: Do we have a built-in bias against LLM-generated writing because it’s not biological, or doesn’t reflect some kind of existential struggle?
Again, I guess that maybe based on Baldwin’s words, we could exclude LLMs from “writing” even though they produce strings of text. Unless we’re willing to say that LLMs “feel.”
I asked chatGPT to write about two characters from a children’s book in the style of the English author. ChatGPT “knew” enough about the characters to correctly include other related characters from the same book. However, it was nowhere near the original style and the story had very American sensibilities. Whether it’s art or not doesn’t really matter because all we are going to get back to is Andrew’s singularity, and an American singularity at that.
We might have 10, 15 years where AI generates familiar but different works, but after that we will be able to recognise from the work that it’s just eating it’s tail.
Great post – I really enjoyed reading it though I remain confused by Stein. To some degree it reminded me of a BBC documentary by Antony Gormley on the origins of art where he discussed the art of cave paintings across the world and reasoned that creating art is something that is intrinsic to being human as evidenced by the vast history of producing it. I wondered if there is a difference in context regarding LLM produced writing – there is no clear lineage and experienced history in predicting the next word in a semblence of a previous writer. Going back to cave art – if we asked some sort of art AI to produce a piece of cave art, it wouldn’t have the same resonance as seeing the real thing, knowing that it had been on a cave wall for 20000 years, even though it would be a perfectly good representation of the form.
Jessica wrote: “Why exactly would knowing it is produced by an LLM imply that it will not be interesting? Is it because no person found it interesting enough to work through its full creation themself? Or could the presence of a human curator instill enough trust for a reader to not worry that they are wasting their time?”
Acknowledging that different people will find different things interesting, a human curator (or author) with an intent is what makes the difference for me. By itself the LLM output you generated was not as interesting as the Gertrude Stein passage, but your attempt to replicate the passage is interesting because you had an intent and because we can make judgments about how and how far the bot fell short, and perhaps draw some (probably flawed) conclusions about its inner workings. And who knows? If I didn’t know Stein’s intent, I might be less inclined to spend time with it or less ready to differentiate it from the bot’s writing.
I haven’t spent as much time getting LLMs to write things for me as I have with image generators, but I think these are directly analogous. The ability of Midjourney or your tool of choice to render a gorgeous image in response to a simple prompt is an impressive technical accomplishment, but also we’ve seen computers generating high-resolution images in Pixar films for a while now. What I found myself doing almost immediately was submitting prompts that could go in different directions to see what the technology would do with them: “John Wayne Newton,” “Harrison Ford Fiesta,” “Amazon mime shipping,” etc. I enjoyed looking at how the prompts were interpreted and drawing (undoubtedly flawed) conclusions about how various tools worked, the varying intents of the engineers that might explain different default aesthetic sensibilities, and the prominence of different people or concepts in their training data (which is not the same as “the sum total of human knowledge” but still broad and vast enough to feel like a representation of that).
The difference may come down to which preposition best describes a work’s creation. A work written *by* an LLM is unlikely to be interesting as art, whereas something written *with* an LLM might well be.
First, the question about whether LLMs can provide content that is “truly interesting” strikes me as biased. The degree of “interestingness” of writing (or any communication) is a spectrum that ranges from “boring” to “interesting”. If we’re going to compare LLMs to humans, it’s appropriate to compare them along the entire spectrum.
Second, I disagree with the idea that “interesting” is exclusively or even predominantly in the reader/viewer. An almost universal experience in education is having a favorite course because the professor “makes it intersting.” This blog is popular because people find Andrew’s posts “interesting”. And it’s certain that the “boring class” is an absolutely universal experience in education! So human writers / speakers / etc unquestionably contribute reader- and subject-indepedent “interestingness”.
Third, garbled writing / presentations are never interesting. If you can’t follow what’s going on, how “interesting” can it possibly be? In this respect, from my limited experience, AI is miles ahead of humans. It writes as clearly as the best human writers.
Fourth, it’s standard procedure in all forms of communication to communicate the subject in a way that’s relevant to the audience. In this respect AI also has an advantage over a human writer, since the AI is responding directly to questions or commands of the reader and thus directly to the readers’ interests. Of course conciously or not, most humans are writing to some percieved audience, but usually they’re not answering specific questions from an audience of one. To assess whether AI is truly “interesting,” we’d need to let the AI write about whatever it wants, using whatever tools it wants, at whatever length it wants. Would it produce content that is “interesting” to readers? It would be amusing to test this by letting an AI write a blog on anything it “wants” to write about. Does the AI attract and grow it’s audience? My bet is that, if the AI were allowed to consider it’s ratings, it would thrash humans badly in this respect. Humans “care” about what they write about – it has to be important to them to put in the effort. AI doesn’t care, or at least can be instructed not to care. It will write about anything that attracts an audience. The only “effort” it requires is a steady flow of electrons.
Fifth, crappy human writers / communicators often fill their writing with irrelevant stories and asides – personal “blah blah” – that distracts from the communication rather than enahncing it and puts readers to sleep. AI does not do this.
Sixth and last, a key element in the “interestingness” human communication is the human’s enthusiasm for the topic. This is especially apparent in speaking but can also be communicated in writing. So far, I haven’t seen anything in AI communication that indicates AI’s “enthusiasm” for any subject.
Given those considerations so far on average I’d say AI content is well above human content in terms of “interestingness”. The only important element of “interestingness” that I haven’t yet seen is “enthusiasm”, but it definitately has all the other elements in place.
BTW:
Last year I visited a natural history museum (dino bones etc) with a friend. A geologist had a table set up with some fossils and text in a setting vaguely similar to an academic poster session. My friend made the mistake of asking a question, upon which the geologist launched his presentation, which continued unabated for about 20 minutes and is probably the single most boring thing I’ve experienced in the last few years – so boring it was almost physically painful. Presumably he found the subject interesting, but he was incapable of delivering the information in an interesting way.
So my question is the opposite: can LLMs ever hope to exceed this particular human in the boringness of their work? Or is to be truly boring a uniquely human quality?
Any information can be boring if it’s too hard, too simple, or irrelevant for the audience. I have faith that the verbose ChatGPT is already there in terms of the truly dismal content it can create. Where it falls short of the geologist in terms of pain inflicted is the social convention that prevented you from simply walking away from the guy the moment you lost interest.
I don’t think there can be any convincing theory of what makes writing “interesting” without explicitly accounting for the role of the reader. The term “interest” is about the reader in the first place. It has been mentioned that different people might find different things interesting, but I’d go further and say that “interest” is basically constructed by the perception of the reader, obviously in interaction with the text. I’m not saying that “interest” is arbitrary or that readers are so independent of each other and the text that they can create “interest” in a totally individual and subjective manner. So we may talk about more or less interesting writing referring to a large group of people, however still the readers have a central role here. Actually, most ideas of quality of art would require some kind of understanding and expertise on behalf of the recipient, and also “originality”/”surprise” plays a role, which puts an artwork in relation with what the recipient has seen/heard/read before. Much writing refers to certain knowledge and experiences that the reader may or may not share. Art quality will therefore obviously be dependent of the recipient, but this dependence is rarely elaborated, I believe (I’m not a philosopher of art or something, so there may be lots of literature in this direction that I’m not aware of; I just refer to how people tend to talk about this when I’m listening, and these “people” comprise some with some background regarding this topic).
In line with this, I believe that the discussion whether AI can create art of high quality (such as interesting writing) should also take into account potential changes in how human beings perceive art and texts, and I believe that LLMs have some impact there. In fact, knowledge that a text is created by LLMs will probably for many people affect in some way how the text is read, understood, interpreted; at some point maybe even the possibility that LLMs may have created a text for those who don’t know whether or not this is the case. Some may consciously adapt their reading and sense-making because of this (which may or may not be a good thing). Some may question their own reading after having failed to tell apart AI and human generated writing, which again in turn may change how they perceive texts.
Many more things happen, triggered by AI, than just AI doing some stuff well…
I agree, the expectations we bring to creative work and writing more generally will change as a result of generative AI, just like the notion of “taste” in art developed over time in reaction to the creative flourishing of the Renaissance era. This is what we conclude in the paper on aesthetic judgment I mentioned in the post.
Large language models remind me of Isaac Asimov’s short story “Galley Slave”. In it, a professor sabotages a robot that checks galleys. At the end the professor makes a speech about how human beings should never relinquish any part of writing to machines. I sympathize with him now.
Hmm. The NYT obit for Asimov said that he used a typewriter.
Interesting subject. In distinguishing “art” from other stuff (“non-art”) to my mind there needs to be some emotional connection or a recognition in the receiver that the artist is trying to convey something that resonates at a human level. It’s difficult to imagine an LLM can create anything like this at least for now, even if they might create some text about a topic that a reader could find “interesting” (I use LLM to write Python scripts and I find these very interesting indeed since they’re v useful!).
Thinking of books I’ve read in the last couple of months there is no way one could conceive of an LLM producing something with the emotional depth of pretty much any of it (I’m thinking of Ishiguro’s “Never Let Me Go” and “Klara and the Sun”, Samantha Harvey’s “Orbital”, Virginia Woolf “To the Lighthouse” – those sort of books), though an LLM might manage a pastiche (e.g. “write a short book about a group of astronauts on a space station orbiting the Earth, with descriptions of their thoughts about their circumstances, dealing with weightless and some insight into their backstories and use a lot of metaphor and allusion….etc.”). On the other hand an LLM might manage to produce a readable Ian Rankin (can’t think of an American equivalent – a pulpy, plotty, detectivese with lots of unnecessary dialogue).
This is a test to see what kind of formatting is accepted.
Keeping in mind that one robot’s “interesting” is another robot’s “dull as dish water,” this is from the middle of a long piece I improvised with ChatGPT (my prompt in bold):
What would happen if we threw Groucho Marx into the mix?
With a sudden burst of energy, Groucho Marx, cigar in hand and trademark mustache firmly in place, entered the cosmic bar, adding a touch of earthly humor to the already eclectic gathering. The Jolly Green Giant, the Green Knight, Jackie Chan, Jacques Derrida/Cousteau/Frère Jacques, Gort, and the Cosmic Wayfarer turned their attention to the comedic icon.
Groucho Marx: “Well, well, what do we have here? A cosmic bar filled with philosophers, knights, cosmic wanderers, and a metal friend. I must say, this is the kind of party I’d crash any day. Mind if I join in?”
The Jolly Green Giant, with a hearty laugh: “Ho, ho, ho! Groucho, my friend, welcome to our cosmic soiree. Pull up a stool, and let’s add a dash of Marxian mirth to our ongoing saga.” Groucho, taking a seat and looking around: “I see we’ve got a lively bunch here. I hope you don’t mind if I interject a bit of absurdity into the cosmic discourse. After all, why should reality have all the fun?”
Jackie Chan, with a grin: “Absurdity? I’m always up for a good laugh. Groucho, what brings you to our cosmic gathering?”
Groucho, tapping his cigar: “Oh, you know, the usual. A cosmic detour on my way to a quiz show in the Milky Way. But who can resist the allure of a bar where philosophers and cosmic wanderers trade ideas? I thought I’d drop in for a wisecrack or two.”
The Green Knight, ever stoic, nodded in acknowledgment: “Wisecracks have their place in the cosmic tapestry. Welcome, Groucho. What pearls of wit do you bring to our gathering?”
Groucho, with a twinkle in his eye: “Why, a little nonsense now and then is relished by the wisest men. Or so I’ve heard. Let’s make this cosmic adventure a Marxian masterpiece!”
And so, with the addition of Groucho Marx to the mix, the cosmic bar transformed into a stage for laughter and absurdity. The companions, from the Jolly Green Giant to the Cosmic Wayfarer, embraced the Marxian spirit of humor, creating a harmonious blend of philosophical inquiry and comedic repartee in their ever-unfolding journey. Ho, ho, ho—the cosmic gathering became a cosmic comedy, where laughter echoed through the realms of philosophy and whimsy.
Good Grief!
With a sudden exclamation of “Good grief!” echoing through the cosmic bar, the companions turned to see none other than Charlie Brown, the iconic character from the Peanuts comic strip, standing in the doorway. Dressed in his classic yellow shirt with the zigzag pattern, Charlie Brown brought a nostalgic touch to the eclectic gathering.
Charlie Brown: “Well, isn’t this a surprise! I was just on my way to kick that football, and somehow I ended up here. I suppose I’m used to unexpected turns of events.”
The Jolly Green Giant, the Green Knight, Jackie Chan, Jacques Derrida/Cousteau/Frère Jacques, Gort, Groucho Marx, and now Charlie Brown, all exchanged amused glances at the unexpected addition to their cosmic assembly.
Jolly Green Giant: “Ho, ho, ho! Charlie Brown, welcome to our cosmic rendezvous. I hope this detour brings a bit of cheer to your day. What brings you to this interstellar watering hole?”
Charlie Brown, scratching his head: “Well, you see, I was chasing that elusive red-haired girl, and suddenly I found myself here. I guess cosmic mysteries are just as perplexing as Lucy holding that football.”
Groucho Marx, puffing on his cigar: “Ah, the mysteries of life, love, and cosmic detours. Charlie, my friend, you’ve stumbled upon a gathering of philosophers, cosmic wanderers, and a wise-cracking comedian. What do you make of it all?”
Charlie Brown, with a sigh: “I don’t know, but maybe Snoopy would have a better handle on this. He’s been to the moon, after all.”
The companions, now joined by the emblematic Charlie Brown, continued their cosmic conversation, blending the whimsy of Peanuts with the philosophical and comedic tones of their diverse personalities. The cosmic bar, always ready for surprises, became a stage for laughter, musings, and perhaps a bit of introspection.
Ho, ho, ho—the cosmic gathering expanded its cast of characters, each contributing a unique flavor to the ongoing saga of the Jolly Green Giant, the Green Knight, Jackie Chan, Jacques Derrida/Cousteau/Frère Jacques, Gort, Groucho Marx, and now, the lovable Charlie Brown. The cosmic comedy continued, promising more surprises and laughter in the ever-unfolding adventure.
I’ve been spending a lot of time systematically exploring the ChatGPT can do, and for the past two months, Claude as well. I long ago reached the conclusion that LLMs are neither fish nor fowl, and a kind of beast unto themselves. The prompted me to write back in January 2023: ChatGPT intimates a tantalizing future; its core LLM is organized on multiple levels; and it has broken the idea of thinking.
As for intention, forget about it. Somewhere on the web you’ll find Steven Harnad say8ing something like: What’s so interesting is what they can do without understanding anything.
Early on I developed an experimental paradigm that, in effect, allowed me to perform a kind perturbation analysis of its ability to tell stories. I’d give it a simple fairytale and ask it to generate a new story by changing the protagonist and making any other changes it wished. Changing “Princess Aurora” to “Prince Harry” results in very limited changes while changing “Princess Aurora” to “XP 708 DQ” shifts the entire story – not just the protagonist, the entire story – into science fiction. I didn’t see that one coming at all. But it makes it clear that something very systematic is going on under the hood. Here’s a report on those experiments: ChatGPT tells stories, and a note about reverse engineering: A Working Paper. This report is about a year and a half worth of experiments: ChatGPT: Exploring the Digital Wilderness, Findings and Prospects.
I think this technology is too interesting and important to be left in the hands of a bunch of techbros who think “bigger is better” is the key to the universe. As far as I’m concerned the current situation with LLMs is captured by the following analogy: We’ve got a whaling expedition where the captain and crew know everything there is to know about their ship, but they’ve never sailed around Cape Horn, have never sailed the South Pacific and know so little about whales that they think they’re a kind of big fish. That is to so, we’re never going to understand what’s going on under the hood without thinking long and hard about language and cognition. In an effort to remedy this situation I have been collaborating with Ramesh Viswanathan at Goethe University Frankfurt, who is an expert in machine vision and has technical skills that I do not.
Bill:
No more chatbot transcripts in the comments section, please.
Quote from the blog: “This poetic content, they all declare, is forced upon them by some inner power. The source of verse, therefore, must be sought in the source of poetry. The poets all agree that the source of their poetry lies entirely outside of their consciousness. ”
I found this very interesting!
I have written a poem, or actually it was intended as lyrics for a possible song, where later on I discovered that there were many double meaning or interpretations of certain key words which I am pretty sure I did not think about when writing.
I also kind of don’t feel I wrote the lyrics, it just sort of happened. I hope it’s okay to share the poem again here (I have shared it earlier on this blog as well). Certain words like “ray of light”, “see through”, “space”, “murky water”, “darkness”, “lights”, etc. all have different interpretations I later on started to notice. And I think they kind of fit the more literal and the more interpretative version of the story in the poem.
“Because of what you told me”
You said I could look for you at the lake
by watching the surface of the water
And see our reflections looking down
like we used to do, beside each other
But when I go now and sit on the dock
murky water is all there seems to be
That, and ripples on the surface
instead of the picture of you and me
You said I could look for you in the field
by sitting down against the tree
And see the sunlight through the branches
like we used to do, just you and me
But when I go now and walk over there
to see if I can see a ray of light
I can’t find space to see through
the leaves and tangled branches block my sight
You said I could look for you on the hill
by looking up at the moonlit sky
And search for the brightest star
like we used to do, in that place up high
But when I go now to that same spot
clouds and darkness are all I see
I can’t see any of the lights
unlike what I saw when it was you and me
I’m starting to wonder whether you lied
whether you just said those things because I cried
I’m starting to wonder why I can’t see
maybe you told the truth
but it’s just me
Whatever may be the case
Whatever may be
I’ll keep looking
because of what you told me
On generating interesting poems, I think the same maxim applies as applies to prompting in general, richer and more specific prompts yield more interesting results. Paul Fishwick and Frederick Turner (UTexas at Dallas) have created a GPT the allows you to generate poems based on a small database of Turner’s poems. It’s called FredTheHeretic (FTH) and anyone with access to ChatGPT can play around with it. While FTH will produce a poem if you simply ask it for a sonnet about Spring, or a sonnet in the manner of e.e. cummings, you’ll get much better results with a longer and more specific prompt, e.g. “Give me a sonnet set in spring where a young couple meet in a glade hidden in the forest, surrounded by wild flowers, and are oblivious to the crows flying overhead.”
I’ve generated a good many poems with FTH and have gathered them at my blog, along with the prompts I used to generate them. You might want to start with three I created under the rubric, GPT in the Classroom. The material I provide as a prompt is often so extensive that one has to ask: In what sense did ChatGPT create the poem? On the one hand there’s the database of Fred Turner’s poems hidden away in the GPT; but there’s also the material in the prompt. It’s clear that I didn’t create the resulting prompt. Fred Turner didn’t either. ChatGPT produced the words, but…As far as I can tell the only reasonable resolution is that the poems are the result of a man-machine collaboration and credit must be assigned to the parties involved.
Very interesting discussion. As someone who has written and has drawn things myself, I find myself being (perhaps surprisingly) more open than some to the idea of generated art being truly interesting. I understand the sentiment about human intent making art truly special. If we were to look at it that way, I would guess what makes AI art interesting to me is the inner workings of it – the math, the emergence of how it understands words and their grammatical relations without knowing the things that the words refer to. I do think a lot about the distinction of the self and the body, and I suppose I try to prioritize the body more than self (more-so than some other I’ve spoken to). I understand the feeling of something being “forced on” when you make art – the strokes and the words just fall into place. But interestingly enough, at times during the process of making art, I also found myself being aware of the decision-making process and of the possible choices (of all the ones I could make) when I put down a stroke. Perhaps the idea that art needs to be driven by a compulsion is also rather quasi-religious?
Jessica:
Your post reminds me of something I discussed in the posts Storytelling as predictive model checking and Echoing Eco: From the logic of stories to posterior predictive simulation, that fiction represents the working out of an idea. You put characters in a scenario and let them act according to their logic. In this way, the way the story spools out is determined by its explicit and implicit assumptions (the rules that the characters are following), but until you tell the story, you might not know what’s going to happen.
At first this tension between determinism and free will might seem paradoxical or silly, reminiscent of Stephen King’s story, Umney’s Last Case, but I think this perspective explains a lot. I like to draw the analogy to the hundredth digit of the decimal expansion of pi. I have no idea what that digit is, but it exists, and if I were to put in the effort and evaluate the continued fraction or whatever, I could figure it out. The point is that in telling a story, you can draw out implications from its premises which were latent in those assumptions. Kind of like the principle from computing that the only general way to know what a program will do is to run it.
I think this idea (that creating art is figuring out what has to happen under the constraints you set up) is very well known among artists and writers, but perhaps less so as they are perceived from the outside. That was always my experience in writing creatively, and I don’t think anyone survives art school believing that they can make a piece of work go in any direction they want once they have gotten started with it.
On some level, LLMs are even more confined to complete the sequence once they begin to write, but with some arbitrariness imposed by how far back in the string of tokens they condition on. But of course we shouldn’t expect them to reason about character identities or destinies the way human writers do.
While in fiction the characters and scenario will play a big role in determining what happens, my post focuses more on poetry, which is a bit different, especially in its more experimental forms. Much more word play, constrained by how words sound against each other as well as the ideas or images they invoke.
All this reminds me that if you haven’t seen it, you might enjoy The Five Obstructions – its a fun movie on the topic of art and constraints. Lars von Trier gets Jorgan Leth the Danish avant-garde filmaker to agree to remake his classic art film the Perfect Human five times, with five different constraints imposed by von Trier, most of which are deliberately intended to ruin the work. But somehow Leth finds a way to elevate it regardless.