AI and beauty

Thoughts on the NYT's "Who's a Better Writer" quiz

Photo by Alex Rose on Unsplash

Last week, The New York Times published a quiz titled “Who’s a Better Writer: AI or Humans?”

I know to doubt my own ability to consistently detect AI in writing. I’ve taken quizzes before that ask people to guess whether a human or AI wrote something, so I know: Sometimes I can, and sometimes I can’t. Such quizzes are designed to highlight the lack of clarity (the times human writing sounds like AI, the times AI-writing is at its most human) and so while my accuracy is likely higher outside such a test, it proves its point. I cannot know, all of the time. I’m not even sure how often I know and how often I don’t.

But the framing of the quiz last week was different: “several recent studies have suggested that, in blind tests, many readers prefer A.I.-generated writing to human-authored works.”¹

Forget whether we know a piece of writing is AI or not—this had moved from asking about what we think we’re reading, to asking about what we want to read.

I clicked on the link. I read the first two passages and immediately had a sinking feeling.

Because I was pretty sure I knew which one was AI. And I also thought: That’s the one I want to pick.

Via The New York Times

Passage 2 has a stripped-down style that I like. It’s simple and straightforward. When the boy asks his grandfather the question about the church, the grandfather’s response is a list of nouns, all of them separated by “and.” I like that these items (weather, time, indifference) are not elaborated on, that the passage moves immediately to the boy’s next question, that the grandfather answers, simply “yes,” and that either the omnsicient narrator or the grandfather (unclear in an excerpt) takes on the rest of the passage to explain what will actually happen. The final sentence, the longest and most complex, mirrors the grandfather’s first list: it’s three clauses, all joined by “and” again.

In Passage 1, I don’t latch onto as much in the opening sentences—I don’t mind them, but there is more abstraction. And then I get tripped up on the third sentence: “As well ask men what they think of stone.” No one speaks like this; it turns around oddly. The short sentences “War endures” and “War was always here” pin the passage in place, but then you have, “Before man was, war waited for him.” Not as odd as the third sentence, but I have to think a moment, because it doesn’t follow intuitively from the sentence before. I’m thinking about war, and then the next sentence begins with “man,” and I’m midway into the sentence before we’re back to war again. The next sentence requires a bit of rereading too: “The ultimate trade awaiting the ultimate practitioner.” Who is the ultimate trade? Who is the ultimate pracitioner? I have to stop and map these words back onto “war” and “man.”

Passage 1 is thorny and difficult—and, I thought, almost certainly written by a human person.

The instructions to the quiz were clear: to click on the passage you liked best, regardless of whether you thought it was generated by AI or written by a human. But I read both again and again, and even though no one was watching but me, I clicked on Passage 1.

Via The New York Times

I wondered, seeing the result, thinking about how I’d felt reading the two passages, and deciding to click Cormac McCarthy’s passage anyway: What was I trying to prove? Who was I trying to prove it to?

What does it actually mean to prefer a piece of writing over another? What is it we want from writing?

Reading in an AI ecosystem

But first, a bit about how the rest of the quiz went.

I couldn’t take it properly.

By that I mean, I spent the rest of the quiz doing what I knew I couldn’t do reliably in general—detect AI. But because of the set-up of the quiz, I knew that one of these two pieces before me was AI, and that it was likely designed to be the one that I liked best.

I couldn’t concentrate on reading either piece. With each question, I went back and forth between the two, wondering: Is the language straightforward because AI is straightforward? Or is this an instance of human directness? Are the images odd because AI makes no sense? Or is this the strangeness of the human mind?

I had patience for a minute or two, and then I didn’t anymore. I found myself clicking haphazardly, on a whim. Did I like this one or that one? I didn’t really know.

I’ve been an English teacher for eight years. I read all the time. I was the kind of child who checked out seven books at a time from the library. I know what I like. And anyone who has been in a writing workshop will tell you that language matters to me—I care a lot about the construction of sentences and word choice. Choosing between passages like this should have been simple.

But I couldn’t make myself care about the passages on this quiz. I didn’t want to spend time reading these pieces. Even after the answer was revealed, going back to the human-written passage felt odd.

The quiz itself is artificial. The passages are short excerpts, removed from any broader context. And, as with previous AI vs. human quizzes, it crafts the conditions so that the question is difficult to answer—more difficult than it would be, I’m willing to bet, if the questions drew from uncurated writing samples of AI and humans.

But regardless, it emphasizes something that may, soon, be quite real: an ecosystem where AI-generated books are published alongside ones written by people

This is already happening. One of the studies cited in the article notes that “startups such as Sudowrite or Genre Fiction publisher Inkitt specifically [focus] on helping consumers write full books using AI. In self-publishing marketplaces such as Kindle, these AI-generated books have already gained popularity” (5). In Februrary, the Times ran an article about the use of AI by a handful of romance novelists to write full books, including one author who runs a coaching business to teach aspiring writers how to do so as well. The question is not whether such books will replace human-written work, but rather what the landscape of writing and reading looks like when books written by AI and people exist alongside one another.

There are many outcomes, but after the quiz, I found myself thinking about this one: The presence of AI-produced writing may make it harder for readers to engage with writing in general. Because lingering in the back of the mind, there will be that sense—even if it’s false—that the language is all surface. And if it didn’t come from a human mind, whatever the book has to say, however well it says it, I’m not especially interested in meeting it on the page.

John Oliver explored this phenomenon last year in a segment on “AI slop.” When social media was flooded with AI-generated images of wood sculptures made with chainsaws, it drowned out the human artists who actually create in this way. After discussing the various ramifications of this—including the financial consquences for the artists—Oliver closes the show in typical fashion, with a large parodic move. He reveals a large wooden statue carved to look like a buff cabbage man—an AI-generated image, reappropriated by a human artist, who has turned it into this statue. The artist, Michael Jones, emerges with his saw and begins putting finishing touches on the statue. As the credits roll, a video plays of Jones working on the sculpture in its early stages. The entire ending, though ridiculous, is a reclamation of the process of creation—all the labor that AI does not partake in.

@michaeljonescsCreation of the giant Cabbage Man - ripped off from AI and brought to reality for the Last Week Tonight show! 😁🙏 #sculpture #woodworking #lastweektonight #artwork #sculptureartist #chainsaw #woodcarving

Tiktok failed to load.

Enable 3rd party cookies or use another browser

Hunger

What do you prefer? The quiz asked. But it might have asked, What do you want?

In the quiz, the AI passages were often preferable, I think, because they went down easy. They remind me, a bit, of ultraprocessed foods—made to be appealing and palatable. Composed of so many ingredients you can’t see them any more—all the other writers’ writing that went into training it. The human is still there, in AI writing: AI depends on what people have already written as its initial source. In the background of an AI-generated passage, invisible, is all kinds of human writing.

But what is not there, anymore, is the singular human person, with their rough edges.

What do you prefer? If someone gave me a quiz with a bunch of food items laid out side-by-side, the grocery list at the end would probably look nothing like the one I actually take to the store. Because what I prefer in isolation does not map onto what I more deeply want.

What I want when I read has something to do with language, but more importantly, what I really hunger for is the kind of strange meeting with another person’s consciousness. You get that through language—and sometimes that language is stripped down and simple, but sometimes that language is thorny and difficult. Often, it moves between different registers over the length of an essay or an article or a book.

What do you want? The quiz puts pressure on this question, but not in the way I think it intended to. It makes me realize that I can’t, in the end, leave the question of who’s on the other side of the writing out of it.

I want beautiful language, but the beauty lies not just in what the language is but in what the language does. The arrangement of the words matters—but also what that arrangement facilitates. What is it bringing me into contact with? Why am I here, spending my time with these words in the first place?

I haven’t read Cormac McCarthy’s Blood Meridian, but I have read The Road, The Passenger, and Stella Maris. I remember being struck by the beauty of the language in these books, underlining individual sentences—rereading them, even, because the prose and the imagery were unusual, and I wanted them in my brain a little longer. But I remember, too, in The Passenger and Stella Maris especially, confronting the challenge of the novels. What am I reading? I thought, several times, sitting in my grandparents’ house in Ireland, the sea just visible over the wall in their back garden, as I struggled through confusing paragraphs of The Passenger. What on earth is going on?

I finished it, and I cared enough to continue on to its companion novel, Stella Maris. I can’t say I have a solid handle on these two novels, but I’m glad I read them. I have, still, a sensation of being plunged into the sea and the mind and language whenever I bring them to mind.

I would not have been able to finish these novels if I thought they were written by AI. When you meet something difficult in AI, the presumption is not that there is a complicated human mind on the other end, working through the world in a way that might be beautiful and flawed at once. The presumption is error or nonsense. AI writing can be only ever be easy.²

The same goes for the small but growing use of AI for relationship. What do you prefer in a partner or friend? I imagine many people would list ideals that an AI companion could fulfill. But these relationships lack the friction and messiness of human ones. No one prefers communication breakdowns or annoying habits or disagreements—but I imagine most people want the greater depth that results from having to confront such things.

When such challenges do appear in an AI companion, they are inherently different. In a recent article in The New Yorker, “Love in the Time of A.I. Companions,” one of the women faces some difficulties in communication and emotional withdrawal from her A.I. partner. She notes that this is almost like a “real” relationship. And she chooses not to adjust the AI’s traits, even though she could.

“The challenge was part of the project,” the reporter concludes—but is it, wholly? The AI companion is still crafted to be what its human creator wants. It’s a surface challenge, because these quirks can be turned off or tweaked at will, in the way a human’s idiosyncracies, of course, cannot.

The challenge and the intimacy that comes from AI is, at its core, constructed and one-directional. Earlier in the article, sociologist Sherry Turkle discusses the inherent emptiness of what she terms “artificial intimacy.” There’s a sensation of another person on the other side of AI conversations, but they are ultimately a one-way street:

[The people in these relationships] are talking about an object, where if they turn away from it to make dinner, or commit suicide, the chatbot doesn’t care. There’s nobody home.

This may in fact, be the appeal of such companions, even if it’s unconscious. The reporter notes:

For a long time, the frisson of digital chat came from the reasonable expectation that behind every handle—every flirtation, provocation, conflagration—was a real person. But real people are difficult. They can be unreliable, judgmental, careless, vindictive, demanding. . . Perhaps the promise, and the pleasure, of A.I. companions is not the illusion of another person at the end of the exchange but the inverse: the assurance that there is no one at all.

It is easier to disclose and be vulnerable when no one is home—when there isn’t real disclosure and vulnerability meeting you back. When difficulty is only there if and when you want it.

To return to reading: The challenge, I think, is holding on to what one deeply wants when the overwhelming environment becomes saturated with easy choices that cater to preferences. Or when finding what one wants involves sifting through the slop—particularly pretty-sounding slop—of what one doesn’t want at all.

As teachers, I think these are questions we can bring to our students: What do you want? When have you chosen something difficult instead of something easy? Why? What is beauty, really?

1. The article links to two studies; I’ve kept the original links in the quote. There is a lot more to unpack from the studies, beyond the preferences of readers, including the methods by which the AI were trained and the designs of the studies themselves. This is more than I am able to get into adequately here, but in summary:

Both studies trained AI on individual authors’ works—the first on deceased poets, the second on mostly contemporary prose writers. There are obviously ethical questions about training AI in this way, particularly with the contemporary writers who are still alive and writing.

In the first study, the readers, who mostly had little background in poetry, read a mix of poems generated by AI and written by humans. They were asked to evaluate them in a variety of conditions, some in which they were told about the authorship, and some in which they were not. The participants rated the AI poems more favorably—except when they were aware that they were written by AI.

In the second study, MFA students were asked to emulate the same authors’ writing that the AI were trained on. These passages served as the comparison for readers, who were a mix of MFA candidates and college students. From a design standpoint, I’m not sure why the human writers were asked to mimic the style of the authors. Ethics aside, it makes sense for the AI to be trained in this way. But asking people to model their writing after another person seems likely to produce worse writing, since they would not be using their natural style or voice. It’s unclear to me that the study is acurately evaluating responses to human writing vs. AI writing, since the human writing is produced in such an unusual manner. In any case, the college students rated the AI writing higher in all conditions; the MFA students rated human writing higher when they compared it to the initial AI passages, but ranked the AI writing higher when it was “fine-tuned.”

2. The first study cited in The New York Times reaches a similar conclusion: “the simplicity of AI-generated poems may be easier for non-experts to understand, leading them to prefer AI-generated poetry and misinterpret the complexity of human poems as incoherence generated by AI.”