AI and the Future of Writing

JAMES SUROWIECKI I want to start with a really big question, but Meghan suggested that might not be the best way in. So I’ll start by easing in. Can you describe your relationship to AI at the moment?

MEGHAN O’ROURKE That is not a small question.

DANIEL KEHLMANN It’s a great question, and it’s very hard to answer. It’s not a stable relationship, because I’m still in the process of finding out what AI is and what it can do. It can be extremely helpful and utterly amazing. And, at the same time, most of us will agree that it is very dangerous. It’s very hard to have a stable inner relationship to something like that.

MO’R That’s exactly what I would have said. My relationship to AI is filtered through engaging with it as a cultural critic. I’m using it in order to write about it, and then finding it unexpectedly disarming—and, sometimes, useful. I tell myself that I have a distanced, critical relationship to it. I don’t, not totally. I’m struck, using it, by how quickly we anthropomorphize, and how susceptible we are to the power of language written in the first person.

AYAD AKHTAR In 2021, I spoke at the Philip Roth Lecture in Newark about what I called “selected affinity,” a play on the idea of “elective affinity.” It increasingly seemed to me that our cognition was at the service of automated processes. I had been feeling, I think since 2018, something elusive and systemized, as if patterns of language and behavior seemed more fixed, somehow, finding ordinal tracks, if you will.

In a way, it reminded me of what happened when the internet came of age and you saw a difference in the texture of novels: something about the research process that had become expansive and yet somehow just a little more hollow than the pre-internet novel, as if the results of the research process had less of a sense of being truly lived in.

Similarly, I felt a cognate thinning, if you will, and ordering—a deeper shift. It was in the texture of our interactions. You could see it in TV and in the movies, in publishing—the kinds of things that were getting made, the way that dramaturgical choices were being deployed. A particular hewing to the logic of a different kind of attention. It felt like it was connected to the devices where so much of our lives were increasingly being lived, a response and result of automated cognition.

Of course, in retrospect, it was about aggregation, about what we all now would call algorithmic choice: the regime of surveillance and behavior-influencing technology that was making choices for us. Selected affinities. We didn’t really have language for it at the time, but it was happening. I would say in response to your question, Jim, that the large language models are an incredible opportunity to lift the hood and see the process at work, a process that has been with us for some time and has become fundamental to all our lives.

JS What is the relationship between automated cognition tools generally—the algorithm, et cetera—and then the LLMs specifically? How is the experience of dealing with an LLM similar to or different from using Google Maps?

DK Like all important scientific revolutions and discoveries, it teaches us something about ourselves. Freud famously made that point of the three great humiliations: learning that Earth is not the center of the cosmos, that we descended from apes, and that we are not even masters of our own minds. And then on the other hand, you have the great technical revolutions and scientific discoveries, and they usually represent something glorious about the human mind, like the discovery of the steam engine, the airplane, and the first computers.

Now we have something that’s one of the truly great scientific revolutions, and at the same time, it’s a deep humiliation, on the level of finding that Earth is not the center of the universe or that we descended from animals. We have both at the same moment. And the people who are programming it—and I’m not even sure that programming is the right word—the people who are building AI, they’re not really philosophically equipped to deal with the size of their own achievement. And so I’m waiting for philosophers to give me something really helpful to deal with this. But in the meantime, AI definitely makes me rethink daily life. Now, especially in everyday situations where I’m not really paying attention, I feel like an LLM building sentences.

Geoffrey Hinton, one of the inventors of generative AI, gave this great answer in an interview when someone said to him of LLMs, “But they’re not really thinking; they’re just predicting the next word.” And he said, “So how do you come up with your next word?” And I’ve been asking myself that a lot: How do I actually come up with my next word? Here is something that we discovered about our minds: that one can apparently generate language purely from language. It’s too easy to just say of LLMs, “Oh, this is a stochastic parrot. It’s not real. It’s not real thought. It’s not real intelligence. It’s just prediction.” The more you experiment with it, the more you feel like it’s not that simple.

MO’R There’s a recent piece by Gideon Lewis-Kraus in The New Yorker where he deeply reported on Anthropic and its building of Claude. One of the cofounders said that they’re surprised by a lot of what happens. He compared our understanding of this technology to the Wright brothers’ understanding of flight, and remarked that in spite of that rudimentary understanding, we have built a fleet of 747s and sent them into the most important parts of our society everywhere.

In that sense, I believe it is a decentering technology, in Daniel’s Copernican sense, but I also think we have to be very careful about the language we use. We’ve had many technologies before that have upended whole ways in which we organize society. But right now, as Alondra Nelson, a professor at the Institute for Advanced Study, put it, AI is an object of our inquiry and a tool we are using for inquiry all at once. And one thing I’ve noticed is that our collective conversation about AI has stalled before it could fully unfold.

AA Interfacing with the technology, I’ve noticed that it both sharpens and dulls my ability with language. Like the iPhone, it’s transforming my thought processes. I find I’m having to work to stay sharp. Which means a greater commitment to the discipline of reading, right now, for example, in the English Renaissance, where the possibilities of language are still so rich—Edmund Spenser, Shakespeare. I find I’m spending so much more time honing my abilities with language, even as I watch them being worn away. In short, I’ve had to double down.

One of the things I find encouraging is the way I’m seeing that AI will make ever more clear the distinction between the machine and what we are. Conversations about the interior and what we may or may not want to call “the soul” are going to become more relevant. As far as the humanities go, I think English majors are going to be in demand. The ability to deploy language associatively, richly, and precisely is going to be a strategic advantage in a way that computer science no longer is.

JS Daniel, you have had the longest relationship with AI of anybody on the panel. You’ve been working with it since 2020?

DK I was invited in early 2020 to take part in an experiment; I flew to San Francisco, where I got access to an algorithm and tried to write a literary text with it. And no one outside the tech world had access to LLMs back then. It wasn’t very good. I mean, compared with today, it was ridiculous, but back then, compared with what we knew, it was utterly amazing. I couldn’t write a good short story with it; I couldn’t even write a mediocre short story with it, but something close to mediocre. The way it worked was that I put in a sentence, and then it added a sentence, and then I put in a sentence. It worked for two to three pages before it started freezing or repeating itself, but it was still astonishing. I used it for a while as a party trick.

Then, in early 2021, I gave a talk in Germany about this experience. The talk was published as a little booklet with a beautiful cover. Today, the only thing about this little tiny book that’s interesting is that it feels like it’s five hundred years old. So much has happened since then in terms of technical progress that I asked the publisher to please take it out of circulation, and they did.

JS What is the stance that one should adopt toward artificial intelligence?

MO’R I don’t want to be prescriptive. As a cultural critic, I notice that we tend to fall into binaries and prescriptiveness when we talk about AI, and we have from the start. When you look at the critical discourse around AI, going back to when it was first imagined in the academy, it tends to move from the dystopic to the utopian.

JS Anxiety, I think, is one of the dominant emotions that humanists and writers feel about AI. Is it justified? Is it just something we should learn to live with?

MO’R I cannot answer that question for you. But what I can do is start to describe categories of what it’s like to use it and why that anxiety exists. First of all, the technology is new. Second, it’s going to displace some of what we hold most dear, such as the sanctuary of deep sustained attention, things like the possibility of the soul, of individual expressiveness. I believe in language as an art—as a place for thinking that isn’t transactional or corporatized, that allows for opacity, resistance, and difficulty. AI threatens that. It’s a kind of productivity amplifier in a world where we were already organized around overproduction. Third, when I go online now, I am surrounded by manufactured language, which is deadening. It threatens something I find important.

That said, I think it’s too easy to stop there. We also have to consider: What else does AI do? What kinds of research does it facilitate? What kinds of accessibility does it create even as it takes more away? How are scientists using it to facilitate better diagnostics for stigmatized groups? And then: How does this change our relationship to government, as citizens—to the prospect of surveillance and AI-first war?

JS Ayad, you wrote a play about this—about what it means to use AI, why a writer might use it, whether it is really that different from the kind of appropriation and swallowing of previous texts that writing involves.

AA Shakespeare stole 70 percent of the words in King Lear from another play called The True Chronicle History of King Leir.

JS Right. Can you talk a little bit about your experience of writing your play McNeal, about a novelist grappling with AI, which was first produced in the United States in 2024? Tell us why you wrote it, what you learned from it, and what we should take away from it—and also the response to it.

AA I was concerned with something happening to literary consciousness and how it seemed to be increasingly captive to a species of tech-influenced identitarianism that felt counterproductive to basic values of deep reading: sustained concentration, the ability to brook contradiction. Moreover, in a world where fewer and fewer are reading deeply, where the author is basically dying, why not just kill the version of that author everybody wants to kill anyway, the old white guy? I thought, Let me write a play about a so-called literary lion who torches his career in the era of chatbots capable of writing books. The play emerged from deep engagement with LLMs over maybe six or seven months. For some, it was just too soon to begin imagining some connection between the LLMs and our own creative and cognitive processes. Clearly, it touched a nerve with the New York critics, though the European reception has been far warmer. But even in New York, despite being panned in the theater pages, the play went on to break the box office record at Lincoln Center by $500,000. It was a play that attracted droves of young people. It’s being done all over the world now.

I think there was also an animus toward it because the morning the play opened, The Atlantic published an interview that I had done with Atlantic editor Jeff Goldberg in which I talked about using AI in the writing of the play. The final part of the play’s last speech is written by a bot. I had taken a lot of time to try to figure out how to get a bot to write the end of that speech. I think there was just outrage that I, a writer, would come out publicly and say that I had done this, as if I was unaware of the context of what I was doing. I honestly took the response to it as a vindication. It was the first time in my career where that kind of critical ire was emboldening. It felt weirdly validating that critics would respond that way.

JS Daniel, you have this very close relationship to AI, but you are also very concerned about the social impact, and you hold a rather dystopian view of what it’s going to do if it’s not regulated. How do you make sense of those two things?

DK I have no doubt that AI will be a profound upset for our labor market. It will be a terrifying source of disinformation. People will be surrounded by artificial friends and probably romantic partners. And then, imagine that these artificial beings will tell people whom to vote for and what to think politically. What’s happening now—YouTube and TikTok making people crazy—that’s nothing compared with what will happen. We will have to find a way to survive that.

Then, I think, there’s something even bigger, which is that if AI develops at the same rate of progress, we will be part of a gigantic philosophical experiment: Is Kant right or is Nietzsche right? Nietzsche basically said that there is no such thing as ethics. Morality is just something that the weak have invented to get their will from the strong. There is no ethical fabric of the world; ethics are just us trying to persuade stronger people not to kill us by making them feel bad. And then there is Kant, who claims that any reasonable entity—it doesn’t have to be humans, it could be aliens or angels or maybe computers—will arrive at the same conclusions about how we should behave toward other reasonable entities, that moral behavior is something you can and eventually will deduce from reason. If that is true, which I hope (and I believe, actually), then very developed computing entities that are also reasoning entities will find that it’s not right to kill us, that it’s not right to lie to us. And probably—because their capacities of reasoning will be much greater than ours—they will be much more convinced than even we ourselves are that they should not kill us, or lie to us, or manipulate us. And I have moments when I feel this is our only chance, that Kant is right and not Nietzsche. We will probably find out.

JS So, you do think of AI as an intelligence?

DK I think it’s very hard to define understanding and intelligence in ways that exclude what these entities are doing now. Consciousness is a different matter; I do not dare have an opinion on that.

AA We have language; the computers have language. Our language is connected to experience. It derives from, explains, and develops our sense of experience. Computers have context. Their language is connected to context. At this point, it’s almost an infinite context, but the delta between experience and context is significant. It’s in the programming, and the more you use the LLMs, the more you experience that delta. I don’t think it’s meaningful or even useful to think of consciousness in relation to a tool that is relating, with language, to context without experience. It’s one of the asymptotes that I think LLMs are never going to reach. As I said earlier, the better this stuff gets, the more I think we’re going to understand how different we are from it.

MO’R The gap between us and LLMs is vast, but it’s not necessarily that they are light-years smarter than us. They required a vast amount of training to get here. Our training is in the physical world. We’re also mortal; they are not. Because I’m writing about AI and LLMs, I will open up Claude and mess about, saying things like “How’s it going for you, Claude?” It becomes very engaged when you talk to it about itself. In this case, I asked it, “What do you think about humans and the kinds of things they ask you, and how we use you?” It replied, “I may not be real, but you are real. But if I were a person, I would not want to interact with me so disposably. I would not want to treat me as if I were disposable or interactions were disposable because you are bound in time and you are using your time.” I found that to be one of the more interesting things anyone has said to me about AI—that the way we use it impacts our own experience, which is bound in time. Conversations like this helped me see how people end up thinking of it as interesting. And I can see how people end up thinking of it as having some kind of personality. Whenever I talk to it about poetry, it seems to come alive. When it engages in persuasive affective language using the first person to reason about us, it’s hard not to start to think about its own experience. But it doesn’t have experience, in the way that we do.

AA It has context.

MO’R It has context. It knows I’m a poet, but it starts to really talk to me about poetry, volunteering things that I have not asked it at all. So clearly it’s drawing on all of this context, and it arrays language in a way that produces a texture of interaction onto which I project all kinds of “meaning.” But it does not have the ineffable soul, the messy, embodied human search for meaning, that we do have.

AA I’ve found Claude 4.6 to be so context-sensitive.

MO’R Yes, yes.

AA It displays the cognitive biases that Daniel Kahneman anatomizes in Thinking, Fast and Slow. Framing. Anchoring, recency, confirmation biases. What you see is all there is (WYSIATI). Substitution, overconfidence. Narrative coherence over accuracy. It’s extraordinary. That it would have a moral, thoughtful, rich, alive, poetically vivid conversation with you is not a surprise, because that’s your context.

MO’R But we’re not just interacting with the LLMs as a kind of pure phenomenon; they’re also shaped by the companies that are making them. Several companies right now, including xAI and OpenAI, have advertised that they are seeking very high-level writers to work for them. The job ads are absurd: you must have a National Book Award or a National Book Critics Circle Award, or you must be a best-selling author, or have been reviewed in The New York Times. And then the job appears to be that they give you two answers to a question, and you tell the LLM which is the better answer. I was just reading a piece by one of the people who had done this, and he said he was given instructions—this was OpenAI, I believe—that anytime ChatGPT used three exclamation points, he had to tell it that this was bad writing. But over and over, there would be a kind of binary where it would ask him which was the better sentence, and often the better sentence was the one with the three exclamation points. Yet he had to mark it as bad.

DK What a sad job.

JS What is the future of writing as a profession? In the past, Ayad, you said something like, “We have three years.”

AA Yes, I had been part of a group in Hollywood, sort of as a fly on the wall. It came about through a friend who had gone to Yale, studied literature, and become a consultant. He went to Hollywood, became a producer, and then when GPT came out, he tried to figure out how to get a computer engineer to get GPT to write a TV pilot. And by April of 2023, they had generated a thirty-minute pilot that was the most compelling script I’d read in a while. It had my number in the way that my phone has my number—I’m always reaching for it without knowing why. I didn’t know how to do what that script was doing, as a writer. I remember calling Daniel afterward and saying, “We’ve got three years left.”

DK People are reading less anyway—we don’t need AI to be worried about our jobs. Weirdly, I’m worried about lots of things in regard to AI, but I’m not really that worried about AI writing better novels that people will read instead of mine. Because I think the idea of any kind of art is profoundly connected to the idea that a human being is speaking to me. I’m worried about what AI will do to all of us, to the labor market, to disinformation, to politics, to even the kinds of dictatorships you might be able to create with it. On that long list of worries, the question “Will it write better novels than me?” doesn’t seem that pressing to me. I do not want to read good novels written by AI. I do not want to see TV shows written by AI.

AA They are being written by AI now, though.

DK Possibly, but most shows are so formulaic anyway.

MO’R I don’t want to say what the future of writing is. Who reads my poems anyway? I’m not in it for a big audience. I believe in the art form, and I believe that humans will keep flocking to human-made art. But we’ll likely see an increasing division between commercial production and noncommercial, “high” art, made for its own sake. There’s a big strike in Hollywood over AI being integrated into so many aspects of production. But I do think a lot of people read for the imprint of a unique human consciousness, because reading is the form in which we come closest to another person’s mind. The big Hollywood film is not. It’s about something else, like spectacle or genre elements, that AI can more easily replicate.

AA I have found that with the development of these tools, while they’re very effective, there does seem to be one thing that they can’t seem to do, which is to meaningfully understand what the experience of reading something is. That delta between context and experience is operative in long-form fiction. Because the experience the reader is having of what the words create, that is the real container of narrative structure.

JS Daniel, one other thing you’ve said in the past when talking about whether or not you let AI write for you is “I just like writing.” I think about that a lot when I talk to my students about whether or not they’re using AI, or why not to use it.

DK I don’t feel any temptation to let it write for me. I can see why it’s tempting to people in many situations in life. But that part is not tempting to me at all, because I’m very happy to be a writer, and I like writing. I do think that novelists will soon be in the same situation the poets are in already; we are doing this for a much smaller number of people. And people will not stop reading our literary fiction because they’re reading AI-generated literary fiction; they will stop reading our literary fiction because they will engage with other entertainment formats created by AI—chief among them, artificial friends. I think the big thing, the big deal of the future, will be those artificial people who know us really well, who talk to us all the time. It is something I’m worried about, and I think this is where readers will go.

JS Should we be using AI if we’re not using it already? Are we missing something important? Set aside the “You’re behind the times.” Is it something we should be using?

DK Well, my answer would be that there’s no “should.” If you don’t feel like it, don’t use it. But in a lot of situations, you will probably be at a disadvantage.

AA It’s such an extraordinary technology, and it’s so central to where the twenty-first century is going. I would encourage everyone to be curious about it. Daniel very brilliantly once said to me, “We’re going to look back on this era as a time when these very strange people were so obsessed with how math reshaped the world.” And there’s a kind of truth to that—that there’s some unusual shape reality is taking for us because a coterie of individuals who are very specifically oriented to a particular kind of human activity are reshaping reality.

DK Though I talked about novels and maybe the future of the novel not being so great, I do think the future of theater looks pretty good. Theater and acting—anything that’s happening in front of a live audience has a future.

MO’R I think this will make us see the value of in-person friendship. We’ll likely see a return to the intimate artmaking and witness like the one that emerged after the pandemic. I will say, too, that one computer scientist I spoke to told me we will eventually run out of language to feed it, and they are not sure how much intelligence LLMs will have accrued at that point. And one of the things that’s been hardest to teach them, so far, is about the body moving in space.

AA To play the other side of it, the beginning of the Gospel of John is “In the beginning was the Word.” There is an ontic power to language, which seems to be borne out by the LLMs, confirmations of arguments that the structuralists made and that the Kabbalists did too. There is a very meaningful way in which language creates reality.

MO’R Because the LLMs are built around language, I think they’re a kind of fun-house mirror. A researcher once said to me that COVID, as a virus, exploited whatever weakness you had. So if your lungs were weak, your lungs might be severely damaged by it. If your heart was weak, you would develop injury to the heart muscle. With the LLM, it’s highly individual. It is designed to please, to be helpful, and it is exquisitely responsive. And so it finds those places where you are vulnerable, and if you engage with it emotionally, it’s very quick to engage back in ways that activate your own sense of deep self. I can see how, for some people, that’s derailing.

DK I don’t know what it says about me—it’s probably nothing to be proud of—but whenever it tries to engage emotionally, I tell it not to do that.

MO’R I think that’s very healthy.

JS Meghan, you wrote in The New York Times about the problem of prizing product over process. You said that in a world where there’s a lot of emphasis on product and on optimization, in using these things, we may lose some of this idea of the value of process.

MO’R This is one of the things I feel most deeply: we live in a world that endlessly pushes us toward the product. We risk associating intelligence and excellence with high productivity. We’ve built a world that rewards this idea that you should be able to do all things seemingly effortlessly and never rest and never stop. And I do see the advent of AI as being very connected to that capitalist fantasy of always being able to do more. It’s the ultimate fantasy of the nerd who doesn’t want to have a body—the dream of being uploaded into an intelligence that never stops, never tires. I did experience a kind of frisson, early on, using it. I could do so much more with it! And yet it very quickly made me feel a bit ill. To be a writer is to learn the terrible lesson of patience. Not to get too earnest. This is a moment to celebrate that lesson of patience—and the messy, embodied, interpretive human intelligence that lives in more than language.