The Limits of Language
with Vauhini Vara and Zoë Hitzig

ZOË HITZIG
It’s nice to meet you. When I read your book it was like, “Oh my God, I can’t believe there’s this kindred spirit out there.”

VAUHINI VARA
I have your book right here, and I feel like there’s so much to talk about. But Zoë, I was curious about your path as a technologist and your path as a writer, how each of those started and where they converged.

ZH
Well, I’ve always wanted to be a writer, to make sense of the social world. At the same time, I’ve always had a deep love of math. And in a way that combined two distinct impulses, I was always drawn to formal renderings of social problems, whether through economics, which is what my PhD was in, or through technology. 

VV
Where does computer science come in for you?

ZH
It’s mainly through my PhD. My PhD was at the intersection between economics and computer science. So I actually don’t have any degrees in computer science.

VV
But you were also always writing while pursuing that academic path.

ZH
I made choices along the way to make sure that I always had a writing community around me. I know that you also have written in different genres, but the fact that I have my first two books in poetry feels kind of random. I love poetry, but in some ways poetry was easier to do on the side than writing a novel. I haven’t written a novel, but it seems like it takes a lot of focused, intense energy.

VV
I write in so many genres, but poetry is the one genre that feels beyond my reach, so it’s funny to hear you say that. Poetry seems really hard to me. My sense is that you can make significant headway in poetry with shorter bursts of time, whereas if you’re working, at least for me, on a novel or something longer form, it’s really difficult to accomplish with smaller bursts of time. I think, if that’s one way you can define easier versus harder, it’s easier to find the time to write poetry. 

When I was reading your book, I was struck by the relationship between how you use language in this text, and the way in which AI deals with language, as units of information. I feel like there are more poets experimenting with AI than there are prose writers. My theory is that it’s because there’s something about these [large language] models that is in tension with the narrative goals of a prose writer. Whereas looking at it as this machine that produces words is potentially interesting to a poet.

ZH
Part of what you’re saying, I think, is that poetry’s allowed to be a little bit more conceptual than other forms of writing. It can be closer to conceptual art—it can play with found texts. That said, your book also contains found texts, a sort of new category of text which sees text as a by-product of digital life. Poets have long been drawn to the aleatoric—we love to look for chance symbols—I think of all those poets who were obsessed with Ouija boards. There was a similar thing in early large language models where they were asking, “What’s the machine going to say?” And really, we were just rolling really high dimensional dice where all the sides were words. What could be more interesting to a poet?

VV
There are all kinds of arguments writers could make about why AI isn’t interesting as a writing tool; one is that literature should always involve authorial intent. But I think of those Oulipians of the past, of Dadaist poets and prose writers, who actually push back against that conception of literature as requiring absolute intent in that a human is producing every word in a particular order.

ZH
That’s a great way of putting it. How did you decide to put your conversations with ChatGPT into the book? I know that you got access to GPT-3 [a predecessor of the AI models underlying ChatGPT] after you profiled Sam [Altman] and you did this beautiful piece, “Ghosts.” You’ve had some relationship with these tools, but did it feel natural to put them into the book?

VV
Searches started out as a series of experimental essays using language created in relation to technology products: Amazon reviews, X’s list of my interests, my own Google searches, as well as the essay “Ghosts,” for which I used GPT-3.

The subtext of putting all this material in the book had to do with playing with that line between what these products offer us, in terms of self-knowledge, knowledge of others, and knowledge of the world, and what makers and investors of these platforms are then able to get from us through that use. Then, I gave it to my editor, who said, “A lot of what you’re doing here is in subtext rather than explicit. What can you do to make it more explicit?” I suddenly was like, “Wait, she’s right. I think I need to have an entire narrative arc in the book that actually makes explicit this argument I’m making about our complicity as users of these technologies in the exploitation of ourselves and the planet.”

Another thing my editor said was, “I wonder what would happen if you fed your book into ChatGPT?” At first, I cringed at that. But then I was like, let’s see what happens. I was specifically interested in seeing how ChatGPT would potentially reveal its own flaws and biases in a conversation. There are these mini-conversations with ChatGPT that frame the book, and I feel that there’s an arc that develops through that conversation, which advances a kind of argument I’m making about what happens when we engage with ChatGPT.

ZH
Reading those ChatGPT moments in the book, I had this peculiar feeling of pre-nostalgia. Not that I’m particularly fond of ChatGPT––I’m definitely not––but one of the things that is animating a lot of my life right now is imagining how much more of all of these tools there will be in the future, and how they’ll inevitably become more advanced, more seamless. Your recorded conversations, flawed as they may be, felt like a meaningful preservation of something that, in a decade, we’ll have largely forgotten: what ChatGPT sounded like in 2024.

VV
Something I was really interested in asking you about is the progression of the models underlying AI products. The models have increasingly been fine-tuned to be more predictable and reliable. The language they produce reads to me, as a writer, as less creative or original than what early models produced. Because you brought up the future of AI models—is there, in your view, an inherent tension between predictability and accuracy on the one hand, and creativity and originality on the other hand? If the companies and organizations behind large language models want to make money by building things that are marketable, then that will always naturally be in tension with models that produce creative language, right?

ZH
That’s a really great question. A lot of our conversations about AI tend to focus on technologies that are easy to access and widely used, which usually means they’re big models, built by big or at least very well-funded companies. But there’s also this exciting parallel universe of open-source and even quite small models. These small, open-weights models, which are somewhere in between open-source and closed-source, might not win silver medals on the International Math Olympiad or figure out how to book you a vacation from end to end, but they can be creatively interesting precisely because they’re not limited to being ultimately sold as a product. I’m especially excited about the possibilities in open-source communities and the ways creators could push small open-weights models into really creative and unexpected directions. As a writer, I always have a thesaurus on my desk, and I’m constantly flipping through random books, looking at words. Why not build small, writer-led AI tools for that kind of linguistic serendipity?
The other thing that I would say is that there is a lot of discussion about whether there’s a trade-off between creativity and safety. The predictability and accuracy stuff is, in some ways, what makes it boring, but also scarily capable, but it’s also what allows us to make sure that these things aren’t going to be used for great harm.

VV
It’s interesting that you bring that up because that then suggests that the really big models have an incentive to go in the less surprising, less original, less creative direction because that’s what you need to do to align with safety, accuracy, and reliability, to reduce the risk of bad things happening because of the platform. It’s also true that it’s becoming cheaper and easier to build smaller AI models that can do all kinds of different, individual things.

Right now, when we talk about AI models and art or literature, there’s this assumption that we’re talking about the big corporate-owned models. But in a future, in which big corporate-owned models are not interested in art or literature in the way that you, me, and the readers of this magazine might be, but rather something blander and less original, then the models that are potentially being used to create what we might characterize as “real” art or literature are not corporate-owned, but, rather, communally owned projects. That makes the ethical conversation and the artistic conversation about those relationships more interesting.

ZH
Totally. The visual artists are ahead of us writers on this––they already have some community-governed datasets for building models. There’s this amazing new database of images, PD12M, which some artists put together, where every image in the database is an image that someone actively wanted to be included, and they’re playing around with collective governance procedures to think through who’s allowed to train on the images. I’m not all that involved in the world of AI and art, but I follow it to some degree.

VV
Toward the end of your book, there’s a sort of deconstruction that takes place where the syllables of words get broken up. The poetry takes the form of syllable, dash, syllable, dash, syllable, dash. And I wondered about the extent to which you were thinking about language in a computational way where language was becoming just these units of material and the meaning was getting deconstructed.

ZH
It’s such a weird poem, isn’t it? What I was thinking about there, again, was this idea of imagining all the by-products of our lives being rendered as data—something that shows up frequently in your work as well. I started picturing a machine-driven future where there’s an overwhelming amount of information and everyone’s life stories are just out there, accessible. I was curious about what would happen when a machine tries to parse all that—mining it for tokens, breaking it into syllables in an attempt to make it legible or usable for some other, alien purpose. What would that feel like to the subject? How does it feel to envision a future where the sum total of our lives is picked over by various machines? That’s why the syllables break in unexpected, unsettling ways.

VV
I always think about—and I can tell you do, too—the relationship between form and meaning. It feels like you found the right form here to simultaneously render the way we as humans glean meaning and the way a large language model does.

ZH
You have a chapter in your book made up of your Google searches across ten-plus years. It’s quite remarkable that you kept all of them! Was there a specific moment when you consciously decided to keep those searches?

VV
Once I learned what was being tracked, I had to ask myself, do I want to turn this off? Or, do I find it to be such a useful, interesting, and even moving archive of my own life that I’m going to proactively decide to make that trade-off and have Google continue to collect this information, knowing that it contributes to its surveillance of me? I have that section at the very beginning of the book because in some ways I feel like it sets up the whole idea of the book. Oftentimes we talk about the relationship between technology products and us in this binary way, as in, we as users will say, these companies are just exploiting us. And then the companies will say, understandably, you’re using these products because you choose to, therefore we must be delivering some significant value to you. So in that essay, it felt like I was just showing readers what that archive looks like.

In my own work, the question I was asking myself is: Is it possible to reappropriate this language for my own emotional and artistic use, and then also for my own profit, because I’m going to sell this book with my Google searches in it?

ZH
One of the biggest challenges I encounter when thinking and writing about the information ecosystem and the ways our data are collected by entities that don’t have our best interests at heart, is people responding with, “Who cares?” It’s a reaction I constantly run into, and one that I see as central to your book, too. I’m curious how you respond to this. For me, it feels fundamental—almost like a matter of life and death. What’s the point of living if you can’t have the freedom to become who you want without being surveilled or having your identity, tastes, and experiences mediated and manipulated by outside forces?

VV
My sort of philosophical answer to that line of questioning has to do with the primacy of being an autonomous human being in community with people we engage with outside of economic systems. But I think people are interested in what the material consequences of that are. And for that, I return to Shoshanna Zuboff’s idea, which she explores in The Age of Surveillance Capitalism and her New York Times pieces. Basically it’s this idea of a kind of knowledge gap between what we ourselves as individuals can know and what these companies can know—and then the way in which that knowledge can then be instrumentalized by companies into material gain, including in the form of political gain. That feels to me like a legitimate, convincing argument, the consequences of that knowledge gap.