Is this AI?

Why the answer tells readers less than they think.

There’s a new kind of social interaction on the web. Someone posts an essay, and someone else replies with a screenshot from Pangram showing that it’s 100% AI generated. Substack, amid a deluge of AI content, recently integrated Pangram into its site, making it easy for anyone to check the provenance of any newsletter.

This strikes me as a strange social act. It feels, well, rude to run someone else’s work through an AI detector. It’s an effort to impose some social cost on the use of a technology, generative AI, that is itself deeply contested. But to what end? When the commenter flags a piece as AI generated, what have they discovered? And what are readers supposed to do with that information?

For some readers, the process that produces text matters deeply - because of ethical concerns around LLMs, because of a desire to avoid workslop or to support human creativity. But if you’re not categorically opposed to AI in the writing process, a detector can’t tell you much. It can’t signal whether a piece is true, whether it offers insights that will change the way you approach your work, whether it’s worth your time. It can’t reliably tell you whether the writer wholesale generated a piece, used AI for copyediting, or something in between.

The detector gives us a rough proxy for one part of the process that produced a piece of writing. The Pangram screenshot, though, seems to act as a more severe verdict based on that proxy, casting judgment on the work’s quality and/or the writer’s authenticity. But a detector can’t map the things that matter most about process: where ideas came from, how judgment was exercised, who’s taking responsibility.

So what do we evaluate instead?

(To state up top - I draft most of my work myself. I also use AI extensively, to help structure my rambling thoughts, do research, and edit for clarity and structure.)

Reading for what?

As a reader, I want to read things that are good. There are a few dimensions that roughly capture what I mean by good. First is utility - the piece contains some information that is useful to me. Second is prose - the writing is well constructed and enjoyable to read. And third is trust - I have reason to value the author’s perspective, whether because of expertise, judgment, or lived experience.

You can map all kinds of writing along these three dimensions. A literary essay often depends heavily on strong prose. A research paper often leans entirely on the utility of its findings and trust in the methods. A personal essay relies on the trust the reader places in the author’s perspective and lived experience.

The specifics of these dimensions can vary from person to person (e.g., your idea of good prose may differ from mine). But historically, we have evaluated these qualities through the work itself. We read the prose, the evidence, and the source, and we infer whether someone exercised sound judgment in deciding what to say and how to say it.

AI complicates that inference. A text can now signal expertise, perspective, or care without actually having them. The complications are nuanced, but we can begin with a clear red line: AI slop.

Slop

AI slop is decidedly not pleasurable to read. It is full of distracting tics, empty phrases, and structures that double back on themselves. It is also not very good at conveying information. An LLM can summarize a paper, synthesize sources, and deliver a three-page report on almost any topic, but odds are that you will struggle to latch onto the material.

The deeper problem is that information needs a perspective, an argument that makes clear why the reader should care. AI can simulate a position, but it cannot be accountable for one. It has nothing at stake in whether the reader accepts its argument.

Slop, then, fails on all three dimensions. It is also the easiest case - readers hardly need a detector to spot it. Things become much more complicated as soon as a person exercises even a modest degree of judgment.

Utility

A useful piece gives me new information, a convincing argument, or a way of seeing a problem that changes how I think or act. An AI detector can’t tell me anything about these qualities.

The process behind a useful piece can take many forms. A writer might develop an argument through years of research and use AI to turn an outline into prose. Or they might write every sentence by hand, but use AI to supply the central framing. From a reader’s perspective, neither description on its own tells me whether the finished piece has anything valuable to say.

That is nothing new. Human authorship has never guaranteed quality or integrity. Long before generative AI, people made bad arguments, fabricated evidence, and plagiarized. AI makes empty writing cheaper to produce, but it does not change the ways in which readers need to evaluate claims, evidence, and arguments for themselves.

Prose

Readers can evaluate prose directly, the way we always have: Is the writing clear and well constructed? Is it enjoyable to read? Does it make the argument easy to follow? Once again, a detector can’t answer these questions for us.

AI prose has recognizable tendencies, but those tendencies would be a problem whether the text came from a human or a machine. They’re simply poor writing, and human writers can churn out tedious prose, too.

Here an AI detector can tell us something about the text itself, but the qualities it picks up on are also evident in the text itself. If they diminish the writing, readers are able to judge that from the page. If the prose works, we have little reason to go looking for a detector score.

Trust

Unlike the other dimensions, trust can’t be entirely evaluated through what’s on the page. Process plays a big role in building trust - Did a reporter witness the event they said they did? Did a researcher conduct an analysis accurately and ethically? Is an essayist making an argument in good faith? An AI detector can’t answer any of those questions. AI can inject a text with markers of expertise or care, without those qualities actually being present; but then, so can a human author.

Process can also look a lot of different ways, with a lot of different approaches to translating ideas into words on the page. A single generated sentence might contain an essay’s central claim, while pages of generated prose might express an argument worked out in detail by the author. Writing has always been highly individualized, and we’ve always used highly idiosyncratic and personal ways of deciding whether we trust it. That huge variance in process and perception can’t be boiled down to a simple detector score, because a detector can’t measure accountability.

The screenshot

The things that make writing good haven’t changed: utility, quality prose, and trust in the author.

That is what makes posting the Pangram score such a strange social act. It invites others to dismiss the work and distrust the writer. All this starts to feel like policing individual writers against highly variable purity standards.

Process matters. If a work resonates with me, I may want to know how it was made. But I want to know from a place of curiosity, not from a desire to interrogate its creation. Readers still have to decide what deserves their attention in the same imperfect ways we always have: by seeking out what seems interesting, engaging with it deeply, and filtering out the slop.

In Part Two, I’ll tackle the question from the writer’s perspective: what we owe our readers, and ourselves.