[Essay]
False Positive
This week, I finally installed the AI detection software Pangram in my browser. For months, I’d been intrigued by social media users who’d post about analyses they’d done on suspected AI text. Then there were the AI scandals that made headline news: Hatchett’s cancelled publication of the novel Shy Girl after Pangram showed it was likely to have been written with AI, the pope’s X account flagged by Pangram for a suspected AI-generated tweet, a Modern Love column in the New York Times determined to be AI by a researcher using Pangram. At first the narrative appeared relatively positive. With much hand-wringing over the future viability of human-written text in a world of AI, here was finally a tool humans could use to fight back. Previous AI detection software had been spotty enough to apparently determine the Declaration of Independence was AI. Pangram, meanwhile, boasts an accuracy rate of 99.98%. In the hands of responsible administrators, it could potentially offer an effective deterrent to AI writing — not all that different from the anti-plagiarism software of the early 2000s. But in a digital world supposedly awash in AI-generated text, what would happen if everyone had access to this tool? What if we all started using Pangram to do our own analyses of what is AI and what is human? And how does this thing even work?
Pangram is itself an AI, part of a long tradition in Silicon Valley of offering up more technology as a solution to problems their technology created. The simple explanation of how it works is that it is trained on datasets of both human and LLM-produced text to determine the probability of each individual input (e.g. word or punctuation mark) belonging to a human or AI. Based on the probability of each input appearing in particular order over the course of the entire document, Pangram provides a likelihood that the author is human, AI, or somewhere in between. For example, if a human might use an em dash 50% of the time, an LLM might use it 90% of the time. The more inputs you have to compare, the more accurate the detection. And Pangram is, by most accounts, accurate. The company claims a false positive rate of only 1/10,000 and my own efforts to trick it have so far been unsuccessful. It correctly identified all 10 writing samples in the New York Times AI vs human writers quiz, something I could annoyingly not do. Even running AI-generated texts through “humanizers” — AI tools used to replace LLM tropes with more natural language — didn’t help avoid detection (others have apparently had more success). This all seemed promising, but I couldn’t shake the feeling that using Pangram to detect AI text wasn’t all that dissimilar to using an LLM to generate text. In both cases I was dropping text into a black-box and waiting on the results.
With Pangram installed in the browser, the experience of using the tool changes. Instead of copy and pasting text into the app, you simply highlight and right click on what you’re reading for an on-the-spot analysis. When you browse social platforms like X, Substack and Reddit, that analysis is automated so that users and posts are identified as human or AI as you scroll through the feed. This might be as dystopian as Richard Deckard scanning for replicants, but it’s also boring and not the least bit empowering. We already know that our feeds and search results are stuffed with spam and slop, whether AI-generated or not. AI detection is only really interesting when there is a human on the other end, either intending to deceive or being accused of deception. And it’s here where the narrative around Pangram starts to shift.
Last Sunday, the Atlantic ran a headline stating that America Has A Pangram Problem. Considering that the Atlantic was one of the foremost publications to buy into the benefits of Pangram, this caught my intention. The problem, according to writer Matteo Wong, is that Pangram’s accuracy has emboldened users to conduct witch hunts for AI using Pangram and cite the results as evidence. He points to a recent case where journalist Taylor Lorenz was accused of using AI after a Pangram scan, only to be later vindicated when she was able to show her edit history (ironically, Lorenz had recently declared Substack inundated with AI after she had performed a Pangram analysis of the platform). Wong also mentions efforts by Pangram users to highlight passages in Pope Leo’s AI-skeptical encyclical that may have been written with AI, accusations that the Vatican, of course, denies.
In addition to the bad-faith actions empowered by Pangram, there are also technical and conceptual problems with AI detection. LLMs evolve quickly, always toward the aim of appearing more human. It’s unlikely that tools like Pangram can keep pace, a prospect Wong likens to “building a sandcastle at low tide.” Conceptually, AI-detection might be too far downstream of where the biases of the LLM have the most influence on the writer, like during research. Finally, there is the question of how much people even care that something was AI-generated. It might feel rude or icky to encounter AI during what you thought were personal exchanges, but when it comes to art, studies continue to show that people prefer AI to humans — a finding writer Max Read explains by suggesting that people just like bad art, which seems undeniably true.
What Pangram appears to have accomplished is at least temporary proof that AI can be effectively used to identify other AI. But the product marketed to users has a slightly different objective. If Pangram really wanted to rein in AI bot slop, it would sell to the platforms, not the users. The reason that they don’t isn’t just that most platforms have little incentive to filter out AI, it’s that what Pangram is selling isn’t AI detection but agency. The AI companies have convinced us that AI is smarter than we are and thus a convenient tool for deception. Pangram sells us a false security of being able to root out deception.
In F is For Fake, Orson Welles’s excellent documentary essay on fakery, Welles explains that as long as there are fakers, there have to be experts. He then asks: but if there weren’t any experts, would there be any fakers? I doubt we’ll ever find out.
