Does AI Detection Make Things Better When It’s This Flawed?
Whet are readers and authors supposed to do with this?
I’m sure I’m not the only one doing this kind of thing but I’ve done a few unscientific experiments with Substack’s new AI detection. And it’s not inspiring.
In one example, I plopped a few paragraphs into an article, maybe 300 words or so. Checked, and it came back “100% human”. Ok great. Then I added a few paragraphs from AI (maybe Claude, maybe Gemini, don’t recall for sure). Still says, “100% human”.
I deleted those AI paragraphs and added one at the end of the article, and boom! It flipped to 100% AI generated.
Same exact article that was 100% human with a few sentences from AI added and now it’s 100% AI according to the detector. Not 2% or 5% or even 50% but 100%.
If we’re being told we should be using this stuff to “catch” writers, that’s clearly a problem.
A colleague noted whether readers really care that much. Or do they only care because Substack is telling them to care? Tim Requarth talked about this in a piece on Slate (paywalled) and elsewhere.
I’m watching an AI-detection-fueled train wreck in media and publishing, and I fear people aren’t taking into account the limitations of AI detection software (or more precisely, AI “inference” software), nor the conflict of interest of the companies riling up the mob.
I appreciate the idea of transparency, I suppose, but even that can feel like hive mind conformity, something we know a lot about from our polarization work.
Sure, we don’t want Substack to be filled with nothing but AI slop (even the definition of that is squishy), but do we need this tool to deal with that? Does it even help with that problem?
If readers get value out of “AI generated content” is that ok? If it’s really “slop” won’t those publications have difficulty maintaining an audience? (There’s plenty of human generated “slop” too, BTW). If they do find an audience, aren’t they providing perceived value to that audience?
I don’t have the answers to all these questions, but as usual, I’m skeptical.