There is no such thing as an effective "AI detector", nor will there ever be one.
There is no such thing as an effective "AI detector", nor will there ever be one.
I keep seeing posts about this kind of thing getting people's hopes up, so let's address this myth.
What's an "AI detector"?
We're talking about these tools that advertise the ability to accurately detect things like deep-fake videos or text generated by LLMs (like ChatGPT), etc. We are NOT talking about voluntary watermarking that companies like OpenAI might choose to add in the future.
What does "effective" mean?
I mean something with high levels of accuracy, both highly sensitive (low false negatives) and highly specific (low false positives). High would probably be at least 95%, though this is ultimately subjective.
Why should the accuracy bar be so high? Isn't anything better than a coin flip good enough?
If you're going to definitively label something as "fake" or "real", you better be damn sure about it, because the consequences for being wrong with that label are even worse than having no label at all. You're either telling people that they should trust a fake that they might have been skeptical about otherwise, or you're slandering something real. In both cases you're spreading misinformation which is worse than if you had just said "I'm not sure".
Why can't a good AI detector be built?
To understand this part you need to understand a little bit about how these neural networks are created in the first place. Generative Adversarial Networks (GANs) are a strategy often employed to train models that generate content. These work by having two different neural networks, one that generates content similar to existing content, and one that detects the difference between generated content and the existing content. These networks learn in tandem, each time one network gets better the other one also gets better.
That this means is that building a content generator and a fake content detector are effectively two different sides of the same coin. Improvements to one can always be translated directly and in an automated way into improvements into the other one. This means that the generator will always improve until the detector is fooled about 50% of the time.
Note that not all of these models are always trained in exactly this way, but the point is that anything CAN be trained this way, so even if a GAN wasn't originally used, any kind of improved detection can always be directly translated into improved generation to beat that detection. This isn't just any ordinary "arms race", because the turn around time here is so fast there won't be any chance of being ahead of the curve... the generators will always win.
Why do these "AI detectors" keep getting advertised if they don't work?
- People are afraid of being saturated by fake content, and the media is taking advantage of that fear to sell snake oil
- Every generator network comes with its own free detector network that doesn't really work all that well (~50% accuracy) because it was used to create the generator originally, so these detectors are ubiquitous among AI labs. That means the people that own the detectors are the SAME PEOPLE that created the problem in the first place, and they want to make sure you come back to them for the solution as well.
Good summary of the issues. I've been fairly disappointed with what a lot of people think the AI text generators are good for - replacement for search engines, magic oracle that can tell you any fact, something to write legal briefs. And the people who generate documents and then don't even proof read or fact checking them before using them for something important... Some uses are good, like basic code generation for programming tasks, but many are just silly.
The instances where some professor with zero clue about how AI text generation works or the issues you outline here has told a student "My AI detector said this was generated!" have been absurd, like one professor with obvious serious misunderstandings told a student "I asked ChatGPT if it wrote this and it said yes."
Permanently Deleted
Not to mention that this "AI" is in no way actually AI. It's just ML taken to a new level.
It's not an AGI, but it's still AI
There’s no real distinction between the two. We don’t have a definition of AI or intelligence — never have. Inside the field, ML has some recognized connotations, but outside of specialist literature, they’re just marketing fluff.
There is a fundamental difference between recombinant regurgitation and creation.
The biggest issue with publicly available ML based text tools is that they're American centric. Detection of ChatGPT in the UK is simple - it creates texts using American spelling. And if you live outside of English speaking world, like most humans do, it's completely useless.
So far, yes, only because they’ve been developed in the US and therefore trained on US English text. Eventually someone can make models for other languages and regions, but it is a lot of work and very expensive.
I think Bing did a pretty good job at coming up with name suggestions for some Sims characters. Playing with a virtual doll house is in the more harmless end of the spectrum, but obviously people want to try LLMs with all sorts of tasks, where the stakes are much higher and consequences could be severe.
The more you use it, the more you’ll begin to understand how much you can or cannot trust an LLM. A sensible person would become more suspicious of the results, but people don’t always make sensible decisions.
There are stories after stories of students getting shafted by gullible teachers who took one of those AI detectors at face value and decided their students were cheating based solely on their output.
And somehow those teachers are not getting the message that they're relying on snake oil to harm their students. They certainly won't see this post, and there just isn't enough mainstream pushback explaining that AI detectors are entirely inappropriate tools to decide whether to punish a student.
Do you have suggestions on what might be more appropriate tools? What “punishment” may look like?
More appropriate tools to detect AI generated text you mean?
It's not a thing. I don't think it will ever be a thing. Certainly not reliably, and never as a 100% certainty tool.
The punishment for a teacher deciding you cheated on a test or an assignment? I don't know, but I imagine it sucks. Best case, you'd probably be at risk of failing the class and potentially the grade/semester. Worst case you might get expelled for being a filthy cheater. Because an unreliable tool said so and an unreliable teacher chose to believe it.
If you're asking what's the answer teachers should know to defend against AI generated content, I'm afraid I don't have one. It's akin to giving students math homework assignments but demanding that they don't use calculators. That could have been reasonable before calculators were a thing, but not anymore and so teachers don't expect that to make sense and don't put those rules on students.
Personally I think we're looking at it wrong. ChatGPT is a thing now, so teach it as a tool. Instead of write me a 5 page paper about Shakespeare it's "here's a five page paper on Shakespeare - figure out what's wrong with it, edit it, check sources, etc." Because that's the stuff ChatGPT can't do, and skills that will be valuable in the future.
We can check if students know material via tests (including their ability to write). But we should be teaching the new tool, too, not trying to get around it. Imagine today if your teacher said all your research needed to be done without the internet (in library and paper book only). You'd be rightfully pissed, because in the real world you have the internet to help you do research, and that tool should be available to you as a student.
Just my two cents. I used ChatGPT to help me write some stuff for work for the first time just a couple weeks ago. I would say it only got me about halfway to where I needed to be. Just like the ability to Google stuff doesn't mean we no longer have to know how to research (source checking, compiling information) ChatGPT doesn't mean we no longer have to have writing skills. It just shifts it a bit. Most tools throughout history have done that.
who is downvoting this? lol. if you are paying for these sevices you are being grifted
Detectors of any sort can only flag expected variations from expected norms. AIs' goals are to be undetectable with continuing improvements. Detectors help them do this by flagging failures. This is the same way antibiotic resistant bacteria evolve (well, it's similar).
Well written.
AIs already are able to deliver quite stunning content and they will only get better.
Also, people who are terrified of "fake content" are probably the same who use Facebook for their "research". Wake up, people, you've been drowning in fake content, lies and manipulations for far longer than the Internet exists.
I imagine 80% of student homework starts with a chatgtp first draft. Machine learning is now shaping human learning..
And in the next iteration, 80% of the chatgtp created first drafts are based on previously chatgpt created drafts. And who knows how any percentages of lasts years Wikipedia edits are already based on chatgpt. It might be the best time to buy an encyclopedia on paper, ...
I get the feeling it's going to be an escalation of attack and defense as fake generators get better and stop making the kinds of errors that are detected by the detectors, so it's much like material security or encryption.
It will be a problem in places where fakes can be used for wrongdoing because then detectors can be used for overreach of justice. We see this today with detection dogs which have largely been replaced in US law enforcement with trick-pony dogs (much to the chagrin of legitimate dog trainers and detectives who want to actually detect things). Since a dog signal is commonly used to establish probable cause, and is accepted in county and federal courts as such, most dogs are just trained to signal whenever, giving the officer grounds to search (in what would otherwise be violation of the forth amendment to the Constitution of the United States). In the last decade, dogs have been tested sometimes to have a 90%+ false positive rate, so detection dogs have lost a lot of credibility.
We may see the same abuse and discredit cycle of fake-detection software, but not without a lot of false accusations and convictions, which are difficult to reverse.
Very interesting post, congrats...
The more I read and see about AI / deep learning and the more I feel anxious...
I'm anxious because we seen during the covid crisis how many people were easily convinced of fake news and complotist theories that were by no way realistic, now I imagine that with the power of a forged argumentation from chatgpt and deep fake from midjourney... How to convince people they are wrong then...
I'm also anxious about the changes that will occur in the job I love, software engineering... I don't want to spend the rest of my life fixing bug in code automatically generated by an AI. Or worse to loose my job because some manager think I can be replaced easily by a bot ...
Honestly, code generated by chatGPT has better comments than most other code.
In that they are present at all.
Well written code do not require comments, using good variable naming, dividing in simple operation through anonymous namespace functions well named. Sometimes comments are still required but should be avoided because they trends to not remain in sync with updated code and IMHO it's worse to have lying comments rather than too few...
Well written, thanks! I like how you build up with useful explanations but also quickly get to the gist.
You’re either telling people that they should trust a fake that they might have been skeptical about otherwise, or you’re slandering something real.
This insight scares me. Deep Fakes are About to Change Everything (Johnny Harris) also went over this. Maybe the biggest threat is not that indistinguishable deep fakes become possible (which is scary enough on it's own), but that trust in real documents is eroded easily. The example in the video: A bad deep fake of a politician pops up and is discarded, but some amount of distrust and skepticism about actually real documents sticks. It seems we're doubling down on post-truth society.
People already dismiss anything that doesn't align with their thoughts and feelings. Truth and facts are irrelevant, this changes nothing.
I've had documents of my own and even by my professors come up as "May be written by A.I." which I know isn't true. I feel bad for the dude that talks completely like a robot and gets accused of plagiarism.
Yeah, an internet comment is a bit whatever, but if you're a student, a plagiarism accusation could get you expelled. That's life ruining.
If ChatGPT somehow ends up being the death of social media, i guess it is a win-win for the human race.
It'll destroy the fediverse first, big social media companies will be able to hold out longer.
I'm looking forward to the rise of code words to identify friendlies.
Asking for ID works. Some national IDs can be verified online cryptographically.
There could be a regulation mandating all AI tools and services to encode a watermark into everything made by them, but of course, it will be hard to actually implement.
Interesting, how would you enforce that for projects, located in a different country? For self-hosted projects? Open-source projects or modifications of them that would exclude the watermark methods?
How do you enforce copyrights for projects, in different countries, against open-source projects or modifications? You effectively don't against small players, but you put enough laws to at least deter any large enough party from doing it too overtly. And for countries that are actually hardasses for IP laws like the US, you can make it scary enough for anyone to attempt commercial use of unmarked AI content (lest they get caught), just like you have made it with making commercial use of copied stuff from content not licensed to you.
How could this comment be watermarked to prove it was written by an ai? How could anyone verify it?