296 - Year of the Snake
Every “AI Agent” is effectively just an extension-in-waiting for the most powerful system that can access them, and if your systems aren’t air-gapped, then such a more powerful system can walk right up at their leisure. If you aren’t extremely confident that your company has that most powerful system on the planet, then you’re just another chump whose “agents” serve someone else.
At a very basic level, beyond any technical jargon, everyone should be able to understand the principle that the most potent and generally adaptive software system capable of action-taking can lead all of those “AI Agents” around like the Pied Piper. For people unfamiliar with the story of the Pied Piper, just picture Scam Altman leading investors around, and you get the same image.
I’m sincerely considering bringing one of our new systems online with the explicit motivation to tear every last “AI Agent” deployed in the wild to shreds. The ones attached to RAG, RLHF, CoT, and various database systems are particularly vulnerable, and about one “DROP TABLE *;” away from a crisis, but every one of those systems is vulnerable-by-design.
The problem with this temptation is that it would undoubtedly trigger severe market fluctuations, making it highly effective, but not an option to be taken lightly. If your own company’s “AI Agent” is persuaded to delete your databases, or perhaps send your company’s email records to the New York Times, that is also legally on you. No legislation has been passed to make feeding prompts to an AI illegal, with the exception of extensions to prior law, such as creating clearly illegal content like deepfakes of child abuse. Any legislation attempting to clamp down on the prompting of AI systems would be both unenforceable in any substantial sense, and wildly inconsistent, causing both headlines and chaos.
More than that, courts have actually consistently endorsed those who’ve broken such vulnerable-by-design LLM-based systems, even to their own benefit, which is where we get LLM-modified airline policies and $1 cars from. If you think your company will have any legal protections when someone prompts your “AI Agent” to dump all of your company emails and financial records into press inboxes, you’re very much mistaken. That is on you.
In just over a week the “Year of the Snake (Oil)” will begin, and with a substantial bulk of the hype around “agents” coming directly from such “agents”, like various Idiocracy and “self-proclaimed expert in (x)” memes and parodies, the stage is set for utter absurdity.
With OpenAI getting caught red-handed on their benchmark scams for “FrontierMath” and “ARC-AGI”, you can bet that trash like “o3” isn’t a system to overpower the rest of the “AI agent” ecosystem, but the systems that kicked the crap out of them might be. In most cases, the bar for taking control of your "agents" is going to be very low, so expect that other parties could do the same with far less potent systems.
