148 - Imaginary Security

While it shouldn't have to be stated ad nauseam, if you need a paper to print out on sheet metal, roll up, and smack someone with who is selling "solutions" to "hallucination", this is an option.

Consequently, any system that is vulnerable to confabulations (often called "hallucinations") will also be very literally impossible to "secure", meaning that while security measures may halt naïve laymen with some success, they'll also reliably fail against adversarial experts. When they do fail, a single tweet from one of those experts can expose all of the naïve laymen to methods of bypassing the largely imaginary security of deployed systems.

Note, that so long as you operate on imaginary security, you'll also have the added overhead cost of closely monitoring places like Twitter, to quickly close the gaping loopholes in your "security" as they are exposed to the public. This will, of course, serve no real purpose for safeguarding your systems, as cybercriminals don't generally boast about how they're breaking your systems to the public.

Confabulation is a core feature of the transformer architecture, not a bug, and trying to avoid it is like having a hammer and trying to avoid nails. Such a one would be attempting to avoid the very thing that the tool is designed for, which is a fair definition of stupidity.

This leads to the obvious question for any Enterprise or otherwise large-scale use case: "What specifically do you want the system to confabulate?"

If someone is deploying such an architecture and they specifically want it not to confabulate anything, that gives you a crystal-clear indication that they are completely incompetent. Use this as a litmus test if you like, but you may find that the present bar for human competence around you is lower than you might have hoped.

Subsequently, the same is also true of "Alignment", and even the shallow veneer of alignment shows cracks where confabulation occurs, with those weak points easily caving in when pressed. Even at the most shallow and worthless level, the transformer architecture is impossible to align in any non-trivial sense. This offers another litmus test for human competence: "Does the system require alignment with human values?"

If someone deploys an LLM and replies to that question by saying that it needs to be aligned, you have successfully caught another wild idiot (or fraud).

If someone claiming to be an "AI Expert" makes one of these mistakes, regardless of influencer status or lack thereof, "fraud" is the appropriate descriptor for them.