311 - Anomaly Detection

"Anomaly Detection" may be correlated with "Fraud", but it is far from anything that can automate "Fraud Detection". Rather, what fully automated systems built on anomaly detection do in practice is to automatically harass and all too often maliciously discriminate against anyone and any activity that falls at the tail end of any statistical distribution that it optimizes over.

This makes any such system a massive liability if not both properly supervised and properly configurable. Even if 99% of people have 99% of their activity registered as normal at any one time, eventually a majority of people will face unusual circumstances. Some less typical people will face those circumstances with orders of magnitude higher frequency, and for them, such anomaly detection systems often need to be selectively disabled, as otherwise those systems break or lock up completely.

UX nightmares stick with people for years or even decades after they occur, and anyone who travels the world is going to have a vastly higher rate of those UX nightmares directly thanks to naïve Anomaly Detection systems, poorly configured and poorly implemented. The more regulations are present in an industry the more severe these UX nightmares often become, and the less likely they are to ever be resolved.

Companies also aren't nearly as shielded from the consequences of their actions as they pretend to be, even where simple lawsuits are concerned, particularly when they systematically discriminate against people based on very poorly and naively implemented Anomaly Detection systems. Systems Architecture can be systematically discriminatory too, not just the narrow AI models that some will contain.

Take the finance sector in the US for example. It has been thrown into regulatory hell (particularly among expats), following the example of certain countries in the EU, to the point where the practical solution is quickly becoming to collapse entire US-based entities and re-establish them in more competent jurisdictions elsewhere in the world. I've had to give this advice in the past 24 hours, and I'm sure I'll have to give it again, and again until the 250 years' worth of collective bureaucratic sludge are eventually erased by more extreme events.

Anomaly Detection isn't a silver bullet, it is a cognitive bias built from custom data, and when it is wrong, it will be systematically wrong. If you choose to implement it naively then you too will be both systematically wrong and systematically liable.

Consider probabilities without sacrificing edge cases, as edge cases often determine the future of your business, or lack thereof.