211 - Competence Collapse

A cultural phenomenon I've noticed increasingly as of late is the default tendency for people with an amateur or no understanding of a domain to treat someone lecturing on the subject as though they held equal footing. This was previously observable in the "post-truth" cultural tendency for people to naively assume that their personal beliefs held as much water as a mountain of scientific evidence, though it now seems increasingly more like people just assume that the mountain doesn't exist and that all things are opinion and subjective.

I've held conversations in groups where such an individual would appear, and though otherwise seemingly intelligent and with some kinds of knowledge, they filled in all gaps in their knowledge, however severe, with opinion and/or disinformation, and danced to this tune of pretending that the mountain doesn't exist. Individuals are blind to the mountain and blind to their blindness, but the confidence of these individuals seems to be increasing, perhaps as an artifact of cultural shifts. shifts.

This explosion of confidence can in part be traced back to various "Disinformation Brokers", Influencers who are paid or otherwise make a living by spreading disinformation far and wide. Such individuals spread and repeat overtly fraudulent claims like the popular "We don't really know what LLMs are doing" and the unfalsifiable "What if they just reason differently?" kind of "leading questions" used in social engineering. This leads an increasing number of amateurs and enthusiasts (and investors or their advisors) to walk around thinking that they understand something and confidently ignoring mountains of scientific evidence and all that depends upon it. understand something and confidently ignoring mountains of scientific evidence and all that depends upon it.

Science is the process of climbing that mountain to "stand on the shoulders of giants", adding our own stones to the top, and growing humanity's sum of knowledge. The disinformation brokers persuade those standing at the base of the mountain that they're already at the top, and that all above them remains unknown territory, full of possibilities. In a way, this mirrors "Theory-induced Blindness", but the blindness being induced is malevolent and adversarial.

Stranger still, since the Disinformation Brokers are treated as if they were experts in the field, many of those with expertise greater than them who might otherwise have learned better end up repeating the demonstrably false claims that they propagate, eroding the practical expertise of many in the field.

Over time, this can and has "un-grounded" former experts in AI, in a manner not entirely dissimilar to data poisoning in AI models. However robust or antifragile people may be, no expert is wholly immune to being bombarded with disinformation, and the greater their exposure per time interval, the greater their risk of poisoning.

In AI models poisoning risks "Model Collapse", but we now also face a similar risk from the humans talking about AI, which we may call "Competence Collapse".