332 - Terms of Competence

Terms that everyone in a given domain should know:

Domain: Statistics (AI/ML, minus the "magical thinking"):

Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure"

Map–Territory Relation: "A type of reification fallacy where a model is confused with the thing being modeled."

Specification Gaming: "The behavior of artificial intelligence in working towards a poorly specified reward rather than the intended outcome."

Wittgenstein: The researcher whose name became synonymous with the inability to linguistically and logically define common terms and categories. The common example is tasking someone with defining what constitutes a "game".

Bullshit (Frankfurt): An indifference to the truth, which describes all common forms of AI/ML.

Domain: Cognitive Bias:

Collective Intelligence: The mechanism by which diverse groups reduce cognitive bias through convergence, usually either by averaging or iteration.

Noise: The observable presence of undefined, undifferentiated, and/or unmeasurable cognitive biases in arbitrary combinations, sometimes simplified to "The variability of error". This may be further divided into "System Noise", "Level Noise", and "Occasion Noise".

Hanlon's Razor: "Never attribute to malice that which is adequately explained by stupidity."

Domain: Systems Architecture, Ethics, and Governance:

Campbell's Law: "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures."

Cobra Effect: "When incentives designed to solve a problem end up rewarding people for making it worse."

Lucas Critique: "The observation that it is naive to try to predict the effects of a change in economic policy entirely on the basis of relationships observed in historical data." See Also N. N. Taleb on "Black Swans".

Sunk Cost Fallacy: The tendency to favor continuing to double down on an option merely because unrecoverable costs have been incurred by it, in the (emotional) hopes of eventually justifying those sunk costs.

Loss Aversion: The tendency to over-emphasize avoiding perceived losses relative to achieving potential gains.

Wittgenstein, Noise, and Collective Intelligence (again): See above.

• Feel free to test people who should know these, or add additional terms.

There will be a quiz later, and those who fail will have to Narfle the Garthok.