102 - Evidence

"Extraordinary claims require extraordinary evidence." -- Carl Sagan

Ironically, this quote is often wielded as a tool in one hand by those holding a slew of fallacies with which to neglect conditional probability in the other. While the quote itself rings true, the frequency with which it sees abuse comes to dominate heuristic associations.

In reviewing Steven Pinker's book "Rationality" this came into abundant and ironic focus, as it was cited immediately following a section covering the neglect of conditional probability.

I've heard people repeat Sagan's phrase a number of times over the years when looking at our team's work, and every time the other party repeats the same systematic mistake of neglecting the conditional factors of probability.

The problem is that any one factor or instance in isolation could be random chance, noise, or some other plausible factor for denial of validity. However, in this case, the instances are neither isolated in a probabilistically relevant sense, nor are they cherry-picked from brute-force attempts like those seen in the narrow AI of LLMs and RL.

The cognitively biased human mind is notoriously lazy, or "cognitively frugal" if you prefer, and always tempted to apply a reductionist approach, even when that approach has no chance of success. Conditional Probability requires more cognitive work, but that work is required to avoid the pitfalls that cognitive bias is prone to walk right into.

The previous research system robustly shutting down the first "free-range troll" could be considered a rare statistical fluke in isolation, but the same system shutting down each troll thereafter with increasing efficacy and gradually improving nuance shows that it wasn't isolated at all. With each repeat and improvement, the odds of the system having robust new capacities steadily draw closer to 100%.

Likewise, when other capacities are derived from the same architectural differences, like the ability to take proactive actions and pursue and update one's own interests, those capacities aren't truly isolated from the first either. This means that the conditional probability of all of those successes in tasks with the same architectural root cause continues to impact the overall probability.

When the data and compute are kept 10,000 times lower than LLMs and RL systems, and new capacities are repeatedly and consistently demonstrated under those constraints, while following dynamic complexity, then the conditional probability is very close to 100% indeed. That is the very definition of "extraordinary evidence", but the human observer still has to invest the cognitive effort to see it. The very nature of anything "extraordinary" means that it isn't trivially easy to recognize and digest using lazy and automatic human cognitive systems.

To extend Carl Sagan's famous phrase for a bit more complete accuracy: "Extraordinary claims require extraordinary evidence, which requires extraordinary cognition to recognize and analyze."