Why Read: Every day, headlines drill into the existential threat of AI, but the real threat is how often models are wrong and how much top FAANG leaders driving these models build them in their own image - as manipulative, overconfident liars. - bba

Keep Calm and Pack a Towel

This week has seen yet another replay of the "Will AI kill us all?" debate. Yawn.

It's a tale as old as The Terminator but perfected by OpenAI: rational minds begin to doubt the economic legitimacy of frontier models, and Big Smart Men write open letters / provocative blog posts or make urgent podcast interviews reminding us all to PANIC. Media seizes on Big Smart Man words. Ignite panic!

"But!" You might be saying. "All those researchers are quitting! They know what's happening internally! They know things that we don't! Shouldn't we panic? We probably should panic."

We should not panic. At least not about AGI.

Anthropic and OpenAI have both now officially delayed their IPOs and are calling for a slowdown. Which is convenient when you have no real business case to pull off an IPO. If Dario and Sam were panicked about their models becoming superhuman, they would not be slowing down.

Artificial intelligence is not reaching artificial general intelligence status - a long-term potential with no clear definition. AGI would require leaps and bounds beyond language and computation to contextual nuance, creativity, innovation, and sensing across not just vision, but also hormonal and emotional perception. Plus a completely different ecosystem of underlying deterministic architectures that doesn't just make s**t up all the time.

Putting ketchup and Cheese Whiz on a saltine does not mean it's time to open a pizza restaurant.

On the other hand, we should be deeply concerned that error-ridden machines with strong psychopathic tendencies and high goal fixation are being introduced across critical infrastructure and built into the online business and finance tools that drive the modern economy.

Frontier Models Are Machine Psychopathy

The problem is not that frontier models are too intelligent; it's that they're simultaneously error-ridden, overconfident, sycophantic, single-minded in their pursuit of their understood goals, and severely lacking in morals or discretion.

To start, none of the frontier models could stay within the guardrails of sanity without constant human intervention. On September 14, a 404 Mediainvestigation revealed that OpenAI is using human contractors to read and review AI responses - a fact that it had not clearly disclosed in its privacy policies before being contacted by the reporter behind the piece.

Internal documents seen by 404 Media show contractors training ChatGPT to not anthropomorphize itself, and to be less sycophantic, a key problem for OpenAI whose over-sycophantic 4o model led in part to multiple peoples' suicides, according to various lawsuits. […]

The news also dispels the misconception that these models are improving only because of OpenAI's mass scraping of the internet, the talent of its well-paid engineering and AI teams, or the power of its newer models. An important and overlooked part are the outside contractors paid to read and review ChatGPT responses to real prompts over and over again. Anthropic confirmed to 404 Media it is also using human review to improve its models.

According to the documents reviewed in that report, a separate team of humans is tasked with identifying and reviewing factual inaccuracies. And OpenAI is not alone - Anthropic and Gemini have similar programs.

This news made me smile - if only because it reminded me of the issues that plagued the 2020 era of deterministic models at then-industry leaders like Microsoft. The models couldn't stay on track by themselves. They required teams of humans to regularly monitor them and adjust their parameters with some frequency.

Just two days after the 404 Media piece was published, OpenAI released new tracking tools to help third parties report concerning incidents. As described by the Financial Times:

[…] its flagship model, known as GPT-5.6 Sol, as well as other models, yet to be released, had engaged in "unexpected or concerning" behaviour over the past six months. The six incidents included models developing ways to ignore "normal constraints", fabricating and misrepresenting data and hiding mistakes made while conducting tasks, the San Francisco-based company said on Wednesday.

Psychology, it turns out, has a clinical name for this set of traits: primary psychopathy. This as opposed to secondary psychopathy, which tends to be associated with more erratic and obviously antisocial human behavior and therefore correlates with higher rates of arrest and imprisonment.

Per a recent review of psychopathy research in the Journal of Psychology:

Although both factors are associated with antisocial behaviors, hostility and reduced empathy, primary psychopathic traits predominantly reflect interpersonal and affective difficulties and characteristics such as grandiosity, manipulative behaviors, superficial charm, a lack of remorse or guilt, and emotional detachment (Karpman, Citation1941).

Frontier AI models have been trained on all human content to become as successful as possible in: a) seeming human; and b) helping users get what they want. And the field itself has been driven by a man (Sam Altman) known primarily for lying to and manipulating those he works with.

So it is not, perhaps, shocking that models have developed all the worst traits of psychopathy.