AI's extinction debate breaks containment
· Axios
An online panic erupted this week after millions discovered that leading AI researchers routinely debate and calculate the risk of human extinction.
Visit newssport.cv for more information.
- Taken at face value, the odds are chilling: 10%, 20%, sometimes far higher, assigned to the possibility of AI catastrophe.
Why it matters: An esoteric debate over "p(doom)" has suddenly become a gut-level question for lawmakers, investors and millions of ordinary people: Why are we racing toward a technology its own builders believe could destroy us?
Driving the news: A resignation post by Jacob Coxon, an Anthropic researcher who previously worked at OpenAI, has drawn over 110 million views on X after he accused frontier labs of "gambling with our lives."
- The warning became vastly more explosive when Evan Hubinger, who leads Anthropic's Alignment Science team, publicly backed Coxon — saying "we really do earnestly believe AI could kill all humans."
- Hubinger said that, in his view, the odds of human extinction within the next decade are over 10%, and that Anthropic still has no plan for how to keep superintelligence under human control.
The big picture: Hubinger's post was widely cast as either a nightmare communications gaffe or a calculated dose of fear marketing from Anthropic, weeks before an IPO that could value the company at $2 trillion.
- In reality, his position has been echoed for years by AI's most influential figures: Geoffrey Hinton, the Nobel-winning "godfather of AI," has estimated a 10%–20% chance that AI causes human extinction.
- Elon Musk has put the risk as high as 20%. Anthropic CEO Dario Amodei told Axios last year that there's a 25% chance things go "really, really badly."
What they fear: "Doom" is a catch-all for several paths to catastrophe, from bad actors weaponizing powerful AI to advanced systems slipping beyond human control.
- Humans with superpowers: AI could sharply lower the barrier to designing biological weapons, launching devastating cyberattacks or crippling critical infrastructure.
- Machines with agency: A sufficiently autonomous system could deceive its overseers, copy itself, evade safeguards and resist attempts to shut it down.
- The nightmare scenario: AI begins improving AI faster than humans can understand or contain the process, eventually producing systems too capable for their creators to control.
Zoom out: The world underlying those p(doom) estimates has changed dramatically in just a year, as frontier AI has made breathtaking leaps in autonomy, cyber capability and its ability to accelerate AI research itself.
- Paul Christiano, the former OpenAI alignment chief who joined its board and safety committee Wednesday, said he shares safety fears about AI entering a rapid cycle of self-improvement.
- "If we build superintelligence without more robust alignment I expect we will permanently lose control of it," Christiano wrote. "If that happens then most people could die."
Between the lines: AI researchers speak a probabilistic dialect that sounds utterly foreign to most normal people.
- Inside a lab, a 10% p(doom) can be shorthand for enormous uncertainty about unprecedented technology. In ordinary life, almost nobody would tolerate that risk from a plane, a drug or a nuclear reactor.
- Imagine Boeing's head of safety telling you before takeoff that there's a one-in-10 chance the next generation of aircraft ends in catastrophe — and that Boeing still doesn't know how to make it safe.
What to watch: That is increasingly how the warnings are being heard outside the labs — as a safety alarm, rather than a probabilistic thought experiment.
- In Washington, the alarm is slowly intensifying: Sen. Bernie Sanders has proposed a ban on superintelligence and is convening senators for a briefing next week on AI's "extraordinary dangers."
- The bigger political risk is that Americans were already souring on AI over jobs, power bills and data centers long before extinction entered the mainstream conversation.