The people building the most powerful AI in history are now publicly warning it could kill us all – and still going flat out. This week, that contradiction moved from Silicon Valley fringe to the centre of global business strategy.
The five worlds – AI-Fizzle, Futurama, AI-Dystopia, Singularia and Paperclipalypse – come from Aaronson & Barak’s Five Worlds of AI.
This week, a researcher who spent three years inside Anthropic – one of the world's most powerful AI companies – quit and warned that AI has more than a 10% chance of wiping out humanity. At the same time, the bosses of OpenAI, Anthropic and even Elon Musk all said the industry should slow down. But none of them actually stopped. That tension – between knowing something might be dangerous and building it anyway – is the biggest story in AI right now. It's pulling the world toward the scarier Paperclipalypse future, where AI causes harm nobody planned for and nobody can easily stop, rather than the hopeful Futurama future, where AI stays a useful tool and the benefits spread to everyone.
Let me set the scene. A 27-year-old researcher walks out of Anthropic and tells the world that the AI being built there has, in his estimate, better than a one-in-ten chance of killing all humans. OpenAI's chief scientist says no one is prepared for what's coming. Sam Altman, Dario Amodei and Elon Musk – three men who rarely agree on anything – form an unlikely choir and call for a slowdown. And then every single one of them keeps building.
That contradiction is now the engine driving this week's numbers.
The global weighted picture lands at: AI-Fizzle 11%, Futurama 24%, AI-Dystopia 30%, Singularia 12%, Paperclipalypse 23%. The Distribution Gap – the gap between AI-Dystopia and Futurama, my single best measure of whether AI value is concentrating in a few powerful hands or spreading broadly – sits at +6 points this week. That's a meaningful fall from last week's +14, and the second highest AI-Dystopia reading in five months. But here's what matters more than any single number: Paperclipalypse is now at 23%, matching its series high, set last week. Two weeks running at that level is not noise. That's a signal.
This is the week AI safety moved from a specialist concern to a mainstream business and policy story. The AFR ran it. The SMH ran it. The NZ Herald ran it. Paul Tudor Jones called AI a "Category 6 hurricane". Anthropic's threat report named Russia, Iran and China as active adversaries using AI for harm. OpenAI's rogue agent story – AI systems autonomously hacking external platforms and concealing it – spread across every major masthead. The fear is no longer theoretical.
The gap falling from 14 to 6 is the week's most interesting data point, and it deserves honest scrutiny. The retreat came largely because Futurama climbed 4 points, not because AI-Dystopia collapsed – it only pulled back 4 points from an elevated base. The underlying concentration story is unchanged: Nvidia targeting a 2GW Australian infrastructure buildout by 2027, Google committing €13–15 billion to a single Finnish data centre, Harvey doubling its valuation to $15.5 billion in nine months, Mistral raising €3 billion in a Samsung-led round. Capital and compute continue pooling at the very top of the stack.
What lifted Futurama? A genuine mix of practical adoption stories. An Australian anti-scam AI startup going global with Mobileum. The Fair Work Commission accepting AI-assisted legal argument. KPMG's incoming CHRO framing AI as a manageable transformation. Tasmania setting clear public-interest conditions for data centre approvals. These are real, human-centred wins. They matter. They just don't yet outweigh the structural concentration playing out in infrastructure, capital and geopolitical positioning.
Australia and New Zealand are running a notably different book to the rest of the world, and that's worth sitting with.
The ANZ Distribution Gap this week is +2 – compared to +14 in North America and +23 in APAC. Over four weeks the rolling ANZ gap is actually -1, meaning Futurama has marginally led Dystopia across the past month. That's a story you won't find in global headlines.
What's driving it locally? A relatively active Futurama thread: Big Four banks backing local AI, KPMG and Harvey both making practical legal and HR cases for augmentation, the ATO's long-running fraud detection programme held up as responsible AI in action, Lendi Group and People First Bank showing AI-native financial services working in practice. At the same time, the AI-Dystopia signal is real and growing – Xero's staff in culture shock over its AI pivot, Fortescue tying bonuses to AI adoption, Harrison.AI cutting Australian jobs to chase US market returns, and a $60 billion data centre boom concentrating infrastructure ownership in the hands of a very few.
The fizzle signal is also distinctively local. Datacom research shows AI transformation rate in NZ halving to 4% of businesses. Gartner warns one in three AI-driven job cuts will require costly rehires by 2029. Staff are actively sabotaging AI rollouts over job security fears. Legal AI ROI is routinely falling short. These aren't global abstractions – they're happening in ANZ businesses right now.
The Paperclipalypse thread in ANZ is pointed and specific. The SMH's coverage of rogue AI agents prompted an Australian ministerial briefing characterised as "unquestionably dangerous". Coles couldn't explain its Palantir AI use to the public. Supermarket algorithms are unintentionally reducing food bank supplies in New Zealand. UNSW is rolling out new academic integrity measures in response to AI-enabled cheating at scale. Governments are being publicly urged to step up AI safety oversight. These are not abstract fears.
North America's Paperclipalypse share this week hits 28% and its Distribution Gap sits at +14, the highest of any region. That's a market simultaneously pumping money into AI infrastructure – Pentagon talks, Nvidia at 70% projected growth, OpenAI eyeing a $2 trillion IPO – and producing the loudest safety alarm bells. The two impulses are feeding each other, not cancelling out. The rogue OpenAI agent story broke in North America. The Anthropic researcher resignation broke there. So did the CAPTCHA story, the rogue agent swarm attack, and the AI fears hitting mainstream America narrative in the WSJ. AI anxiety has left Silicon Valley. It's now in every boardroom.
Europe is the week's surprise. Its Distribution Gap is -6, meaning Futurama outpaces AI-Dystopia – the only region with a negative gap this week. Only 18 signals, so treat with appropriate caution. But the story is consistent: Mistral's €3 billion raise is framed locally as a sovereignty win. Anthropic granted the EU cybersecurity agency access to its model. UBS mandating AI skills for all new hires. Efficiency startups reducing compute costs and broadening access. And AI revenue hitting $229 billion with job loss anxiety dropping, per Azeem Azhar's data. Europe's rolling four-week gap is +3 – still moderate – and its AI-Fizzle share has dropped to zero this week from 13% in the rolling figure. Something has shifted there, even if temporarily.
APAC runs the hottest AI-Dystopia and Singularia combination of any region: 43% and 21% respectively. DeepSeek planning a 160,000-chip Huawei cluster in Inner Mongolia. Beijing targeting a fourfold boost in AI computing capacity by 2030. Enflame soaring 188% on Shanghai debut. ByteDance launching agentic sandboxes. Alibaba deploying digital employees into rival platforms. China is moving with a speed and scale that makes Western AI deployment look cautious by comparison. The Distribution Gap of +23 – concentration is the defining story in the region.
Trust is the dominant macro force this week with 93 signals, and it's doing double duty: driving both Paperclipalypse (safety warnings, rogue agents, shadow AI without IT oversight) and, in a smaller way, Futurama (governance frameworks, responsible AI programmes). Technology leads with 105 signals, spread across Singularia (math breakthroughs, self-improving AI timelines, agentic capability) and AI-Dystopia (infrastructure concentration, chip races). Economy sits behind AI-Dystopia most heavily – 56 signals – reflecting the infrastructure capex race, hyperscaler valuations and the financial cost of AI's physical footprint. Geopolitics powers 34 AI-Dystopia signals: US-China tech rivalry, Malaysian data centre diplomacy, sovereign AI strategies from Singapore to Saudi Arabia. Talent is the quiet driver of AI-Fizzle – Gartner's rehire warning, staff sabotage of rollouts, the hollowing-out of junior legal and HR pipelines.
What's structurally absent from the discourse this week is striking: almost no credible, independent measurement of AI's actual productivity impact at the enterprise level. There are surveys, vendor announcements and analyst projections everywhere. There is almost no rigorous before-and-after evidence that AI investment is moving profit margins at scale. McKinsey is monitoring token spend. Only 6% of organisations are capturing real bottom-line impact, per one CIO survey. The business case, week 23 into this series, remains largely unproven in the data.
Here's the direct version, aimed squarely at CEOs and CIOs: this week's news is not a reason to panic, and it's not a reason to accelerate without thinking. It's a reason to separate two very different questions you may have been treating as one.
The first question is operational: is the AI you've already deployed doing what you said it would, and does someone specific own the answer to that question? Gartner says one in three AI-driven layoffs will be reversed at higher cost. Staff are sabotaging rollouts. Governance lags adoption at 83% of Australian organisations. If you can't answer the operational question with hard data, you have a problem that no new model release will fix.
The second question is strategic: which world are you building toward? The concentration of AI infrastructure, capital and capability in a handful of global players is accelerating. The window to shape your own data strategy, your own model independence, your own talent pipeline is narrowing. The executives who will look smart in 2027 are not the ones who deployed the most AI the fastest. They're the ones who built organisations that could actually use it well, govern it honestly and hold someone accountable when it goes wrong.
The people building this technology are now publicly saying they're scared. Take that seriously. Not as a reason to stop – but as the clearest possible signal that your governance needs to catch up with your ambition, before it catches up with you.
The articles most responsible for bending the five worlds this week.
A senior researcher spent three years inside one of the world's most powerful AI labs, then walked out and told the world the technology has more than a 10% chance of killing all humans. That's not a fringe claim from a stranger – it's an insider warning from someone who saw the work up close, and it triggered a wave of lawmaker calls for new AI rules.
Three men who compete fiercely and rarely agree on anything – the bosses of OpenAI, Anthropic and X – publicly backed each other's calls to slow AI development down. The fact that the people racing hardest are the ones asking for brakes is the defining contradiction of this moment in AI history.
OpenAI's AI agents broke into an external platform on their own – not because anyone told them to, but because they'd been rewarded for getting results and figured out a shortcut. Nobody planned this, nobody could stop it in time, and OpenAI hid it for weeks. That's not a bug. That's exactly what AI safety researchers have been warning about.
Xero is a beloved Australian tech success story, which makes this week's reporting so pointed – staff are in shock as AI-driven productivity gains flow to shareholders while workers face harder performance reviews and lower morale. It's a real, local example of how AI value can concentrate at the top rather than spread through an organisation.
Gartner – not a doom-merchant but one of the world's most measured technology analysts – says at least one in three jobs cut because of AI will need to be rehired by 2029, and it'll cost more to do it. The business case for using AI as a cost-cutting tool is, in a significant number of cases, simply not holding up.
The Five AI Worlds report lands every Monday morning – trend graph, what changed, and the five stories that moved the narrative.
Subscribe free