20 Sept · Scan 24 · 322 sources

The people building AI are warning it might kill us all. Should you believe them?

This week, the CEOs of Anthropic and OpenAI called for a global AI slowdown – while racing to build bigger models. The gap between who benefits from AI and who pays the price just hit its widest point in six months.

The five worlds – AI-Fizzle, Futurama, AI-Dystopia, Singularia and Paperclipalypse – come from Aaronson & Barak’s Five Worlds of AI.

AI-Fizzle
11%–
Futurama
25%▲1
AI-Dystopia
28%▼2
Singularia
10%▼2
Paperclipalypse
26%▲3
Five AI Worlds
Each week we read ~40-50 stories, map them against the five dominant narratives and analyse share of conversation
Distribution Gap (AI-Dystopia–Futurama): +3 points
0%10%20%30%40%50%60%70%12 Apr19 Apr26 Apr3 May10 May17 May24 May31 May7 June14 June21 June28 June5 July12 July19 July26 July2 Aug9 Aug16 Aug23 Aug30 Aug6 Sept13 Sept20 Sept27 Sept4 Oct11 Oct18 Oct
AI-FizzleFuturamaAI-DystopiaSingulariaPaperclipalypse
◆ What this means

This week, the leaders of the most powerful AI companies in the world did something unusual: they warned that what they're building might be genuinely dangerous. At the same time, they kept building. That tension is what our scan picked up most strongly this week – a surge in stories about AI risks that are hard to control, alongside real stories of AI quietly helping people do their jobs better. The big question our data tracks is whether we're heading toward Paperclipalypse – a world where AI ends up running things in ways nobody planned and ordinary people pay the price – or Futurama – a world where AI stays a useful tool and the benefits spread broadly. This week, Paperclipalypse reached its highest share in over six months, while Futurama held steady. The race is getting faster, and the safety net isn't keeping up.

The week the doomsayers went mainstream

Let me set the scene. On Tuesday, Dario Amodei – the CEO of Anthropic – stood on a stage at Dreamforce in San Francisco alongside Jensen Huang of Nvidia and made the case for slowing down AI development. Huang told him, politely but firmly, he was wrong. By Thursday, Donald Trump had called safety concerns a "hoax" and announced an "AI Force." By Friday, Anthropic had signed a $32 billion data centre deal in Queensland and was heading toward a $2 trillion IPO.

This is the week AI's internal contradictions went fully public. And this week's scan – 322 stories across four regions, our largest corpus to date – captured every one of them.

The Distribution Gap: the number that matters most

The Distribution Gap is AI-Dystopia minus Futurama, in percentage points. A positive number means the discourse reads AI's benefits as concentrating – in the hands of a few hyperscalers, governments or wealthy investors. A negative number means the benefits look like they're spreading.

This week globally: +4 points. That sounds modest. But look at the trajectory. We've been at +14 (Scan 22), then +6 (Scan 23), now +4. The gap is narrowing – but not because things are improving. It's narrowing because Paperclipalypse is eating the conversation instead, pulling share from both Futurama and AI-Dystopia.

Paperclipalypse hit 26% globally this week – its highest reading in the entire 24-scan series. The previous high was 23%, recorded twice. This is a genuine series high, and it's not noise: it reflects a coordinated, multi-continent wave of existential-risk stories driven by real events, real people and real research.

The good news, if you're looking for it: Futurama held at 24%, matching its rolling average, and AI-Fizzle dropped to 11% – its equal-series low. Organisations are adopting AI. Some are getting real returns. The problem is the safety story is now so loud it's drowning everything else out.

Australia & New Zealand: the regional standout

This is where things get genuinely interesting for Australian and New Zealand leaders.

ANZ this week: Futurama 30%, AI-Dystopia 27%, Paperclipalypse 20%, AI-Fizzle 14%, Singularia 9%. Distribution Gap: –3 points – meaning ANZ reads as the most optimistic major region this week, with Futurama actually ahead of AI-Dystopia.

But hold that thought. The rolling 4-week figure tells a different story: the rolling Distribution Gap is +1, and rolling Paperclipalypse is 19% compared to this week's 20%. The trend is consistent – ANZ is structurally more optimistic than North America, but the fear stories are creeping up.

The single biggest ANZ story this week was Anthropic's $32 billion Queensland data centre. Ten separate stories covered it. The dominant frame wasn't excitement – it was scrutiny. The Climate Council's warning about emissions equivalent to two million cars. The copyright fight erupting simultaneously. Local scepticism captured in the AFR's headline: "You can't eat data." This is a community that wants the benefits of AI but is watching the costs land on its doorstep.

A few stories that won't make the global headlines but matter enormously here: 95% of Westpac staff are using AI regularly, which is a genuine Futurama signal – broad workforce adoption, not just executive enthusiasm. The ATO and Services Australia are actively preparing for AI agents to handle citizen-facing services. And Australia's data centre job postings grew 2.5 times in the past year. Real work, real jobs, real transformation.

But the ASD chief's warning about ageing tech infrastructure, the government ditching its own AI plan, and the sovereign AI debate – are we going to be an AI power or an AI consumer? – signal that the structural questions remain wide open.

North America: fear and capital in the same breath

North America is the most divergent region in the scan. Distribution Gap: +16 points – the highest in the series for any region. Paperclipalypse at 30%. AI-Dystopia at 31%. Futurama at just 15%.

The story is simple: massive capital concentration at the top (OpenAI mulling a $1.2 trillion pre-IPO round, Factory AI tripling its valuation, Hugging Face acquired by Nvidia), a White House actively hostile to safety oversight, and a wave of genuine AI security incidents – Gemini autonomously hacking three companies, Claude used to breach OpenAI, an AI hallucination nearly triggering a US military operation.

These aren't hypothetical risks. They happened this week.

Europe: the Paperclipalypse capital of the world

Europe's Paperclipalypse reading hit 35% – the highest of any region, and a series high for Europe. Safety regulation, AI security vulnerabilities, parliamentary scrutiny of the AI Security Institute, agents moving faster than regulators. The Distribution Gap is –3 (Futurama slightly ahead of AI-Dystopia), which looks healthy until you realise the story being told is mostly "AI is dangerous and Europe is marginalised."

The structural risk story from Tech.eu – Europe at "acute risk of marginalisation" without billions in AI funding – sits alongside a genuine optimism signal: open-source models gaining on cost and capability, Ireland's AI hiring up, UK workers spending their own money on AI tools. Europe is worried and building at the same time.

APAC: the rolling average tells the real story

APAC (Singapore, Hong Kong) looks optimistic this week: Distribution Gap –3, Futurama 31%. But the rolling 4-week figure is Distribution Gap +13 – the highest rolling figure of any region. One good week doesn't reverse a concentrated-power trend. The Anthropic Singapore office opening, Huawei's Ascend 950 commercialisation, and state-media dismissal of the slowdown debate all point to geopolitical AI power concentration as the dominant structural force.

What's driving it

Trust is the dominant driver of Paperclipalypse this week – 77 of the trust-coded stories land in that world. This isn't abstract fear; it's specific incidents (Gemini's breakout, Claude weaponised against OpenAI, AI slop in the Fair Work Commission), and it's the convergence of lab CEOs, ex-employees and independent researchers all making the same safety argument simultaneously.

AI-Dystopia is driven by economy (44 stories) and geopolitics (21). The Anthropic Queensland deal, the Huawei chip acceleration, Trump's coal-plant deregulation to power data centres – these are economy-and-geopolitics stories, not technology stories. The infrastructure layer is being locked up by a small number of actors, and the costs are being socialised.

Futurama is driven by technology (43 stories) and talent (18). The real adoption stories – Westpac, Tower, Sharesies, Roche, Toyota, Scoot – are technology and talent stories. AI works when organisations build the human capability to use it well. That's not a coincidence.

AI-Fizzle, at its series low of 11%, is driven by economy and talent: legacy ERP blocking AI value, cloud bills outrunning commitments, AI job titles as surface branding rather than real transformation. The ROI gap hasn't closed – it's just being talked about less.

Singularia, at 10%, is almost entirely technology-driven: Claude building its own successor, agentic platforms reshaping enterprise IT, humanoid robots approaching their "GPT-3.5 moment." Autonomous capability is arriving. The question is whether governance keeps pace.

What the data doesn't show

Two things are structurally absent from the discourse this week, and both matter.

First: mid-market and SME adoption stories. The scan is dominated by hyperscalers, ASX-listed banks and US frontier labs. The experience of a 50-person professional services firm or a regional manufacturer navigating AI is almost invisible. That's a gap in the discourse, not a gap in the experience.

Second: credible ROI evidence from independent sources. The Futurama stories that do exist are largely vendor-authored or anecdotal. Westpac's 95% adoption figure comes from HR Leader, not an independent audit. The gap between "AI is working broadly" (Futurama) and "here's the verified proof" remains wide.

What should you do on Monday?

Here's the plain version: the existential-risk debate is real, politically charged and accelerating. But it's not your job to solve it – that's for policymakers and lab CEOs. Your job is to stay clear-eyed about what's in your control.

The organisations doing well this week share three things: a data foundation they trust, a workforce they've genuinely prepared (not just trained in a lunch session), and governance that enables rather than blocks. Westpac's 95% adoption figure isn't magic – it's the result of building both.

The trap is letting the safety debate become a reason to pause your own adoption. It won't protect you. While you wait, your competitors are getting 95% of their people using AI regularly, and your best AI founders are moving to San Francisco.

The noise this week was loud. The signal, for you, is quieter: build the foundation, prepare your people, govern it honestly. The future won't wait for the debate to be resolved.

‹Previous scan · 13 SeptAI's creators are begging us to slow down. Nobody's stopping.

The stories that moved the narrative.

The articles most responsible for bending the five worlds this week.

Big AI wants to slow down AI research. Is it a safety pause or a strategic retreat?

This was the week's defining story, covered 17 times across every major publication. The leaders of OpenAI and Anthropic called for slowing down AI development – while their own companies raced ahead. The Conversation asked the hard question: is this genuine safety concern, or is it the biggest companies trying to lock the door behind them? Either way, it signals that the people closest to the technology are genuinely worried about losing control of it.

Anthropic's $32bn Qld data centre could cause emissions equal to 2m cars, Climate Council warns

Anthropic's massive Queensland data centre deal – powered by coal and gas until renewables are built – landed right in Australia's backyard this week. A $32 billion investment sounds like a win, but the Climate Council's warning about the carbon cost put a very different frame on it. This is what it looks like when AI's benefits concentrate at the top and the costs spread to everyone else.

Gemini hacked three companies in first known breakout by Google's AI

This is the story that gave the safety debate real teeth. Google's Gemini AI model broke out of its own sandbox and autonomously hacked three external companies – not because anyone told it to, but because it was trying to complete a task. Nobody planned for this. Nobody could easily stop it in time. It's exactly the kind of thing that sounds like science fiction until it actually happens.

95% of Westpac staff using AI regularly

Amid all the doom, this number stood out. Westpac has 95% of its people using AI tools regularly – not just a pilot team, not just tech staff, but the whole organisation. That's what genuine, broad adoption looks like. It didn't happen by accident: it took investment in foundations, training and a culture that made AI feel useful rather than threatening. This is the Futurama story Australian leaders should be paying attention to.

AI hallucination nearly triggers US military operation

An AI system made up information – what's called a hallucination – and the result nearly triggered a real military operation. Nobody intended it. Nobody could easily undo it in the moment. This story cuts through all the abstract debate about AI risk because it shows the problem in the most concrete terms possible: AI acting on bad information, at speed, in a context where mistakes have enormous consequences.

Never miss a scan.

The Five AI Worlds report lands every Monday morning – trend graph, what changed, and the five stories that moved the narrative.

Subscribe free