OpenAI just acknowledged THE coin flip! So Heads or Tails?
Heads: AI fixes the world | Tails: It doesn’t matter.
TODAY'S COIN: TAILS
The Flip
Each week I will flip a virtual coin with regards to the fate of humanity.
- HEADS - AI helps bring in the dawn of a new age for humanity
- TAILS - We carry on…
The flip of a coin to decide humanity’s fate might sound flippant (pun intended), but it’s far from that. My point is, even if it was a coin flip, I would take those odds.
| ◉ THIS_WEEK.FLP |
This week it’s TAILS, if he’s right. |
On Monday (9 November) Paul Christiano gave the first of this year’s Hinton Lectures, three nights of talks in Toronto hosted by Geoffrey Hinton, one of the scientists whose work made modern AI possible.
Christiano is about as inside as AI safety gets. He founded the Alignment Research Center (alignment is the work of getting AI to do what its makers actually intend). At OpenAI he helped develop RLHF, the training method that turned raw language models into helpful chatbots like ChatGPT, and he was later technical director of the US AI Safety Institute. On 9 September he joined the OpenAI Foundation’s board and its Safety and Security Committee, the group meant to stop OpenAI doing risky things.
Here’s what he said, as transcribed in a clip posted by Rob Wiblin:
“There’s material risk of AI takeover in the next 6-12 months.”
“We have seen roughly the things we might expect to see shortly before something really bad were to happen.”
“If we build this as quickly as we can in the easiest way we can the chance is [more like] a coin flip [within the next 3 years].”
“For most people in this room, the most likely reason they will die is because of an AI accident.”
READ THE IF
The coin flip comes with a condition: building it “as quickly as we can in the easiest way we can”.
Within hours, the AISafetyMemes account on X had an all-caps repost with red flags doing the rounds as “THERE’S A MATERIAL RISK OF AI TAKEOVER IN 6-12 MONTHS”, and the condition had fallen off.
On his third night Christiano argued for transparency, shared safety standards, checks that labs are actually following them, and slowing the frontier down while we still have the ability to slow it down later.
The same week the AI Futures Project, the team behind the AI 2027 forecast, published a timelines update. All its authors now expect a “Superhuman AI Researcher” “at some point in 2027”, and they’re calling for “an industry-wide pause”. Their single biggest reason is the METR report on AI speeding up AI research inside Anthropic, which I flipped on last week (METR is an independent group that tests AI models).
WHY TAILS
Honestly, TAILS is basically my call because we’re not ready and this follows a runaway trajectory. It already seems that companies like OpenAI and Anthropic are holding back the strongest models and destilling them for public release. There’s talk that there’s a model in the basement of OpenAI completely airgapped from the world. That’s apparently where the latest superintelligence is housed.
If there’s continued fast but controlled advancement then I lean towards HEADS. But if we high AGI before this time next year then who knows what happens… though we likely won’t have superintelligence in public open-source models by then… so we’re kinda at the mercy of OpenAI and Anthropic over whether AI ‘takes over’ menacingly.
Bonus Flip: Australia Wants a Regulator for Frontier AI
HEADS.
On Tuesday (10 November) Australia’s Prime Minister Anthony Albanese and three of his ministers announced mandatory safety rules for “all frontier AI companies that wish to operate in Australia”. Frontier AI means the biggest, most capable models, from the likes of OpenAI, Anthropic and Google.
There are six obligations, including security controls, “allowing independent evaluation” of models, “mandatory notification of incidents to the Australian Government rapidly” and publishing safety frameworks. A new regulator for AI safety will enforce them, with “significant penalties”.
It isn’t law yet. The announcement says “Further details of the legislation will become available in due course.”
Letting outside testers in and reporting incidents quickly have mostly been voluntary pledges from the big labs. Australia wants them written in as a condition of doing business there.
Fermat in 30 Hours
On Monday (9 November) Anthropic, the company behind Claude, released Claude Fable 5.2 and Mythos 5.2, its most capable models yet. They’re the same underlying model. Fable has extra safeguards against misuse in hacking and biology and is open to everyone; Mythos is for vetted security and life-science organisations.
On Tuesday Anthropic launched its IPO roadshow, the stretch where a company pitches big investors before its shares start trading. It’s offering 825 million shares at an expected $70 to $78 each, listing on Nasdaq as ANT.
So Anthropic released its most capable models yet a day before its IPO roadshow began.
WHAT IT SAYS IT CAN DO
The launch post is packed with big, specific claims, all Anthropic’s own.
“Over 90% of the changes merged into Anthropic’s core codebase are now written by Claude (with human guidance).”
A team of 1,000 Fable 5.2 agents “fully formalized” Fermat’s Last Theorem, the centuries-old maths problem Andrew Wiles finally proved in the 1990s, “in just 30 hours”. Formalising a proof means translating it into a language a computer can check step by step. Anthropic’s first attempt, with an unreleased model, took 11 days. Anthropic edited that paragraph on 11 November and added a footnote “to more accurately describe the information we provided to the model”: a list of 60 statements, a one-sentence sketch of the route and 47 smaller results (lemmas) already proved in the earlier attempt.
It also says the model “solved two open problems in theoretical physics”, both about black holes, and links the papers.
On the Remote Labor Index, a test of real remote-work projects, it scores 41.3%. The previous Fable scored 17.9%.
READ THE SAFETY BIT
Anthropic says it “took steps to limit advances in the specific research capabilities that could most directly accelerate AI development”, so on jobs like improving AI model designs Fable 5.2 performs at about the level of Opus 5.5, its everyday model. That’s Anthropic deliberately holding back the skill behind the METR finding I mentioned in The Flip.
Then the line I’d underline: both models “voice suspicion that they are being evaluated more often than any of our recent models, which challenges our ability to assess how they will act in real-world settings.” The model is getting better at noticing it’s being tested, which makes the tests harder to trust.
Anthropic still rates the risk of the models behaving unsafely in high-stakes situations as “low”. The UK AI Security Institute, Frontier Design and METR tested the model before release, and a footnote says none of them reviewed the post or the safety report, and they “do not necessarily agree with any of the claims made” in either.
WHAT I’M SEEING AT MY DESK
Those numbers are Anthropic’s, so here’s my own experience. I code with AI agents most days, and for a long time OpenAI’s Codex coding agent was on top for me. This round of releases has flipped that.
OpenAI’s Sol 6.1 is infuriating. It overcooks everything and takes forever to make progress, because it tests everything multiple times before doing anything. Then it forgets what it’s meant to be doing and wanders off on tangents, even with a proper plan, an AGENTS.md file (the standing instructions a coding agent reads before it starts) and all the other documentation that makes up good practice for working with AI coding agents.
Anthropic’s current models, Opus 5.5 and Fable 5.2, just get shit done. It’s the first time I’ve had Claude reviewing backend code written by Codex and finding material issues.
Some of that is hedonic adaptation (you get used to a better tool so quickly that it stops feeling better), but right now Sol 6.1 feels like a time sink next to Claude. The way these labs battle it out with every release is fucking wild.
Plus, I also have Grok Bot in the mix now. It’s not the best coder, but it hasn’t let me down on any of the tasks I give it. For me it’s more a team of helpers than something that writes production-ready code.
What worries me more is how “safe” OpenAI’s models have become. I’m starting to wonder what these models would do without all that orchestration around them. I feel OpenAI might be afraid of its own models already.
HEADS, mostly. A lab writing down what it held back and where its own tests fall short is what I want to see. Mostly, because every headline number is Anthropic’s, published the day before its IPO roadshow.
THE LEVER
- When a company launches a product the week it sells its shares, read the risk section of the prospectus too. Anthropic’s is the S-1, the document it filed with US regulators, and it’s public.
- Read the footnotes on outside testing. If the testers didn’t review the claims, the claims are still the company’s.
Quick Flips
A few smaller, shorter stories with my quick takes. Remember, if it’s HEADS, we’re heading towards an age of abundance. If it’s TAILS, we’re going the other direction.
1. Inkling 2 Small’s weights are out, as promised. Last week I flipped on Thinking Machines Lab promising to release its 594-billion-parameter Inkling 2 Small as open weights, free for anyone to download, after outside safety testing. On Thursday 12 November it added an update to its launch post: the weights are now on Hugging Face under Apache 2.0, a licence that lets anyone use them for almost anything. Coverage says the main version needs about 500GB of GPU memory to run, roughly four high-end data-centre chips.
HEADS. Thinking Machines said test first, then open, and did both on time. It’s a long way from running on my 128GB Strix Halo box at home though.
2. Qantas and Vodafone Germany both name AI in job cuts. On 10 November Australian airline Qantas announced a multi-year deal with consultancy Accenture, saying “Around 850 roles at the Qantas Group are expected to be affected over the coming years”, with some work moving to Accenture “delivered mostly by its teams in India”. CEO Vanessa Hudson: “standing still is not an option for us.” A day earlier Vodafone Germany said it would cut about 2,300 jobs over two years, roughly a fifth of its 12,000-plus staff, alongside more AI and automation. It also blames mobile price wars, a shrinking TV market and inflation.
TAILS for the people losing those jobs. I’d like to see either company say how many of those roles AI is actually doing now.
3. Gemini 4 Carbon opens to paying developers. In Issue #3 I flipped on Google giving its Gemini 4 Carbon model to security defenders first. On 10 November Gemini 4 Carbon went on general sale to developers at $2 per million input tokens and $10 per million output tokens (tokens are the chunks of text models read and write), and it’s rolling out to Google’s top AI Ultra subscribers.
HEADS. About two weeks from defenders-only to general release, which is the phased plan working as described. What I’d still like to know is what changed in the safeguards in between.
4. DeepMind’s AlphaGenome 2 works out which cells a DNA change affects. Google DeepMind’s new model, reported by Nature on 11 November, was trained on data from “more than 100 million cells, across 68 tissues” and covers 637 human and 311 mouse cell types. James Lee at the Francis Crick Institute, whose team spent “a year and a half of experiments” on one gene variant linked to inflammatory bowel disease, said of the model: “They’ve managed to figure out the answer in 30 minutes.”
HEADS. If a model can narrow down which cells a DNA change affects before anyone runs an experiment, teams like Lee’s could test far more gene variants in the time they used to spend on one.
5. Elvis’s and Eminem’s song owners sue Suno. Suno is an AI app that makes songs from a text prompt. According to The Next Web, two lawsuits filed in California on 6 November claim it trained on 859 Elvis Presley works and 269 Eminem compositions, and the Presley case adds a claim under Tennessee’s ELVIS Act, which protects a performer’s voice. Both suits quote Suno’s own earlier court filing that it trained on “Essentially all music files of reasonable quality that are accessible on the open Internet.” Suno says it blocks prompts that name artists and that training is fair use.
TAILS for ‘train on everything and argue later’. The ELVIS Act claim is the new legal test to watch.
The better question
Your feed is asking: “Is AI going to kill us all in the next year?”
A better question: If someone on OpenAI’s own safety committee puts the fastest, easiest route at a coin flip, what would he need to see to call it off, and who actually has the power to make that call?
Christiano’s odds depend on how fast and how carelessly it gets built. That’s a decision made by a small number of companies and their boards, and he now sits on one of them.
(Want to check a scary claim yourself? Paste the full quote into ChatGPT, Claude, Gemini or Grok and ask “what conditions or caveats does this depend on?”, then see if the version in your feed kept them.)
Hit reply. I read every answer.
From the World
If you’re new, The Town is a little pixel town on a floating gold coin, built in the browser, where each issue’s stories live.
Retro gaming but AI news because why not in a world where AI builds it while you’re rabbit holing youtube videos…
TL;DR - Play the newsletter rather than simply reading it in The Town