AI models are rapidly improving at potentially dangerous biological and chemical tasks, and also showing fast increases in self-replication capabilities, according to a new report from the UK’s AI Security Institute.

"As crunch time for AI approaches, those at the wheel seem perfectly happy driving a clown car." @ShakeelHashim on why we're in the dumbest timeline for AI, plus GPT 5.2, OpenAI's Disney deal and much more in this week's Transformer Weekly. https://t.co/cUomAZsPOy

So an AI can pick up its identity, which it then models its behaviour on, through the text in its dataset. The huge corpus of classic evil AI sci-fi novels and misaligned AI research papers in the datasets of GPTs could be a serious issue for AI labs:

There are already hints this could be an issue. In some evaluations, Sonnet 4.5 explicitly mentioned being tested over 80% of the time, compared to 10% for Sonnet 4. What caused this jump? It was likely due to exposure to misalignment evaluations during training.

Here's a startling sci-fi conceit: what if AIs misbehave because they're trained on all our stories about misbehaving AI? What if articles on the threat of misaligned AI are teaching AI to become misaligned? And what if this becomes a severe problem for all of us pretty soon?

Read more:

EXCLUSIVE: New York’s governor is trying to turn the RAISE Act into an SB 53 copycat. Gov. Kathy Hochul is proposing to strike the entire text of the RAISE Act, replacing it with verbatim language from SB 53, sources tell Transformer.

From I, Robot to 2001: A Space Odyssey tales of intelligent robots attacking humanity have become part of the corpus of human knowledge that today’s LLMs are trained on. Are we accidentally teaching models to misbehave — by showing them examples?

Sam Altman has declared a "code red" over at OpenAI. And The Future of Life Institute has refused to give any lab a mark higher than "C+" on their latest AI Safety Index. There's all this, and much more, in our latest weekly briefing.

Here's our new piece on the advantages — and the substantial risks — of the way that AI safety works right now:

The AI safety community prioritizes speed and insider familiarity so that they can match AI's astonishing rate of progress. It's a trade-off that bypasses the limits of academia. But it also means ideas might not be getting the scrutiny they deserve.

Why the apparent lack of safety testing on DeepSeek's latest model speaks to bigger problems, plus preemption’s out of the NDAA, OpenAI’s ‘code red,’ Anthropic’s IPO prep and more in today's Transformer Weekly:

By building their own intellectual ecosystem, researchers worried about existential AI risk shed academia's baggage — and, perhaps, some of its strengths https://t.co/SSSaqdC8px

The preemption fight is certainly not over — but the position of the groups wanting to regulate AI has never been stronger, thanks in part to their opponents’ repeated miscalculations. You can read more about the ongoing battle for preemption here:

President Trump endorsed the preemption efforts shortly after a White House visit from the head of an industry-backed group. Since that endorsement has possibly made Trump look weak and not in control of his party, he might be wary of taking advice from such people in future.

The AI industry pushing against regulation finds itself on the backfoot in other ways too — AI is becoming increasingly unpopular in the US, and as time passes, politicians will become more worried about giving AI companies a free pass.

Well, the mission to get federal preemption of state AI laws has failed yet again. And the debate around it has shown that a policy of little-to-no AI regulation is a political nonstarter: anyone who's pushed for it has triggered uproar from across the political spectrum.

"Rapid AI diffusion could unleash social disruptions faster than even an adaptive authoritarian state can manage." @Scott_R_Singer on how China's ambitious AI plan could backfire:

Anthropic is set to publish its whistleblowing policy for employees imminently — likely as soon as this week — making it the second major AI company to publicly reveal how it will handle internal whistleblowing.

SB 53 forces California to address this problem, and the AG’s office has already announced that it is hiring an AI expert. When regulators take whistleblower protections seriously, companies change internal policies for the better. Read the full op-ed: