I'm confused why global AI coordination is so strongly associated with a concentration of power. Global coordination => no single leader => diffusion of power?.

Which phrase for "an international AI slowdown before superintelligence" would you want to be popularized?.

RT @inferencemag: Inference is hosting some of the world’s leading experts for a debate on the possibility and potential consequences of au….

It's not obvious to me that developers will need this level of precision. But I think it's helpful to write the most thorough version of a safety case as a starting point for simplifications and relaxations.

We err on the side of crossing i's and dotting t's—for instance, including quantitative risk estimates and models of uplift. See this fun interactive website that explains our model:.

How do we know if AI systems are safe from misuse?. The short answer: have a red team try to misuse them, and measure the effort required. But the devil is in the details—which I dive into in a safety case written with @_robertkirk and others.

Principles for a positive ASI future:.- People maintain at least their current standards of health, living conditions, and political representation. - The benefits created by ASI are distributed so as to promote equitable human empowerment. - Humans can control the resources at.

The greatest AI security threat isn’t stealing secrets—it’s planting secret loyalties into models. Whoever controls the AI that seeds an intelligence explosion controls the future.

no matter what you think about ai alignment, we'll eventually need gov oversight of AI to preserve democracy. otherwise, there's no way to stop ASI from manipulating voters. "If I don't do it, my competitor will." those will be the last words of the free world.

do you think plan A should be to coordinate a multi-year global slowdown of AI before ASI?.

RT @AliciaP59828402: Guys I'll read the book but pleaseee - you still have time to change the cover art! 😩

this is bad news. the more nations have lots of AI chips the more difficult coordinating a multi-year slowdown will be.

I'm glad for more serious investigation of existential risk from think tanks, but I think ASI will be much better at identifying paths to human extinction than these authors. I wish they did not make sweeping claims like "Extinction threats posed by AI are immensely challenging".

figure out sleep. figure out stimulants. if only i knew earlier how important those goals are.

(this is a real question, not a rhetorical one).

What's preventing companies from reducing the time to deployment?. In the early bend of an intelligence explosion, I hope companies deploy faster than they do now (so the world can see the crazy amount of progress happening). Right now the lag seems to be 4+ months. Why so long?.

AI companies can potentially share internal misalignment incidents without hurting their reputation -- just give them to a third party to publish who will anonymize their origin.

RT @NeelNanda5: New post: I'm all for investment in interpretability but IMO this overstates its importance vs other safety methods. I disa….

RT @DKokotajlo: Indeed. Good luck to us all.

There's an argument for setting a norm of caution early. I'm not arguing against that. I'm arguing against restricting deployment. ASI poses a pretty similar x-risk whether it is deployed or not. The norm should be "cautious about development, and quick about deployment".