Today, we propose the Conditional AI Safety Treaty in @TIME as a solution to AI's existential risks. AI poses a risk of human extinction, but this problem is not unsolvable. The Conditional AI Safety Treaty is a global response to avoid losing control over AI. How does it work?

AI self-improvement is a central part of the classic Yudkowsky/Bostrom fast takeoff scenario. It has been out of fashion since the advent of LLMs, which need time to train. However: 1) Prompt optimization and scaffolding gain importance, and can self-improve 2) New paradigms may

Are you around in Amsterdam this weekend? Feel invited to our Existential Risk Christmas Drink in Bar Bukowski! Details and RSVP here: https://t.co/YCtwkR9JSP We hope to see you there!

🚨NEW YORKERS: Tell Gov Hochul to SIGN the RAISE ACT before it's too late!🚨 Tell Governor Hochul to sign the RAISE Act, which has already passed the NY legislature, so it becomes law before Trump's unconstitutional AI preemption executive order is signed. Link 👇

"AGI is a conspiracy theory" is a conspiracy theory. You have to believe that many leading scientists are in on it. This is drivel.

Yes. We have to worry about AI and robotics. Some questions:

Former environment minister and journalist Zac Goldsmith says the UK should “resume its global leadership on AI security by championing an international agreement to prohibit the development of superintelligence until we know what we are dealing with and how to contain it.”

Senator Bernie Sanders (@SenSanders) says there is a real concern that artificial superintelligence could replace human beings in controlling the planet. "That's not science fiction. That is a real fear that very knowledgeable people have."

🎥 Watch: Conjecture CTO Gabe Alfour's opening statement to a Canadian House of Commons committee. @gabe_cc argues we can't continue to ignore warnings by top experts that superintelligence could lead to human extinction; countries must halt the development of superintelligence.

Even in a world where we solve technical alignment, ASI might well lead to lasting suppression by a single power. We don't have to accept this. We can use peaceful measures such as export controls, sanctions, tariffs, or a trade embargo to stop ASI development. Great paper!

So far, most xriskers have felt too good for anti-data center campaigning. We made fun of data center water usage and electricity consumption, even though these are actual problems. Already, these issues are big enough for politicians from left to right to win elections on.

🎥 Watch: MIRI CEO Malo Bourgon's opening statement before a Canadian House of Commons committee. @m_bourgon argues that superintelligence poses a risk of human extinction, but that this is not inevitable. We can kickstart a conversation that makes it possible to avert this.

Not the Christmas cards you were hoping for. New system cards from OpenAI, Anthropic, Google and xAI indicate AIs are becoming more capable in dangerous domains such as biological weapons and automating AI research. Read more in our latest article! https://t.co/twKYGLx9z6

.@ForHumanityPod has an amazing track record communicating the risk we all face to the public. This work is badly needed to get crucial meaningful regulation off the ground. We support GuardRailNow and so should you.

Leading AGI researchers who raise billions have at best half-baked safety ideas. This sector badly needs regulation. A good start would be to implement a Conditional AI Safety Treaty as we proposed:

Congressional leadership is trying to force AI preemption into the National Defense Authorization Act (NDAA). "Preemption" means that the states would be banned from regulating AI. Given the lack of federal AI regulation, this would be a disaster. Call and Email NOW below 👇

If controllable at all, AI and mass robotics can easily be used to suppress us. We don't have to live in a dystopia. We can sign treaties outlawing robot/AI armies, police, and secret services. But we need to put in the work to make it happen.

Our @BartenOtto shared his insights on a talk for The School for Moral Ambition, AI Circle (thank you Karolina Gruzel for organizing!) He goes into our long-term future (and shares some critical notes on longtermism), existential risk, why AI may be the largest existential risk,

.@mustafasuleyman is right: AI development must be humanist. To get there, we need an international AI Safety Treaty, making sure red lines are respected. @MicrosoftAI, will you use your lobbying power to make that happen? If not, humanist AI is an empty phrase.

Sometimes, it is hard to believe that this is all real. Are people really building a machine that could be about to kill every living thing on this planet? If this is not true, why are the best scientists in the world saying it is? If this is true, why is no one trying to do