The Science of AI Is Too Important to Be Left to the Scientists. In a new opinion piece for Foreign Policy I explain why I’m worried for the UN’s new “Independent International Scientific Panel on AI”. (link in reply)

RT @JonKBateman: Carnegie is hiring fellows or senior fellows to work on tech policy, particularly AI geopolitics, governance, or global so….

RT @bill_drexel: Great points from @PougetHadrien here on int'l AI talks at the UN. Without stronger political involvement, they'll likely….

Check out the article here:

What can we learn from previous experiences in climate change, biodiversity, and AI? And why might UN member states still prefer to go down the wrong path? Read my article on the FP website, or feel free to reach out if you need help accessing it.

For the findings to reach and bind decisionmakers, the scientific process must include them. It's harder and messier – but if the UN does not do it, no one can.

But the panel could also easily miss the mark, wasting time we don’t have. Insistence on the scientific “independence” of the panel’s work seems commendable, but can be a recipe for irrelevance.

The UN’s new panel could build agreement between global decisionmakers on the facts of AI’s risks – a precursor to much-needed international cooperation.

How is France approaching its "AI Action Summit," the successor to the UK's AI Safety Summit?. Check out my new analysis! I explore France's unique position, and offer thoughts on how it might make the summit a success.

RT @JonKBateman: Should AI models be “open” or “closed”? Today Carnegie joins with leading voices from across the spectrum to help advance….

RT @mattsheehan88: Moving from high-level AI regulations to implementation & compliance is going to be really hard. @PougetHadrien & Ranj….

That being said, the EU shouldn't do nothing - this is a unique chance to lead the world on a critically important issue. I'm very excited to see what comes out of the negotiations!. 16/16.

Allowing for evolving definitions, risks profiles, and standards is important - both to continue to meet the challenges AI will bring, and to stay aligned with the international community. 15/n.

(4) Overall.This is a tricky policy area, rapidly evolving and full of uncertainty. The EU is in a complicated position, sandwiched between member states and the international community. Just like the EO, the EU should be careful not to be too definitive. 14/n.

The EU could do something similar. For a small number of risky systems, it could require registering the system with the EU and running red-teaming exercises / other safety evaluations, regardless of open-source status. 13/n.

However, the EO's call for inputs shouldn't distract from the fact that requirements in Section 4.2. already apply to open-source dual use foundation models: .- reporting training details.- reporting safety testing results. 12/n.

In the EU, exemptions from requirements for open-source systems are being considered. (it's worth noting that this exemption only applies if you're not monetizing the open-source system in some roundabout way, so it's not clear models like Llama 2 would be exempted). 11/n.

(3) Approach to Open Source. What to do about open-source models is a controversial topic on both sides of the Atlantic. The EO is not definitive in its solution; section 4.6 starts a formal process to solicit input and establish pros, cons, and options. 10/n.

(quick aside: Advanced AI systems aside, @Johann_Laux and I argue that the AI office could fill a lot of important roles if the AI Act is to be effectively implemented). 9/n.

The EU may not be able to do the same (member states will likely want to do this themselves). Creating and resourcing the AI Office is a bit lift. But the Office should at least be able to engage with the results of evaluations and the new standards being developed. 8/n.