Oliver Guest
@OliverEGuest
Followers
181
Following
174
Media
4
Statuses
103
@iapsAI's new report on AI policies for preserving CoT monitorability is very thoughtful! They analyze a few coordination mechanisms: when coordination desirable (monitorability tax!), when it's feasible (collective action problems!), and how to verify compliance.
New @iapsAI report with @OliverEGuest and @araujonrenan: "Policy Options for Preserving Chain of Thought Monitorability." CoT reasoning is a boon for controlability—e.g. in tests OpenAI caught a model explicitly planning to cheat through its CoT. Let's keep CoT human-readable!🧵
0
2
7
New @iapsAI report with @OliverEGuest and @araujonrenan: "Policy Options for Preserving Chain of Thought Monitorability." CoT reasoning is a boon for controlability—e.g. in tests OpenAI caught a model explicitly planning to cheat through its CoT. Let's keep CoT human-readable!🧵
1
3
11
🚀 IAPS is hiring! We’re seeking Researchers and Senior Researchers to join our Frontier Security, Compute Policy, and International Strategy teams. Apply by August 10: https://t.co/JzzjlHQ7SO
0
1
7
@Scott_R_Singer, @kelmgren, and @OliverEGuest's latest paper provides the first detailed analysis of CnAISDA, China's self-described AI Safety Institute counterpart. Learn about what CnAISDA signals about the direction of Chinese AI policy more broadly 👇 https://t.co/1Tl1U4lFpG
carnegieendowment.org
The emergence of the China AI Safety and Development Association (CnAISDA) is a pivotal moment for China’s frontier AI governance. How it navigates substantial domestic challenges and growing...
0
1
5
unbeknownst to some, China now has an AISI: the China AI Safety and Development Association, aka 中国人工智能发展与安全网络 (name, as always, perhaps subject to change) unbeknownst to more, you can now read all abt it in a new report from myself, @Scott_R_Singer & @OliverEGuest
Over the last year, those of us who follow China's AI governance have been carefully watching whether China would establish an AI Safety Institute (AISI) to match those in the UK, US, and globally. That institution has now emerged, and it tells us a lot about the state of debate
2
18
90
Over the last year, those of us who follow China's AI governance have been carefully watching whether China would establish an AI Safety Institute (AISI) to match those in the UK, US, and globally. That institution has now emerged, and it tells us a lot about the state of debate
10
107
455
and really cool to see that it substantially expands on a dataset that @oscar__delaney and I put together for this paper
@OliverEGuest and I analyze the AI safety publications of leading companies, and find that their focus is heavily tilted towards some areas. In @aipolicybullet we highlight topics like evals, AI control, and novel safety paradigms that government and philanthropists can focus on.
0
0
2
Interesting new research suggesting that research on AI safety and reliability neglects post-deployment topics.
To guard against corporates driving research away from deployment risks (Figure 1): We recommend structured access into deployed AI systems’ telemetry data and artifacts to systematically analyze real-world risks and harms.
1
1
5
Other data efforts appear small-scale. Companies in the partnership program shared datasets valued in total at only ~$2 million. Plans for a crowdsourced data labelling platform seem to have been abandoned.
1
0
0
Another focus was training data: Government-backed BAAI released three iterations of a "Chinese Corpora Internet" dataset (~1 terabyte of Mandarin text from mainland websites). This stands out as one of the clearest and most significant outcomes.
1
0
0
A key focus was compute power—Beijing set up a "partnership program" claiming 8.5e18 FLOP/s of resources. Even taking the officials at the word, that's a comparable amount of throughput to the cluster that was used for GPT-4, two years ago
1
0
0
My deep dive suggests the outcomes here have been fairly modest.
1
0
0
When Chinese government agencies announce ambitious AI plans, how often do these actually become reality?
1
0
0
Firstly, Beijing’s municipal government itself plays a significant role in China's AI ecosystem. But more importantly, it's a data point for a key question in China x AI policy:
1
0
0
To clarify, these measures were from Beijing's municipal (i.e., provincial-level) government. Why does this matter?
1
0
0
Two years ago, Beijing's municipal government announced ambitious policies to boost "general AI" (通用人工智能). Today, how impactful have those measures really been?
1
0
0