Patricia Paskov
@prpaskov
Followers
228
Following
2K
Media
0
Statuses
62
AI evals + policy @randcorporation & @aigioxford | prev. @wb_research @poverty_action | alum @bse_barcelona @eui_eu @uwmadison
Joined April 2021
AI is evolving too quickly for an annual report to suffice. To help policymakers keep pace, we're introducing the first Key Update to the International AI Safety Report. 🧵⬇️ (1/10)
19
90
294
Tonight in NYC, an incredible set of speakers and attendees will gather for a lively conversation on verification mechanisms for international AI agreements. Join us!
Looking forward to holding a great event in NYC this evening on Verifying International AI Agreements, a very technically rich topic. We’ll have a keynote from @benharack, followed by discussion with @MauricBaker, @janet_e_egan, @jordanschnyc, and @prpaskov. There are a few spots
0
0
1
Join us for Verifying International AI Agreements, an event on Thursday, September 25th at 7pm in New York City during UNGA week. Capacity is limited, register interest here:
0
0
1
Looking forward, see you soon, DC!
Dario and I are gathering policymakers in DC on September 15th to give an inside look into Anthropic’s latest progress, and share live demonstrations of how AI is being applied to national security, science, and public services. Register to attend:
0
0
1
How can we verify that AI ChemBio safety tests were properly run? Today we're launching STREAM: a checklist for more transparent eval results. I read a lot of model reports. Often they miss important details, like human baselines. STREAM helps make peer review more systematic.
2
17
82
"data filtering can send WMDP-Bio scores to random chance without hurting general performance" -- promising research on open-weight safety hot off the press by @StephenLCasper, Kyle O'Brien, @BlancheMinerva and co-authors at @AISecurityInst & @AiEleuther
Are you afraid of LLMs teaching people how to build bioweapons? Have you tried just... not teaching LLMs about bioweapons? @AIEleuther and @AISecurityInst joined forces to see what would happen, pretraining three 6.9B models for 500B tokens and producing 15 total models to study
0
0
8
Excited to be leading this workstream with @sbmisi!
🚨 AI Evals Crisis: Officially kicking off the Eval Science Workstream 🚨 We’re building a shared scientific foundation for evaluating AI systems, one that’s rigorous, open, and grounded in real-world & cross-disciplinary best practices👇 (1/2) https://t.co/AQdEKtJS3l
0
1
3
Fresh off the press from the U.S. AI Action Plan -- building an AI Evaluations Ecosystem! đź’Żđź’Żđź’Ż
2
4
17
After I left OpenAI, I knew I wanted to be at a non-profit but wasn't sure whether to join or start one. Ultimately I started one bc [long story redacted] but RAND is one I considered + their pivot to taking AI seriously is a great thing for the world:
My team at RAND is hiring! Technical analysis for AI policy is desperately needed. Particularly keen on ML engineers and semiconductor experts eager to shape AI policy. Also seeking excellent generalists excited to join our fast-paced, impact-oriented team. Links below.
7
12
221
Join our team!
🚀Come join my team at RAND! We’re looking for research leads, researchers, & project managers for our compute, US AI policy, Europe, & talent management teams. All teams have urgent, important work to do & broad options for the future. Some roles close July 27⏰
0
0
1
🚨 New paper alert! 🚨 Are human baselines rigorous enough to support claims about "superhuman" performance? Spoiler alert: often not! @prpaskov and I will be presenting our spotlight paper at ICML next week on the state of human baselines + how to improve them!
1
8
21
Artificial intelligence has created a new digital divide, separating the nations with the computing power for building cutting-edge A.I. systems and those without. The split is influencing geopolitics and global economics. Here's how.
nytimes.com
As countries race to power artificial intelligence, a yawning gap is opening around the world.
1
17
29
Pleasure to collab on this!
Cooperation on AI safety is necessary but also comes with potential risks. In our new paper, we identify technical AI safety areas that present comparatively lower security concerns, making them more suitable for international cooperation—even between geopolitical rivals. 🧵
0
0
2
Very stoked to see these desiderata and standards directly referenced in the EU AI Act Codes of Practice draft (Measure 10.3 on rigorous model evaluations. pg 42-43), released last week. https://t.co/zyphYnZgsR
digital-strategy.ec.europa.eu
Independent experts present the second draft of the General-Purpose AI Code of Practice, based on the feedback received on the first draft, published on 14 November 2024.
0
0
0
Our latest paper outlines four desiderata for AI model evaluations -- internal validity, external validity, reproducibility, and portability -- and example standards for AI providers and evaluators to uphold them. https://t.co/KE6BoZ1uav (Section 3, pg 3-5)
arxiv.org
General-purpose AI evaluations have been proposed as a promising way of identifying and mitigating systemic risks posed by AI development and deployment. While GPAI evaluations play an...
1
0
2
Super excited to be sharing a new paper, coauthored with @AnkaReuel and many others, on Open Problems in Technical AI Governance. Summary in the thread below.
5
14
57
fantastic seeing former fieldwork with @poverty_action transform to publication đź‘Ź
In #Peru, a 3-hour training for Juntos #cashtransfers beneficiaries increased their trust in financial institutions, which in turn led to higher savings (+0.4-1.4 percentage points). Galiani et al: https://t.co/C6c0mkik6Z
1
0
8
The credibility revolution in economics.
BREAKING NEWS: The 2021 Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel has been awarded with one half to David Card and the other half jointly to Joshua D. Angrist and Guido W. Imbens. #NobelPrize
9
441
2K
0
0
2