Magda Dubois
@DubMagda
Followers
377
Following
122
Media
15
Statuses
94
Research Scientist @AISecurityInst working on LLM evaluations 🤖 | PhD in computational cognitive neuroscience @MPC_CompPsych 🧠
London, England
Joined December 2016
Why does horizon length grow exponentially as shown in the METR plot? Our new paper investigates this by isolating the execution capabilities of LLMs. Here's why you shouldn't be fooled by slowing progress on typical short-task benchmarks... 🧵
13
32
257
We're excited to announce the Call for Papers for SaTML 2026, the premier conference on secure and trustworthy machine learning @satml_conf We seek papers on secure, private, and fair learning algorithms and systems. 👉 https://t.co/cPFitlsXu2 ⏰ Deadline: Sept 24
0
15
40
Hawthorne effect describes how study participants modify their behavior if they know they are being observed In our paper 📢, we study if LLMs exhibit analogous patterns🧠 Spoiler: they do⚠️ 🧵1/n
3
21
121
In a new paper, we examine recent claims that AI systems have been observed ‘scheming’, or making strategic attempts to mislead humans. We argue that to test these claims properly, more rigorous methods are needed.
4
25
86
Evaluating AI models is essential for improving their performance and understanding their risks. Increasingly, researchers are using “autograders” – having Large Language Models (LLMs) grade model outputs. But how do we know if these autograders are reliable? 🧵
1
6
66
New paper introducing a framework to better quantify uncertainty in LLM evaluations (led by @LLuettgau🙌). A beta Python package (developed by @HarryCoppock🚀) is available if you want to try it out. ➡️Get in touch if you have any Qs/feedback! Paper:
Advanced AI systems require complex evaluations to measure abilities, but conventional analysis techniques often fall short. Introducing HiBayES: a flexible, robust statistical modelling framework that accounts for the nuances & hierarchical structure of advanced evaluations.
0
0
1
🧵 Today we’re publishing our first Research Agenda – a detailed outline of the most urgent questions we’re working to answer as AI capabilities grow. It’s our roadmap for tackling the hardest technical challenges in AI security.
5
53
123
Excited to share our brand-new work shedding some light on the neural mechanisms behind one of human’s coolest cognitive feats: compositional generalization of structural knowledge! A Tweeprint-Thread 🧵 1/n
1
8
30
1/ New paper in Nature shows model collapse as successive model generations models are recursively trained on synthetic data. This is an important result. While many researchers today view synthetic data as AI philosopher’s stone, there is no free lunch. Read more 👇
44
95
666
I am excited to share that our article *Navigating the European Union Artificial Intelligence Act for Healthcare* has just been published in @npjDigitalMed🚀 #AIRegulation #DigitalHealth #EUAIAct #MedicalDevices #Innovation #npjDigitalMedicine #AIinHealthcare
1
8
28
Paper out in @PNASNews! A 'cognitive mapping' lens on language in psychosis, using word embedding models, computational modelling, and MEG. A hint of what's to come at @OxPsychiatry and @UCLBrainScience... With @mcneural_, @YunzheNeuro, Ray Dolan. https://t.co/3uhL6z3eSw
pnas.org
Human cognition is underpinned by structured internal representations that encode relationships between entities in the world (cognitive maps). Cli...
5
38
124
Preprint alert🚨! In this new paper we study how humans decompose dynamical subprocesses and leverage the abstracted subprocesses for compositional reuse of experience in new situations. https://t.co/9UsV5uAcPE Tweeprint to follow soon!
1
24
61
In our lab's latest paper, we introduce a novel modeling approach using RNNs to reveal the cognitive algorithms behind animal decision-making. Check out our preprint, led by UCSD PhD student @Ji_An_Li and co-authored by Marcus Benna:
biorxiv.org
Normative modeling frameworks such as Bayesian inference and reward-based learning provide valuable insights into the fundamental principles of adaptive behavior. However, their ability to describe...
3
28
98
Congratulations to my academic sibling @AlisaLoosen for those (very) well-deserved three shiny balloons
0
0
19
Wanna try out a (cool🦙) alternative to GPT?
🦙Excited to share this demo of Alpaca 🔥Highlights: ~GPT3.5 performance for < 600$🔥 The goal was to have a simple model /training procedure that academics could study and improve with limited resources We achieved that by finetuning a 7B LLaMA on 52K generated instructions
0
0
1
Postdoc position in Boston ⭐️ Great place and amazing person to work with !
0
1
1
A while ago we published this #RegisteredReport in @NatureComms - but was this format of pre-registration really useful? Find some answers in this Q&A with us and one of the reviewers:
nature.com
Nature Communications - We recently published our first Registered Report entitled ‘Value-free random exploration is linked to impulsivity’. We believe the format offers many benefits...
Our #RegisteredReport with @TobiasUHauser is now out in @NatureComms 🤓 We asked how people differ in their exploration - and found that impulsive and anxious subjects explore using different exploration strategies ! 1/ https://t.co/iJ4MjadXox
0
1
2
New preprint ! 📢📢 Very happy to share some recent work looking at the holy trinity of transdiagnostic symptom dimensions (anxious-depression, compulsivity and social withdrawal) and how we can optimise their measurement. https://t.co/gdZHm3YuEx Some key points below… 1/n
4
26
73
IN SILICO is now available for streaming internationally via Vimeo On Demand: https://t.co/HHmEJOeuQl It's a story told over 10 years about an attempt to simulate a brain on supercomputers. Now for more about the movie's plot and its subjects, via GIFs... 🧵👇
6
42
77
The code for the @CompPsychiatry is now available online: https://t.co/ukK6TIQsJL. Check out the @VbaToolbox by @lionel_rigoux , hBayes by @wooyoungahn and Active Inference @Schwartenbeckph, HGF is soon to come...#CPC2018 #opensource @tnuzurich
0
11
9