Jan Brauner
@JanMBrauner
Followers
1K
Following
5K
Media
21
Statuses
471
Technical staff member at EU AI Office, Previously: RAND, ML PhD at Oxford (@OATML_Oxford), and, once upon a time, medical doctor. Tweeting in private capacity.
Joined December 2020
They just keep on delivering. How do they do it?.
New paper & surprising result. LLMs transmit traits to other models via hidden signals in data. Datasets consisting only of 3-digit numbers can transmit a love for owls, or evil tendencies. 🧵
0
0
10
RT @Manderljung: The EU's Code of Practice for General-Purpose AI is out. As one of the co-chairs who drafted the Safety & Security Chapter….
0
21
0
Plot twist I just realised after 10 years: Michael Gutmann invented noise-contrastive estimation, a precursor to GANs. Ian Goodfellow invented GANs. Gutmann is German for 'Goodfellow.'
0
0
10
RT @Yoshua_Bengio: The @EU_Commission is setting up a scientific panel of 60 independent experts to support the implementation and enforcem….
digital-strategy.ec.europa.eu
The European Commission is setting up a scientific panel of independent experts to support the implementation and enforcement of the AI Act.
0
24
0
4/5 Part-time role, 2-year renewable terms, compensation provided. EU citizenship is NOT required (although 80% of spots are reserved for EU/EEA nationals).
1
0
10
3/5 Looking for expertise in AI evals, risk assessment, risk mitigation, key areas of systemic risk (incl. misuse, misalignment, structural risks), cybersecurity, ….Experts from academia, think tanks, and civil society - but you must be independent from major AI providers.
1
0
8
2/5 Panel members can issue alerts on emerging AI risks that go straight to the regulators. The panel will assess frontier AI models and advise on model classification, evaluation methodologies, and cross-border market surveillance.
3
0
8
🧵1/5 The EU is building something unprecedented: a Scientific Panel with real teeth to assess the impacts and risks of general-purpose AI systems. 60 independent experts will directly influence how the world's first comprehensive AI law gets implemented.
1
17
76
Wuestion: According to R1, the magic sauce for reasoning models is simply RL on certain types of tasks with hardcoded reward. But why did it take us so long to get this to work?. We've been doing RLHF on LLMs for 4 or so years, and that seems technically a lot harder.
3
1
7
Another amazing contribution from Owain et al!.
New paper:.We train LLMs on a particular behavior, e.g. always choosing risky options in economic decisions. They can *describe* their new behavior, despite no explicit mentions in the training data. So LLMs have a form of intuitive self-awareness 🧵
0
0
6
Why I love working here:.* Hyper-competent team (15 people, most coming from successful careers outside of the Eur. Commission).* Fast-paced, start-up-like environment (yes, really).* So much exciting stuff to do: detailing the AI Act, enforcement, international collab, R&D. 3/3.
0
0
29
Open calls for legal and policy specialists: Expression of interest for technical and other profiles: 2/3.
digital-strategy.ec.europa.eu
The Commission has opened two calls for expression of interest to recruit new members for the European AI Office. Apply now as Legal or Policy Officer for an opportunity to shape trustworthy AI.
1
2
18
Some news: I recently (*) joined the EU AI Office as the first staff member in their new AI Safety Unit. The initial months have been a crazy, wild ride. It’s probably the most fun, intense, and impactful job I ever had. We’re currently hiring. 1/3. (*) 4 months ago, ahem.
6
20
192
Great opportunity for AI and governance experts. AI office role, deadline on September 6. You need to be a EU citizen, but the role is in SF.
0
10
21
Future Matters has done great work in climate change and they are increasingly expanding into AI Safety and biosecurity. Consider applying. The CEO, Justus, is a good friend of mine; I think it'd be hard to find a better boss than him.
0
2
14
Pure fire coming from Owain et al, as usual!.
New paper:.We measure *situational awareness* in LLMs, i.e. a) Do LLMs know they are LLMs and act as such?.b) Are LLMs aware when they’re deployed publicly vs. tested in-house?. If so, this undermines the validity of the tests!. We evaluate 19 LLMs on 16 new tasks 🧵
0
0
13
RT @geoffreyhinton: New AI safety paper in Science with a lot of authors:
science.org
Preparation requires technical research and development, as well as adaptive, proactive governance
0
415
0