Timothée Chauvin
@timotheechauvin
Followers
296
Following
2K
Media
19
Statuses
169
I'm impressed by the results, but I don't think AlphaEvolve is a big deal, because it's only an alternative to RL in very narrow settings: the range of tasks that can be expressed as "solution of a standard program" and "cheaply and reliably graded" seems small. For instance I.
Pretty big deal imo. Beyond the impressive scientific discoveries, one way to see it is a lower bound for inference time compute: . EVEN if we fail to train reasoning models to productively think for days/weeks about problems, we can still productively dump a lot of compute at.
0
0
2
If anyone's interested, the project is which syncs my readwise documents, highlights and notes into my org-roam collection.
github.com
sync your readwise documents, highlights and notes into your org-roam collection - timothee-chauvin/org-readwise-rust
0
0
1
An interesting implication of Cursor is that you can now start coding in a programming language of your choice, even if you've never used it. I've just done that with Rust, and not only did it work, I found it to be an engaging way to learn the language (I had read the Rust book.
2
2
22
I've just published the recording of my introduction to "Dangerous Capability Evaluations in AI Models", given last month to @Polytechnique alumni working in AI:
1
3
7
24. Cybersecurity's approach to emerging risks is to first let them become a significant problem, before doing something about it. In the context of very rapid progress toward human-level AI and beyond, this approach seems particularly inadequate.
0
0
3
23. While largely technological, how well things will go regarding AI and cybersecurity can also be significantly affected by regulation (as always, in net-good or net-bad ways). Banks made money from credit card fraud before the 1974 [Fair Credit Billing.
1
0
1
22. Despite "cybersecurity" being a frequent keyword in many recent communications on future AI developments, expertise at the intersection of cybersecurity and AI is extremely scarce. This is in part due to [a cultural difference between the two fields](,.
1
0
1
21. When we get to digital minds, or long-running AI agents that we believe to probably be moral patients, their cybersecurity will be much higher-stakes than ours. Being hacked would be like getting abducted, and possibly way worse.
1
1
3
20. Fuzzing is only partial automation, so in practice humans are still the ones finding vulnerabilities (if only by writing fuzzing harnesses and investigating the crashes). AI can be the full automation. We will get to a regime where humans are no longer the ones finding.
1
0
1
19. Thinking that open-weight AI is less secure than APIs does not oppose everything we've learned in cybersecurity over the past decades. We have indeed empirically learned, over and over, that if obscurity is your only layer of security, you can be confident you're doing.
1
0
2
18. In the current era of compute scaling, we can expect AI deployment to keep being quite centralized. When the economy completely runs on AI, this will introduce new single points of failure. Part 2 of [What Failure Looks Like]( tells the story of a.
1
0
1
16. "AI finding new zero-days" isn't that scary: AI being very good at vulnerability detection in source code would be good overall, as long as defenders are able to spend the most compute on that task, and before adversaries. This is discussed in more detail in section 6 of the.
1
0
2
14. "Click and type" agents (that interact with a computer roughly like humans do) are coming soon (because the economic incentives for them are massive, and they don't seem to require major breakthroughs), and will unlock new cybersecurity capabilities compared to previous.
1
0
2
12. Serving different models to different actors will be increasingly important. It would be nice to have good methods for releasing a regular model, and the same model which is good in cyber-offensive tasks. We want the automated pentesters to exist, we just don't want their.
1
0
2
11. We will enter a phase where AI companies will agree to, or be required to, run their frontier models on useful external tasks before release (currently, only evaluations are done). The first applications will likely be in cybersecurity, due to the adversarial nature of the.
1
0
1
10. In the current era of compute scaling, frontier open-weight models might disappear in the near future (for national security or economic reasons). This makes it worth distinguishing between state-affiliated actors, and other actors. The former will have access to hacked model.
1
0
1
9. It is currently very easy for a state actor to get the secrets of any AI company. We are far from where we should be there (more: the [RAND report on securing model weights](). This makes hardware export controls look even more valuable as a policy.
2
0
1
8. Cybersecurity is heavily bottlenecked on labor on both the offensive and defensive side. In other words, there are vulnerabilities everywhere, and very few people looking. This means that you don't need superintelligence to have a massive impact on cybersecurity: human-level.
1
3
2
6. That being said, a typical employee is also quite adversarially weak, though in a different way. Basic, poorly written phishing emails still work. 7. On the same note, installing security updates in humans is very slow and costly. Since humans don't get upgraded, their.
1
0
2