Thomas Miconi
@ThomasMiconi
Followers
2K
Following
5K
Media
258
Statuses
3K
Neural networks, computational neuroscience, evolutionary computation, artificial life.
SF Bay Area
Joined September 2015
New preprint!. Intelligent creatures can solve truly novel problems (not just variations of previous problems), zero-shot!. Why? They can "think" before acting, i.e. mentally simulate possible behaviors and evaluate likely outcomes. How can we build agents with this ability?.
3
12
48
RT @pyoudeyer: New paper in the Alife journal !. Flow-Lenia: Emergent Evolutionary Dynamics in Mass Conservative Continuous Cellular Automa….
0
3
0
What are good examples of meta-training *plastic* neural networks (either optimizing plasticity rules or network structure) to address neurosci questions?. Things like @basile_cfx & @TPVogels' rule search, Tyulmankov et al Neuron 2021, my stuff on transitive inference -what else?.
0
0
1
See @BlackHC 's excellent thread for details:.
I'm late to review the "Illusion of Thinking" paper, so let me collect some of the best threads by and critical takes by @scaling01 in one place and sprinkle some of my own thoughts in as well. The paper is rather critical of reasoning LLMs (LRMs):.
0
0
1
Even after solving almost all Towers of Hanoi problems to N=5, and still most of them for N=7. Apparently, being able to apply a known algorithm to arbitrary novel situations (to the limit of your capacity) doesn't count as "reasoning" any more. Tough crowd!.
1
0
0
"We conclude that LRMs can't reason, because they were unable to extemporaneously come up with a size-general algorithm for the river-crossing problem"
1
1
4
“The Acropolis of Edin-polis” 😌.
0
0
2
Before: meta-optimization can only explore quantitative dimensions within pre-defined spaces (e.g. hyperparam search). Now: meta-optimization can propose *qualitative* changes, like what hyperparams to include, or how to compute the loss
0
0
3
Alternative headline: "DeepMind automates Graduate Student Descent".
Introducing AlphaEvolve: a Gemini-powered coding agent for algorithm discovery. It’s able to:. 🔘 Design faster matrix multiplication algorithms.🔘 Find new solutions to open math problems.🔘 Make data centers, chip design and AI training more efficient across @Google. 🧵
1
0
4
That's the goal of the DeepSeek/Absolute Zero papers:. In domains for which you have a ground-truth verifier (like coding), you can basically generate infinite amounts of RL training data.
RL people, where is data for these huge numbers of rollouts coming from? The only thing that makes sense to me (without millions of users’ data) is API/web agents.
0
0
2
❄️Introducing Absolute Zero Reasoner: Our reasoner learns to both propose tasks that maximize learnability and improve reasoning by solving them, entirely through self-play—with no external data! It overall outperforms other "zero" models in math & coding domains. 🧵 1/
0
0
2
@cu_neurotheory This paper is now out in @NatureNeuro :.
Fully neural mechanisms for structured generalization and rapid learning identified using a meta-learning (“learning-to-learn”) approach to model relational learning (e.g. if A > B and B > C, then A > C). @ThomasMiconi and Kenneth Kay.
0
0
3
Work done at the Astera Institute, Arxiv link: This is very much work in progress! More coming soon.
1
1
14
Results (after much training!):. Agents see a truly novel task, think about it in their world model, then act in the real environment. Thinking massively increases 0-shot performance. No baked-in planning algorithms, no hand-defined procedures - just autonomous "thinking"
1
0
3
2- Endow agents with a world model, and let them simulate possible future before acting. But that's not enough! You need to train the agent to actually make use of its thinking. For this, "evolve" training tasks that favor thinking, i.e. pre-think score >> post-think score.
1
0
4
1- Train on tasks that contain all elements of your environment, but withhold a certain specific *combination* of elements (e.g. pick, zombies, walls). Now any task relying these components is guaranteed novel, while still solvable 0-shot since you know each element's dynamics
1
0
2
We need two things:. 1- How to ensure a task is truly novel, never seen before in lifetime or evolution, while still being solvable based on your knowledge of the world?. 2- How do we make the agent "think" about the future before acting, and use the results of its thinking?.
1
0
3
Interestingly these two types of models seem more oriented towards different objectives - smooth maps, replay, theta sequences vs. "mental navigation". Wondering how they fit together - if they do!.
0
0
3
It's quite striking because other groups (@jcrwhittington, Fiete lab) are converging on models that actually *start* from grid cells. Grid cells anchored by place cells form "navigable scaffolds" on which memories can be mapped and explored internally.
2
0
1