🔥Our paper PhyloLM got accepted at ICLR 2025 !🔥 In this work we show how easy it can be to infer relationship between LLMs by constructing trees and to predict their performances and behavior at a very low cost with @StePalminteri and @pyoudeyer ! Here is a brief recap ⬇️

I'm attending ICML 2025 this week in Vancouver where we're presenting our MAGELLAN paper along with @LorisGaven and @CartaThomas2! 📅 Come discuss at our poster session on July 17 at 11 am East Exhibition Hall A-B E-2803 Or reach out for a chat! https://t.co/Uk8HKmcOHM

Introducing SOAR 🚀, a self-improving framework for prog synth that alternates between search and learning (accepted to #ICML!) It brings LLMs from just a few percent on ARC-AGI-1 up to 52% We’re releasing the finetuned LLMs, a dataset of 5M generated programs and the code. 🧵

New blog post ! What if LLM agents could learn by doing, not just by reading? 🤔 2024 was the year of "agentic AI"—systems that plan, act, and execute complex workflows autonomously. But current agents face critical limitations... 🧵

Curious about LLM interpretability and understanding ? We borrowed concepts from genetics to map language models, predict their capabilities, and even uncovered surprising insights about their training ! Come see my poster at #ICLR2025 3pm Hall 2B #505 !

If you are interested in this line of research of mapping LLMs you might also want to check the amazing work of @EliahuHorwitz https://t.co/sd2RpLH4HY and @momose123456789 https://t.co/LaMMlPEAc9 10/10

In short, PhyloLM is a cheap and versatile algorithm that generates useful representations for LLMs that can have creative applications in pratice. 9/10 paper : https://t.co/fmkdUe6MRX colab : https://t.co/7FT4JyKKPc code : https://t.co/oOLyQRxG8E ICLR : Saturday 3pm Poster #505

A PhyloLM collaborative Huggingface space is available to try the algorithm and visualize maps : https://t.co/8jXk7uFbbZ The Model Submit button has been temporarily disabled for technical reasons but you can play with the data while we fix it ! 8/10

By using code related contexts we can obtain a fairly different map. For example we notice that Qwen and GPT-3.5 have a very different way of coding compared to the other models which was not visible on the reasoning map. 7/10

The contexts choice is important as it reflects different capabilities of LLMs. Here on a general reasoning type of context we can plot a map of models using UMAP. The larger the edge, the closer models are from each other. Models on the same cluster are even closer ! 6/10

It can also measure quantization efficiency by observing the behavioral distance between LLM and quantized versions. In the Qwen 1concequantization could provide additional insights to quantizationould provide additional insights to quantization efficiency. 5/10 @elias_frantar

Aside from plotting trees, PhyloLM similarity matrix is very versatile. For example, running a logistic regression on the distance matrix makes it possible to predict performance of new models even from unseen families with good accuracy. Here is what we got on ARC. 4/10

Not taking into account these requirements can still produce relevant distance vizualisation trees. However it is important to remember they do not represent evolutionary trees. 3/10

Phylogenetic algorithms often require common ancestors to not appear in the objects studied but are clearly able to retrieve the evolution of the family. Here is an example in the richness of open-access model : @docsgptai @Teknium1 @maximelabonne @MistralAI @OpenChatDev 2/10

We build a distance matrix from comparing outputs of LLMs to a hundred of different contexts and build maps and trees from this distance matrix. Because PhyloLM only requires sampling very few tokens after a very short contexts the algorithm is particularly cheap to run. 1/10

🧠 One of the key limitation of LLMs today is their lack of metacognition: they were (mostly) not trained to know what they know or don't know, what they can or can't do. 🚀At @FlowersINRIA, we're proposing an approach to build metacognition into LLMs: MAGELLAN !

Enabling forms of metacognition in LLMs is a frontiers challenge in #AI We've made progress in this direction: 🧭MAGELLAN allows curiosity-driven LLMs to learn to predict and generalize their own learning progress, and navigate in very large spaces of goals 🚀 Details here 👇

🚀 Introducing 🧭MAGELLAN—our new metacognitive framework for LLM agents! It predicts its own learning progress (LP) in vast natural language goal spaces, enabling efficient exploration of complex domains.🌍✨Learn more: 🔗 https://t.co/uGLBSsOgMn #OpenEndedLearning #LLM #RL

We just opened a new (engineering) internship position in the @FlowersINRIA team with @pyoudeyer: https://t.co/OxneS2bIWF We'll focus on developing our lamorel library, which has been central to our recent work on grounding embodied LLMs (more below 👇1/4)

🚀 Exciting Internship Opportunities for AI and CogSci Students🌟 Join @FlowersINRIA and work on these cool topics: 🔧 Curriculum learning of skill libraries in autotelic agents using LLMs and program synthesis wth @PourcelJulien 🎯 Balancing Exploration and Exploitation in