RT @aviral_kumar2: Given the confusion around what RL does for reasoning in LLMs, @setlur_amrith & I wrote a new blog post on when RL simpl….

RT @ellisk_kellis: New paper: World models + Program synthesis by @topwasu.1. World modeling on-the-fly by synthesizing programs w/ 4000+ l….

10/ Animal-like autonomy—flexibly adapting to new environments without supervision—is a key ingredient of general intelligence. Our work shows this hinges on 1) a predictive world model and 2) memory primitives that ground these predictions in ethologically relevant contexts.

9/ Finally, we show that the neural-glial circuit proposed in Mu et al. (2019) emerges from the latent dynamics of 3M-Progress agents. Thanks to my collaborators Alyn T. and @fel_p8, and to @xaqlab for his continued support!. Paper link:

8/ 3M-Progress agents achieve the best alignment with brain data compared to existing intrinsic drives and data-driven controls. Together with the behavioral alignment, 3M-Progress agents saturate inter-animal consistency and thus pass the NeuroAI Turing test on this dataset.

7/ 3M-Progress agents exhibit stable transitions between active and passive states that closely match real zebrafish behavior.

6/ The agent learns a forward dynamics model and measures the divergence between this model and a frozen ethological memory. This model-memory-mismatch (3M) is tracked over time (w/ gamma-progress) to form the final intrinsic reward.

5/ First, we construct two environments extending the dm-control suite: one that captures the basic physics of zebrafish ecology (reactive fluid forces and drifting currents), and one that replicates the head-fixed experimental protocol in @muyuuyum @MishaAhrens et al. 2019.

4/ To bridge this gap, we introduce 3M-Progress, which reinforces behavior that systematically aligns with an ethological memory. 3M-Progress agents capture nearly all the variability in behavioral and whole-brain calcium recordings in autonomously behaving larval zebrafish.

3/ Existing model-based intrinsic motivation algorithms (e.g. learning progress, prediction-error) exhibit non-stationary and saturating reward dynamics, leading to transient behavioral strategies that fail to capture the robust nature of ethological animal behavior.

2/ Model-based intrinsic motivation is a class of exploration methods in RL that leverage predictive world models to generate an intrinsic reward signal. This signal is completely self-supervised and can guide behavior in sparse-reward or reward-free environments.

1/ I'm excited to share recent results from my first collaboration with the amazing @aran_nayebi and @Leokoz8! . We show how autonomous behavior and whole-brain dynamics emerge in embodied agents with intrinsic motivation driven by world models.