#1. New paper alert!🚀 How do we evaluate LM agents today? Mostly benchmarks. But: (1) good benchmarks are costly + labor-intensive, and (2) they target narrow end goals, missing unintended capabilities. In our recent paper, we propose a goal-agnostic alternative for evaluating

#8. Check out the📄Paper: “Estimating the Empowerment of Language Model Agents”: https://t.co/dnWd0ZR5Av This work is by Jinyeop Song(@jinyeop_song) , Jeff Gore, and Max Kleiman-Weiner(@maxhkw). We thank MATS (@MATSprogram) program for mentorship, support and compute.

#7.Finally, Empowerment highlights pivotal states/actions. In the Shopping Admin domain, valid versus invalid authentication shows a clear difference in empowerment. This may be useful for spotting reward hacking, power-seeking, and other safety-relevant behaviors.

#6. We also test in a realistic web-browsing setting (WebArena, Zhou et al. 2024). Across four domains and three models (GPT-4o-mini, o3, GPT-4o), estimated empowerment strongly correlates with discounted reward.

#5. How does this help evaluation? We show that empowerment correlates strongly with task performance (mean discounted reward), and the trend holds across LM-agent variants (scale, CoT, memory).

#4. On language games (GridWorld, Tower of Hanoi), EELMA’s estimates align with direct empowerment estimators and track changes in agent capability across interactions. When there is semantic variation in the observation space, EELMA’s estimates are more robust.

#3. Estimating empowerment for LMs is hard (language is high-dimensional). We introduce Estimating Empowerment of Language Model Agents (EELMA), a method that combines pretrained language embeddings (text → representations) with an InfoNCE contrastive estimator (representations

#2. What is empowerment? The mutual information between an agent’s actions and its future states, i.e., how much control the agent has over what happens in the future. Highly empowered agents recognize the full range of available actions (high future optionality) and can

How do task vectors emerge during pretraining—and can they predict ICL performance? Come see our ICML spotlight poster "Emergence and Effectiveness of Task Vectors in ICL" at 11am @ East Hall A-B (#E-2312) with @jinyeop_song! 🔗 https://t.co/nvzAIRs84Q

Check out our new benchmark "SPOT" for assessing LLMs' ability to detect scientific errors in papers! Collaborative work led by @gson_AI.

#NLProc AI Co-Scientists 🤖 can generate ideas, but can they spot mistakes? (not yet! 🚫) In my recent paper, we introduce SPOT, a dataset of STEM manuscripts (math, materials science, chemistry, physics, etc), annotated with real errors. SOTA models like o3, gemini-2.5-pro

Another huge week in AI Agents 🧵 I summarized everything announced by LangChain, Replit, Cursor, Anthropic, Cobot, AgentOps, Triple Whale, Factory AI, Lovable, Hugging Face, you, and more. Here's everything you need to know and how to make sense out of it. (save for later)

🧩 Why do task vectors exist in pretrained LLMs? Excited to share our new paper with @seungwookh on the emergence of abstractions (task vectors) for in-context learning (ICL)! To read more : https://t.co/V99VQXIKlR.

🧩 Why do task vectors exist in pretrained LLMs? Our new research uncovers how transformers form internal abstractions and the mechanisms behind in-context learning(ICL).

I’m recruiting PhD students to join the Computational Minds and Machines Lab at the University of Washington in Seattle! Join us to work at the intersection of computational cognitive science and AI with a broad focus on social intelligence. (Please reshare!)

1/ our analysis reconciling the conflicting LLM scaling laws of Kaplan (N ∝ C^0.73) and Chinchilla (N ∝ C^0.50) accepted to TMLR 🥳 https://t.co/E58R1VKNQV we explain this long-standing discrepency, with two concrete recommendations for anyone running scaling law analyses...

The computational cognitive science starter kit.

Prompt engineering requires a lot of manual effort. Here are four automatic prompt optimization algorithms that can help to improve your prompt with minimal effort… (1) Automatic Prompt Engineer (APE) [1] searches over a pool of prompts proposed by an LLM–usually ~32-64

🧬 Our 2nd preprint today, MethylGPT: A foundation model for the DNA methylome! Our generative pretained transformer-based model: - Trained on 226,555 samples (154,063 after QC & deduplication) - 49,156 curated CpG sites - 7.6B training tokens - Spans 20+ tissue types from