should've started reading steinbeck earlier 😋

greatest linkedin sales message i’ve ever gotten tbh

At Transluce, we train investigator agents to surface specific behaviors in other models. Can this approach scale to frontier LMs? We find it can, even with a much smaller investigator! We use an 8B model to automatically jailbreak GPT-5, Claude Opus 4.1 & Gemini 2.5 Pro. (1/)

Docent, our tool for analyzing complex AI behaviors, is now in public alpha! It helps scalably answer questions about agent behavior, like “is my model reward hacking” or “where does it violate instructions.” Today, anyone can get started with just a few lines of code!

onboarded onto a new research codebase today

only took me 4 years (+ daily prodding from @mengk20 😅) to realize: - website layout should reflect the info you want to communicate, rather than just copying patterns you saw on other sites - half the elements on my homepage weren’t doing anything

thanks to @upcycledwords @cjquines @clairebookworm1 @laurgao for looking over drafts 🥰 narrator’s opinions not necessarily my own https://t.co/9G3FEAwhYA

i’ve been experimenting with writing AI research fanfiction 🤪🥸🤔 jokes aside, it’s also a story about AI culture / putting people on pedestals / deciding what to believe in. hope you enjoy!

Pro tip: you can basically travel the world by using google maps.

still have some reliability + sensitivity issues to work through also brainstorming fun designs to decorate the top of the pad 🤩 if you have suggestions let me know

building a Dance Dance Revolution pad from first principles! nothing fancy, just wood + aluminum + wires + tape

i think it's really cute that Iowa State University writes language model reasoning papers about agriculture

sometimes you have to apply exponential backoff when texting new people

We tested a pre-release version of o3 and found that it frequently fabricates actions it never took, and then elaborately justifies these actions when confronted. We were surprised, so we dug deeper 🔎🧵(1/) https://t.co/IdBboD7NsP

🤮📉🚫 building yet another LLM benchmark 🥰📈🌈 building a tool that can make every existing benchmark more useful very excited to share Docent: a system that can look through eval results and identify unusual model behaviors, cheating, env setup issues, etc. in just minutes!

To interpret AI benchmarks, we need to look at the data. Top-level numbers don't mean what you think: there may be broken tasks, unexpected behaviors, or near-misses. We're introducing Docent to accelerate analysis of AI agent transcripts. It can spot surprises in seconds. 🧵👇

back when I was young, I thought it was unrealistic for the Volunteer Fire Department to schism into a branch that fought fires and a branch that started them

everybody deserves to see interstellar in imax. i would get a lifetime amc a-list subscription if it meant interstellar would always be in theatres. id go every wednesday night and have my own chair and everything

check out our writeup and demo for more applications of understanding + steering models via neuron descriptions!

i think we have the most compelling explanation so far for why LLMs make mistakes like 9.11>9.9 🙂 1) we labeled every neuron in Llama3 2) when Llama says 9.11>9.9 we see influential groups of neurons about dates and bible verses 3) zeroing those allows Llama to answer correctly