Switched dinner tonight to the tiny table that looks just like the one at my kid’s kindergarten. instantly got the "kindergarten" version of the dining experience: he was super independent & way more chill and well behaved. I feel like I finally get persona prompting for LLMs 😅

unpopular opinion (?): text outputs (like the one below 🤯) can excite, intrigue and move me in ways that no fancy Nano Banana generated image will ever be able to do. as modalities, text is just a gazillion times more interesting than image.

This is not a particularly good take and is indicative of a fundamental misunderstanding of what a top-tier technical college education is suppose to offer. Preparing to understand modern AI as a Harvard or Stanford undergrad is not about learning "prompt engineering", vibe

If you're working on factuality in LLMs, please check out our release of SimpleQA Verified ✅ - a new & improved benchmark for reliably measuring progress in short form factuality!

🚀 New Benchmark Launch: SimpleQA Verified! We’ve partnered with @GoogleDeepMind and @GoogleResearch to launch a curated 1,000-prompt benchmark designed to provide a more reliable and challenging evaluation of LLM short-form factuality. Check out the leaderboard here:

really wish more talks in CS/ML were like this (surely seminars, maybe confs?). It's quite obv that over a uniform sample of accepted Neurips papers, Pr[results will be insightful] << Pr[hearing about the "behind the scenes" of the project will be insightful] (for me personally)

🥳🥳 Happy to share that we have three papers accepted to EMNLP 2025 🇨🇳 (2 main, 1 findings)! What makes this special is that all three belong to a new research line I began last year: LLM-as-a-judge/LLM-as-an-annotator 🤖🧑‍⚖️

+100 for this (surprisingly short) take! "writing the paper" is not something that happens at the END of a research project.. it's an integral part of it. personally, blindly offloading that part to an LLM would be the surest way to hurt the quality of my research.

🪻

new blogpost on writing in the ~glorious~ age of LLMs

new work by @pybeebee shows that LLMs still struggle to faithfully express their uncertainty in words, but cool to see that meta cognitive inspired prompting can go a long way. looking forward to seeing more positive results on this fundamental problem!

מצב המשטרה: השתמשה בצ'ט GPT שהמציא עבורה חוק חדש על מנת לנצח בהליך להחרמת פלאפון בדיון בבית משפט השלום בחדרה. השופט היה המום כשהדבר התגלה: "30 שנה אני שופט וחשבתי שראיתי הכל. כנראה שטעיתי"

we write too much. more than we can read, and many small incremental things. i think there should be some mechanism to restrict paper submissions and acceptances per person per year, to force people to prioritize their best work, and invest more in it.

@sama the single biggest thing you could do for safety/alignment is to put a massive emphasis in the RL feedback loop on basic HONESTY and never misleading, tricking, overstating, exaggerating, etc. It should be like touching a hot stove to the model. Just like how you raise kids

This was a great 30-minute conceptual read. It neatly ties together classic RL, LLMs of the past few years, and where agents are headed next. Honestly, I find the future of agents interacting w the world with less human mediation ("experiencing") both exciting and terrifying

[[ for kicks, I asked chatGPT to rewrite my tweet in MAVERICK style. very useful in truly bringing home the message of how obnoxious this response style truly is!!! 💅💅 ]]

tbc, I’m not saying the benchmark is useless. If you're optimizing purely for likability, it’s probably useful to know that the average user enjoys this kind of overly enthusiastic fluff. but it can't be taken seriously as a measure of utility for general-purpose LLMs.

with the battles being public, it’s now glaringly obvious that completely unfactual responses can easily win, so long as they’re delivered in an aggressively upbeat tone and cheerfully long-winded style.

my completely personal take: Llama-4 blatantly gaming the Chatbot Arena evals (beyond being a neat example of Goodhart’s law in action!) is an important moment for the NLP community ⏩

.@percyliang & @tatsu_hashimoto start the 2nd offering of CS336 Language Modeling from Scratch at @stanfordnlp. The class philosophy is Understanding by Building. We need many people who understand the detailed design of modern LLMs, not just a few at “frontier” 🤭 AI companies.