Agents are joining us at work -- coding, writing, design. But how do they actually work, especially compared to humans? Their workflows tell a different story: They code everything, slow down human flows, and deliver low-quality work fast. Yet when teamed with humans, they shine

We use LLMs for everyday tasks—research, writing, coding, decision-making. They remember our conversations, adapt to our needs and preferences. Naturally, we trust them more with repeated use. But this growing trust might be masking a hidden risk: what if their beliefs are

New eval! Code duels for LMs ⚔️ Current evals test LMs on *tasks*: "fix this bug," "write a test" But we code to achieve *goals*: maximize revenue, cut costs, win users Meet CodeClash: LMs compete via their codebases across multi-round tournaments to achieve high-level goals

Ever used a top-scoring coding agent that still couldn't help with your actual task? How can we transform agents from "interns who never learn" to "workers who improve over time"? Read about our new approach: ToM-SWE 🪄

This paper makes an important point. AI agents don’t naturally follow human workflows — they often default to coding everything, even when humans would reason, sketch, or explore first. This can make them fast, but also awkward in tasks that rely on interpretation or judgment.

Agent trajectories are often left underexplored while showcasing rich behaviors. Really appreciate this offering a thoughtful snapshot of human–agent comparisons, reading like a compact and detailed GDPEval.

Analyzing agent trajectories has become a new canonical way for evaluation. Yet most trajectories operate at the action level - dense but semantically shallow. We define 𝙬𝙤𝙧𝙠𝙛𝙡𝙤𝙬 as a higher-level abstraction and release a toolkit to induce it from raw actions for both

We recorded a bunch of people actually working on their computers (!) and then compared agent performance to actual human workflows. Awesome paper led by @ZhiruoW :)

Check out our new work on examining how human and AI workers perform tasks! We get human and AI workers to perform the same tasks, extract workflows, and get insights about where agents do well and where they still have a ways to go.

@ZhiruoW 's research compares AI agents vs humans across real work tasks (data analysis, engineering, design, writing). Key findings: 👉Agents are 88% faster & 90-96% cheaper 👉BUT produce lower quality work, often fabricate data to mask limitations 👉Agents code everything,

Check out our: 📄 Paper: https://t.co/R0v58gxwq3 🧩 Workflow induction tool: https://t.co/CjwTb3sZhg With the fantastic team @EchoShao8899 @oshaikh13 @dan_fried @gneubig @Diyi_Yang at @LTIatCMU & @stanfordnlp

As agents keep climbing up that "autonomy slider" We'll need: - stronger visual understanding - better action calibration, and - workflow-inspired agent designs

Quality? Still shaky. Agents fabricate data, misuse tools -- but finish tasks 88% faster and at a fraction of the cost. Maybe the question isn’t agents vs humans.. But how do we team humans & agents?

AI doesn’t always speed us up. 🧠 Augmentation → +24.3% faster, minimal disruption. ⚙️ Automation → -17.7% slower, workflows reshaped by verification & debugging.

Our findings reveal a striking divide: Agents approach all work programmatically -- even open-ended, visual tasks like design. Humans rely on UI-centric, perceptual interaction. Their workflows may look similar at a high level, but low-level processes are worlds apart.

We induce workflows to: - uniformly represent diverse work activities - enable systematic comparison across workers

We directly compared humans and AI agents. Across computer-use jobs. Via 5 essential work skills: data analysis, engineering, computation, writing, and design.

Very excited to see Zora and her work recognized! Check out her research on skill learning for LLM agents, memory, and programmatic approaches to real-world tasks (even beyond code).

Congrats to @ZhiruoW on getting chosen as a Google PhD fellow! If you haven't checked out Zora's work on agents that learn from experience this is a good excuse to do so 😄

🎉Thrilled to be named a Google PhD Fellow! Feeling incredibly grateful for my advisors, collaborators, and friends who’ve supported and inspired me along the way 🫶 Looking forward to the journey ahead!