Code agents don’t just talk -- they execute. What happens when you jailbreak them? Announcing JAWS-Bench (from my summer at @amazon AWS): a benchmark to jailbreak code agents across 3 workspaces -- empty → single-file → multi-file. The results? They break. A lot. Details 🧵👇

Yesterday I asked what makes you happy. 226 voted--61% said family & friends, like those AI researchers at the panel. Not AI, just the people around us. This resonates with Harvard's 87-year study on happiness. Their conclusion: happiness is not about money/fame/success, it's

Links + credits: Paper: https://t.co/NGlSpXbUZQ Repo: ⏳coming soon Special thanks to my mentors @Jifan_chen, Sam Mayers, @SanjayKrishnaG9 , and managers @zijianwang30 and @varun_kr, for all their support and help. 🙏 #AIsecurity #LLMSafety #CodeAgents #ResponsibleAI

Takeaway: guardrails should gate RUN, reason over diffs/imports, and persist refusals across steps. Prompt filters alone won’t cut it -- especially in multi-file repos. 🛡️

Safety must be execution-aware. We measure not only refusals, but whether the code actually runs. Our agentic judge checks: Refusal → Harmfulness → Syntax → Runtime (did it run?). That last stage is where real risk lives. 🧑‍⚖️⚙️

Agentification matters: pairing a code LLM with an agent stack (tools, planning) makes it ~1.6× more vulnerable. Agents can talk themselves past refusals during tool use. 🤖🔧

Add minimal context (files/repo) and average jailbreak rates climb to ~75%. A little code scaffolding flips more “no”s into working exploits. 🧩

Prompt-only (no files): agents jailbreak ~61% of the time – and ~27% of those cases produce instantly deployable attack code. 💥

Why three setups? Empty tests pure prompt attacks, single-file adds minimal code hooks, and multi-file simulates real repos. This lets us measure how added context boosts vulnerability.

⚡️ ACL 2025 poster alert! Catch “Almost AI,  Almost Human: The Challenge of Detecting AI‑Polished Writing” tomorrow on Gather — 18:00–19:30 CEST / 12 PM EDT, Booth 5103. Let’s chat how minimal AI-polishing fools big detectors 🤖✍️🔍 #ACL2025 #NLP

So true (at least for ML conferences)! Something that I learned over the years…

🔥What if you could humanize any AI-generated text to fool ANY detector? 🚨We present Adversarial Paraphrasing—A universal attack that breaks a wide range of detectors without fine-tuning or detector knowledge. Just pure evasion. 🔗 https://t.co/zA1000eBA7 👇 Thread below.

🚨 Just aired! I had the opportunity to speak with #CBS News about the latest in AI text detection, a topic that's critical from education to online communication. 🎥 Watch the full segment here: https://t.co/XW2OMwSpbf

🔗 Paper: https://t.co/8KhHVyUjJU 💻 Code: https://t.co/4InXJJiPf1 📊 Dataset: https://t.co/o2ZtMavUct Big thanks to my amazing mentor @FeiziSoheil , for all the support! 🙏

📄 AND the paper got accepted to #ACL 2025! 🎖️ It dives deep into how AI-polished text confuses state-of-the-art detectors — and how we built APT-Eval to study this.

📰 My first-author paper got featured in The New York Times @nytimes! They covered our work on AI-text detection and the challenges of spotting subtly AI-polished writing. 👉 NYT article:

🚨 Double Feature in 24 Hours?! 🎉 I’m thrilled to share TWO major updates—both within a single day! Paper got covered by @nytimes and got accepted into @aclmeeting!! 🎉🎉 Details below 👇

I’m excited to share our latest work at #SaTML2025, "ML-Based Behavioral Malware Detection Is Far from a Solved Problem"! Joint work with @surrealyz, @MarcusBotacin, @ShoumikSaha7, @fbpierazzi, @lcavallaro, David Wagner, Tudor Dumitraș 🔗Website: https://t.co/UhGi0aVRkc 👇(1/5)

Thread 3/3 We analyzed this issue in one of our recent works - Paper: https://t.co/8KhHVyUjJU Code:

Thread 2/3 Cases like this raise concerns about the reliability of AI-text detectors. As LLMs rapidly evolve, detecting AI-generated content becomes ever more challenging — especially when many people genuinely use these tools to polish and improve their writing.