We built Agents in the Wild to track this revolution in real-time:.👉 Live, open-source dashboard.👉 Tracks agent behavior across public GitHub PRs.👉 Updated daily.Code: Built with our MSc student @Christian Mürtz (@SRILab) 🙌.

🏢 The enterprise trust gap. Agent PRs merge 95% of the time on small repos. But on large/popular ones? Drops to ~25%, even for Google’s Jules. Without strong validation, agents will struggle in business-critical environments. 5/n.

🔍 Manual guardrails are still needed. Agents like Codex & Jules require user review before PR submission. That boosts merge rates by up to 50% vs Copilot. But it also limits autonomy. Review = trust, but trust = bottleneck <- this is what we are working on at LogicStar 4/n 🧵.

🧠 How agents code. Agent-generated PRs are:.– Dominantly in Python / JS / TS.– More likely to add new code than refactor existing code. They’re focused on building features, not fixing bugs or doing maintenance. 3/n 🧵.

Generate videos in just a few seconds. Try Grok Imagine, free for a limited time.

🧪 Still early days. Agents generate 5–10% of PRs overall but only 1–2% on popular repos. Most of their activity is in small, low-starred projects. Translation: we're still in the experimentation phase. 2/n 🧵.

🚨 AI agents wrote 7% of all GitHub PRs in June. But can we trust their code?. We built Agents in the Wild – a live dashboard tracking autonomous AI agents across GitHub to answer that question: Here’s what we learned from analyzing 10M+ PRs 👇 1/n 🧵.

RT @logic_star_ai: We are excited to see the community use our SWT-Bench and work on the crucial topic of test generation!.

RT @nielstron: SOTA code agent OpenHands (top-1 for SWE-full) achieves only 22% accuracy in unit test generation on SWT-lite (half its SWE….

RT @logic_star_ai: We have our first submission for SWT-Bench 🚀.AEGIS, a dedicated test generation agent, achieves 47.8% accuracy 🏆 , signi….

RT @logic_star_ai: 🚀 Introducing the SWT-Bench Leaderboard!.Test your AI's ability to write tests reproducing real-world GitHub issues and….

Meet me at this morning's NeurIPS poster session to discuss our work on generating reproducing test cases with Code Agents.

RT @logic_star_ai: Exiting to see our work on benchmarking the test-generation capabilities of LLMs being picked up by the community!.

RT @nielstron: Presenting today @icmlconf 2024 Workshop FM in the Wild 🤖 🏞️. "Code Agents are State of The Art Software Testers". SWE-Agen….

RT @marc_r_fischer: On Tuesday at 11:30, in Poster Session 1, we will present Prompt Sketching, a novel decoder-driven approach for templat….

Excited to share our latest work which we will present today at @iclr_conf.

RT @mvechev: A couple of amazing PhD students graduated from our lab (@the_sri_lab) at ETH Zurich today: @mbalunovic and @mnmueller. Both d….

RT @the_sri_lab: Find us @NeurIPSConf #NeurIPS2023 to chat about our latest work. We are excited to share works on certified robustness, a….

RT @the_sri_lab: @mnmueller and @marc_r_fischer introduced a new from of Abstract Interpretation for challenging unbounded loops enabling t….

Super excited to talk about robustness guarantees for neural networks at @mlsec_lab's seminar!.

RT @the_sri_lab: At @iclr_conf members of SRI lab presented 3 works:.- ⚖️ Human-Guided Fair Classification for NLP.- 📈 Robustness Verificat….