Introducing HUMAINE: the LLM benchmark that puts real human experience first 🎯 21,352 human evaluators. 27 models. 22 demographic groups. 5 evaluation dimensions. In partnership with @huggingface. See insights below 🧵

Its best rank is in Interaction Fluidity & Adaptiveness, where it stands at #6. Gemini 2.5 Pro, DeepSeek V3, and Magistral Medium remain the top 3 human-preference models. See the latest movements and demographic preferences ➡️ https://t.co/fXTvQYFLMk

🆕 @AnthropicAI's Claude Sonnet 4.5 joins the HUMAINE leaderboard at #11 overall and as the top Claude model. Sonnet 4.5 outperforms other @ClaudeAI models in head-to-head comparisons and indicates improvements across the board. Will we see more improvements in the next model?

It was a pleasure to sit down with @IBM’s Dr Bo Wen to discuss the evolving human role, from data provider to AI manager. Hosted by @ana_in_sf, the session unpacks Dr Bo Wen’s paper "The Missing Reward: Active Inference in the Era of Experience.” Watch on demand 👇

Hello, NEW YORK! 🍎🗽 Calling all AI devs and builders! We’re heading to the AI Dev 25 @DeepLearningAI event on November 14th with @Prolific, come say hi! 💻 Visit our demo table: See how to collect human evals in minutes 👩‍🏫 Join us in the private room: Go deeper into your use

Pleased to be supporting @DeepLearningAI's AI Dev 25 x NYC, November 14th. Be sure to stop by the Prolific table for a cool demo on rapid, quality human evals and connect with the team 🦾

How to design studies and get quality data with Dr. Simon Jones 🎥 Join our Senior Research Consultant as he shares expert tips to optimize data quality, and runs through the latest platform features designed to make launching studies faster and easier than ever. Register 👇

Major product updates are live 🔥 Authenticity checks (our AI-generated response detection tool) out of beta, an expanded specialist pool, AI Task Builder interface upgrades for complex annotation, and more. Click for the full list of what's new 👇

330+ Candidates Signed up! Excited to have kicked off the world’s first global multi-modal AI hackathon in London last week. Big shout out to our partners @Prolific @hume_ai @RunwareAI https://t.co/Zx5InK3tKB @aiengine_hack @SusaVentures

We’re excited to dive into HUMAINE more at tomorrow’s networking event in London. Event details ➡️ https://t.co/SqFIUT9wiH Check out the methodology on @huggingface ➡️

Gemini is currently the leading LLM for real-world use 🏆 Our HUMAINE benchmark involves 27 models, 20,000+ diverse humans, and 100,000+ pairwise comparisons. @GoogleDeepMind's Gemini-2.5-Pro takes top spot in 4 of 5 evaluation dimensions (97% of statistical simulations).

Imagine a powerful integration with updates like a delivery app 📱 @Ballpark CEO @mutlu82 noted new possibilities for better customer experience, speed, scale, and quality after integrating with Prolific. Watch the Ballpark x Prolific story ➡️ https://t.co/NqWJ5AEGOS

Last chance to sign up 🚨 Join @ana_in_sf and @IBM's Dr Bo Wen online tomorrow @ 3:00 PM PDT as they explore “Humans as AI Managers” and the evolving human role in the era of experience, from passive contributors to active architects of AI systems. Register and watch live ↓

Are chatbots swaying public opinion? @AISecurityInst just released findings from their large-scale study of AI use for political information-seeking. We’re pleased to have supported this research, providing 2,858 participants for randomized controlled trials. Check it out 👇

Why do AI leaderboards miss the mark? Join our exclusive drinks reception for AI leaders in London 🚨 On October 9th, we're bringing the sharpest minds together to engage in candid discussions about how current AI model benchmarks can be improved. RSVP now ↓

AI researchers and model builders know poor human data = wasted compute, flawed models, and delayed progress. Better AI starts with better human data. Explore the guide to see how Prolific helps you apply these 4 principles of high-quality data collection 👇

HUMAINE captures how diverse users experience AI across multiple dimensions, from task performance to trust and safety. Check out the latest findings on @huggingface: https://t.co/x1wLUxX2ty 📊

Top LLMs that capture the human experience right now ↓ On the HUMAINE human-centered model benchmark, @GoogleDeepMind's Gemini-2.5-Pro is on top overall, and in 4/5 evaluation dimensions. @deepseek_ai's V3, @MistralAI's Magistral Medium, and 2x @Grok models made top 5 overall.

Great to be at the @memories_ai Global Multi-Modal hackathon opening ceremony today in London! At @Prolific we can’t wait to see what the participants build during this month! Want to participate? Sign up here: https://t.co/EpXG9Sa81N

This epic event is beginning today 🙆🏻‍♀️ Sign up here: https://t.co/Y2oTuPpgAo @heybossAI @Prolific @RunwareAI @hume_ai

What's one issue where @GOP and @TheDemocrats voters actually agree? Strangely enough its on the decline of the US as a global superpower 😬 This months @Prolific poll surveyed 1,950 US adults about which country they see as the pre-eminent global superpower today vs. 10 years