1️⃣We are excited to open-source syftr: a powerful tool for automatically finding Pareto-optimal generative AI flows! syftr searches a large search space of agentic and non-agentic flows to surface optimal tradeoffs between accuracy, cost and latency. 🧵

RT @WenSun1: Does RL actually learn positively under random rewards when optimizing Qwen on MATH? Is Qwen really that magical such that eve….

RT @sytelus: A different and interesting work from my ex-colleague Dey: How do you generate Pareto frontier for the agentic workflow? . Man….

RT @roma_glushko: ✨Meet syftr, a new OSS framework to find the best RAG workflows (both agentic and not) balancing cost/latency/accuracy us….

6️⃣Want to get involved?.📖 Technical blog post and full paper (to appear at @automl_conf ). 💻 Try syftr 🙌 Contribute via PRs.

5️⃣syftr is made possible thanks to:.Ray for distributed search orchestration. @anyscalecompute .LlamaIndex for building advanced workflows. @llama_index .HuggingFace Datasets for fast dataset interfaces. @huggingface . Starting with question-answering and actively expanding tasks.

4️⃣Why syftr?.Models are part of complex workflows in the real world, and syftr evaluates them within those contexts. syftr complements benchmarks which evaluate intrinsic capabilities of models. We're developing in the open to build trust and share the best combinations.

3️⃣Here's what syftr does:.🔍 Takes your grounding dataset and question-answer pairs to find the best workflows. 🎯 Uses multi-objective Bayesian Optimization to identify Pareto-optimal solutions. ⚙️ You choose the ideal workflow for your application based on the Pareto-frontier.

2️⃣Are you struggling with questions like:. 🤔 "Which synthesizing LLM and embedding model to use?".🤖 "Should I adopt the latest agentic flow?".📊 "How do I balance accuracy, cost and latency in my AI workflows?". That's why we created syftr.

RT @sytelus: Another important development for achieving o1 like test-time compute scaling is Entropix by @_xjdr. Both of these ideas coinc….

We are growing our fundamental AI research team at DataRobot and are looking for strong researchers with proven publication track record in deep learning in general and generative AI in particular. Please apply at: #GenAI #DeepLearning.

RT @crwhite_ml: 🚨Llama 3.1 405B eval just dropped🚨.🥇 in instruction following.🥈 in reasoning.On par with GPT-4o in math and coding.It’s a g….

RT @crwhite_ml: OpenAI strikes back 💫 GPT-4o-mini is a remarkable model for its price! Check out its performance on .

This is why we need private benchmarks or ones like which change fast to prevent gaming.

RT @chinganc_rl: Super excited to announce our cool project, Trace, on optimizing general AI systems, using LLMs.😎. Trace is a new AutoDiff….

RT @crwhite_ml: Wow! 😮 claude-3.5 is an extremely impressive overall model! It achieves the top score in **every category**, and substantia….

RT @micahgoldblum: 🚨 Announcing LiveBench, a challenging new general-purpose live LLM benchmark! 🚨.Thanks @crwhite_ml and @SpamuelDooley fo….

RT @WenSun1: REBEL is one of the simplest algorithms and implementation out there that can achieve this performance, e.g., no online GPT4 q….

RT @joao_gante: New sampling strategy dropped in 🤗 transformers -- Min P sampling 🔥. Are you tired of having `top_k` arbitrarily discarding….

RT @g_k_swamy: My advisor Drew recently gave a lecture on the past, present (i.e. my work!), and future of imitation learning and how it ap….