Ronak Pradeep
@rpradeep42
Followers
665
Following
1K
Media
38
Statuses
330
PhD at @UWaterloo - LLMs + IR, @TREC_RAG. Now: @yupp_ai, Previous: @Apple @GoogleAI. There is no dark side in the moon, really. Matter of fact, it's all dark.
Joined June 2019
Our paper with @vectara, “Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards”, is now published in the EMNLP 2025 Industry Track! Check out our work on enabling more reliable LLM faithfulness benchmarking in RAG!
Introducing 🔍 FaithJudge: improving how we evaluate LLM faithfulness in RAG tasks, including summarization, QA, and data-to-text and powering a more accurate LLM Hallucination Leaderboard. 🔗
1
3
6
We @yupp_ai just shipped Help Me Chose 🚀 Now LLMs don’t just respond, they self-critique & cross-check each other 🤖⚔️🤖 At day's end, you’re the arbiter of your own taste! Fun example where @OpenAI's GPT 5 & @xai's Grok 4 go at it & learn from the each other (AND SO DO YOU!).
Today, we are launching “Help Me Choose” in @yupp_ai – a new product feature where multiple AIs critique each other and debate among themselves to help users synthesize diverse perspectives and get the best answer out of their own “AI council”.
1
0
7
We’ve raised our $10M Series A, led by Google Ventures. 18 months ago, when we started @useblacksmith, building a CI cloud purpose-built to run CI workloads as fast as possible seemed like a pipe dream to us. It’s reasonable to say that we’ve made that a reality since. To give
17
14
132
Did I say four? Thirteen (: Standard, High, Low, Minimal Reasoning variants for each of GPT-5, mini, and nano! Here's a case where more reasoning definitely helps. Check out https://t.co/mZmYqeVmvZ and the songs!
Four GPT-5 variants free for y'all on Yupp! Enjoy and more soon ;)
0
0
1
Four GPT-5 variants free for y'all on Yupp! Enjoy and more soon ;)
📢 New Model Drop: After weeks of fanfare, GPT-5 by @OpenAI is here, and is available right now for free on Yupp! OpenAI’s smartest, fastest, and most useful model yet, with thinking built in.
0
0
2
Along with GPT5, we're open sourcing a new eval, BrowseComp Long Context! It improves upon existing long context qa evals in data quality and input difficulty. Work with @LK112358, @julieswangg, and our mascot the longham. A bit more below
9
6
46
We ship fast! Check out these models.
0
0
3
Excited to see this go out and see it used beyond IMO -- congrats to the team!! Happy to have contributed some research to this model with @YiTayML and @HuaixiuZheng :D
We’re bringing a version of Deep Think that achieved gold-medal status at IMO to Ultra subscribers in the @Geminiapp (+ the official version is now in the hands of mathematicians). Toggle it on when reasoning through complex scientific literature, tackling a coding problem that
0
3
42
We are out with the official baselines for @TREC_RAG this year: https://t.co/9k5wPI1e2N
@Ushivani3 and I had fun putting together a strong Retrieve (Pyserini) -> Rerank (RankLLM) -> Augmented Gen (Ragnarök) baseline and we hope to see you all beat it!
github.com
Retrieval-Augmented Generation battle! Contribute to castorini/ragnarok development by creating an account on GitHub.
🚀 The official baselines and validation scripts for TREC RAG 2025 are now available! These include both retrieval results (for the AG task) and the corresponding end-to-end augmented generation outputs. Access the baselines and necessary scripts here:
0
1
5
We've onboarded the Gemini 2.5 Flash-Lite along with variants (Thinking, Online, etc.) super quick on @yupp_ai and are already gathering preferences! Check out the thread for more. Here's a fun comparison of the thinking variant (left) with the standard one!
At @yupp_ai we add new models as soon as they drop! @sundarpichai tweeted about Gemini 2.5 Flash-Lite yesterday. Soon after, it was available to our global users. Less than 24h later we’ve gathered 10K+ global user preferences! And it looks great… 🧵 https://t.co/1QSueiXvZL
0
0
3
We’ve been rapidly onboarding models! Do check the two of them out.
📢 New Model Drop: Solar Pro 2 by @upstageai is live on Yupp! Solar Pro 2 is pushing the frontier in reasoning, tool use, and multilingual performance; built to power complex tasks and agent-like workflows across domains. Let’s see how it stacks up:
0
0
4
We use the same web collection as the TREC RAG Track. You can easily adapt your RAG systems for our track to see its performance in helping people better understand daily news.
0
1
1
📢📢RAG 2025 topics are officially now released! 🔍Test narratives are out now (total 105): https://t.co/Eg6IuI2AYA Let the games begin! #TREC2025 #RAG
trec-rag.github.io
The TREC 2025 RAG Competition is Now Live!
0
3
18
We have a poster on Assessing Support for TREC RAG and another for RankLLM by @Sahel_Sharify today at #SIGIR2025. Do make sure to check them out!
0
0
1
We have released the @TREC_RAG 2025 topics and will be out with strong baselines soon. But for those who are eager, go right ahead!
📢📢RAG 2025 topics are officially now released! 🔍Test narratives are out now (total 105): https://t.co/Eg6IuI2AYA Let the games begin! #TREC2025 #RAG
0
2
2
36 hours and over 6K votes later, you have a thread from @lintool on takeaways from our end @yupp_ai on @xai's Grok 4!
It’s been 36 hours since Grok 4 launched and we have an early verdict based on 6K+ preferences of @yupp_ai users globally on real use cases. ‼️ Grok 4 is worse than other leading models: OpenAI o3, Claude Opus 4, and Gemini 2.5 Pro. Grok 4 is liked even less than Grok 3. 🧵
0
1
2
4 weeks since launch & we @yupp_ai have gathered 2M+ preference data on 500+ models. Building a leaderboard capturing the nuances of the global community has been loads of fun. Check out the thread! Onwards🚀
It’s been ~4 weeks since we launched @yupp_ai – a consumer-first approach to robust & trustworthy AI evaluation. We’re still early but have already gathered 2M+ high-quality human preference feedback datapoints on 500+ models across diverse use cases. 🧵 https://t.co/jmJK4lKJcl
0
0
7
We are back again with TREC RAG this year! Do check it out and stay tuned for more interesting updates!
📝 The tentative schedule for the TREC 2025 #RAG is released now! 😎 Details in 🧵: 1️⃣ Test topics & baselines: Mid July 2025 2️⃣ Submission deadline: Mid August 2025 3️⃣ Results & judgments returned: October 2025 4️⃣ TREC 2025: November 2025
0
1
5
More on this thread by @lintool about how this ties into a lot of our group's work including nuggetization for RAG and the sort - https://t.co/KXrL7LqQpi Above all, looking forward to y'all using it and sharing feedback!
In December 2024 @pankaj @gilad @willhorn and I put out a rather cryptic arXiv paper musing about the future of search: https://t.co/CfpU5E2HxN. I’m now able to share what I’ve been up to! 🧵(1/9)
0
0
2