TREC RAG @ 2025
@TREC_RAG
Followers
355
Following
99
Media
4
Statuses
80
Official Twitter account for the TREC RAG Tracks (2024 & 2025)!
Joined February 2024
#TREC2024 Retrieval Augmented Generation (RAG) track is happening live today at TREC! Exciting findings (Retrieval, Nuggets, Support & Fluency) from the organizers and ongoing participants' submissions! 😎 Thanks to all participants, we had lots of runs to evaluate! 😃🙏🏼
0
1
13
TREC RAG 2025 official retrieval baselines are available now! 💥💥💥 Time to start generating those answers and submit them to eval base before August 17th! 🗓️ Let the games begin, well you have less than a month remaining to submit! 🍻
🚀 The official baselines and validation scripts for TREC RAG 2025 are now available! These include both retrieval results (for the AG task) and the corresponding end-to-end augmented generation outputs. Access the baselines and necessary scripts here:
0
4
17
🚨 TREC RAG 2025 update: Official baselines are now available! 📅 Mark your calendars: submissions due August 17
🚀 The official baselines and validation scripts for TREC RAG 2025 are now available! These include both retrieval results (for the AG task) and the corresponding end-to-end augmented generation outputs. Access the baselines and necessary scripts here:
0
3
7
We are out with the official baselines for @TREC_RAG this year: https://t.co/9k5wPI1e2N
@Ushivani3 and I had fun putting together a strong Retrieve (Pyserini) -> Rerank (RankLLM) -> Augmented Gen (Ragnarök) baseline and we hope to see you all beat it!
github.com
Retrieval-Augmented Generation battle! Contribute to castorini/ragnarok development by creating an account on GitHub.
🚀 The official baselines and validation scripts for TREC RAG 2025 are now available! These include both retrieval results (for the AG task) and the corresponding end-to-end augmented generation outputs. Access the baselines and necessary scripts here:
0
1
5
📅 We'd also like to remind participants that the submission deadline is August 17. Please review the updated track guidelines to ensure your outputs conform to the expected format: https://t.co/DbnA4dQF13 We look forward to your submissions!
1
0
2
🚀 The official baselines and validation scripts for TREC RAG 2025 are now available! These include both retrieval results (for the AG task) and the corresponding end-to-end augmented generation outputs. Access the baselines and necessary scripts here:
1
2
15
TREC RAG TOPICS ARE OFFICIALLY OUT NOW!!! 🔥🔥🔥 LET THE GAMES BEGIN! 🥶
📢📢RAG 2025 topics are officially now released! 🔍Test narratives are out now (total 105): https://t.co/Eg6IuI2AYA Let the games begin! #TREC2025 #RAG
0
5
13
We have released the @TREC_RAG 2025 topics and will be out with strong baselines soon. But for those who are eager, go right ahead!
📢📢RAG 2025 topics are officially now released! 🔍Test narratives are out now (total 105): https://t.co/Eg6IuI2AYA Let the games begin! #TREC2025 #RAG
0
2
2
📢📢RAG 2025 topics are officially now released! 🔍Test narratives are out now (total 105): https://t.co/Eg6IuI2AYA Let the games begin! #TREC2025 #RAG
trec-rag.github.io
The TREC 2025 RAG Competition is Now Live!
0
3
18
We had a successful participation of over 45+ teams & 150+ runs last year in TREC RAG 2024! 🔥🔥 We are back with @TREC_RAG 2025 this year! Make sure you participate in one of the RAG tracks this year to help accelerate RAG evaluation! ⚡⚡ Signup here:
docs.google.com
We are having some systems issues, so in the meantime, we are taking registrations through this form. Please use a direct email address and not a mailing list or group alias as the contact email.
📝 The tentative schedule for the TREC 2025 #RAG is released now! 😎 Details in 🧵: 1️⃣ Test topics & baselines: Mid July 2025 2️⃣ Submission deadline: Mid August 2025 3️⃣ Results & judgments returned: October 2025 4️⃣ TREC 2025: November 2025
1
3
8
For all other live updates on the TREC RAG 2025 track, please follow the website homepage. Please RT, share and amplify! 🙏🏻 https://t.co/Vjz9el1gS9
trec-rag.github.io
TREC RAG Official Website
0
0
1
We are using the same corpus as last year: MS MARCO v2.1 document with 10M docs & (chunked) MS MARCO v2.1 segment collection with 113M segments. So a good news for all those who created a dense index last year 🙃 https://t.co/KjjLNmYJPq
trec-rag.github.io
For TREC 2025, we continue to utilize the MS MARCO V2.1 document corpus and its segmented counterpart, as introduced in TREC 2024. This version addresses deduplication concerns and provides enhanced...
1
0
0
The 2025 RAG track guidelines are now available! We largely follow the 3 same tasks as last year: R task, AG task and RAG task! 🔎⚡ There are few changes in the I/O format, please refer to those here: https://t.co/DbnA4dRcQB
trec-rag.github.io
Version 2.0. Last Updated: Aug 13, 2025
1
0
1
Now we know RAG is alive (again). I'll be talking about evals this, evals that, evals!
Phew, thank God my boring talk is over, and everyone now knows RAG is alive. Now we can all focus on the good stuff. Next week, @beirmug will finally answer one of the most asked questions: how do I, in 2025, know which models I should trust to be good on unseen data?
4
4
24
2024 has been a fun year! Initial collab with @vectara on multilingual RAG benchmarking to coordinating a first-ever RAG track (@TREC_RAG) in TREC 2024 and lastly getting to intern at @DbrxMosaicAI! I predict 2025 will be the year of diversity & realistic evals in IR & RAG!
0
4
11
As @ian_soboroff notes kicking off TREC 2024, the 33rd iteration of the series, this is the 30th anniversary of BM25, introduced at TREC-3: https://t.co/YTgxu3N1su
1
9
40
🎙️ We bring to you UMBRELA ⛱️ relevance judgments for the TREC 2024 RAG topics! It should be a good proxy for evaluating your retrieval pipelines with the TREC 2024 RAG test topics. New topics to assess your systems! In time, other judgments shall be made public too. 🧙♂️
TREC RAG 2024 fully automated qrel (using UMBRELA) is officially now released! Check out the website for the qrel (with 301 topics): https://t.co/U3KPtuUoOR
#RAG #IR #TREC2024
0
2
12
⚗️ UMBRELA qrels for the TREC 2024 RAG Track have been posted!
arxiv.org
The application of large language models to provide relevance assessments presents exciting opportunities to advance information retrieval, natural language processing, and beyond, but to date...
TREC RAG 2024 fully automated qrel (using UMBRELA) is officially now released! Check out the website for the qrel (with 301 topics): https://t.co/U3KPtuUoOR
#RAG #IR #TREC2024
0
1
12