llm4eval Profile Banner
LLM4Eval Workshop Profile
LLM4Eval Workshop

@llm4eval

Followers
84
Following
40
Media
9
Statuses
44

The Third Workshop on Large Language Models for Evaluation in IR (LLM4IREval@SIGIR2025)

Padua, Italy
Joined February 2024
Don't wanna be here? Send us removal request.
@llm4eval
LLM4Eval Workshop
4 months
LLMs are reshaping how we evaluate ranked lists, conversations, and even simulate users. At #LLM4Eval @ #SIGIR2025, we’re spotlighting 3 special themes:.🧠 Beyond query-doc.🗣️ Simulating users.🧪 Synthesizing evaluation data.👇 Curious? Read on & submit by April 23!.#LLMs #IR.
@llm4eval
LLM4Eval Workshop
4 months
📢 Call for Papers: LLM4EVAL @ SIGIR '25 - The 3rd Workshop on Large Language Models for Evaluation in IR! @SIGIRConf . 🗓️ Submission: April 23, 2025 (AoE).🔗 Full CFP: 🌐 Workshop:
1
9
15
@llm4eval
LLM4Eval Workshop
9 days
Chirag Shah is this morning giving a keynote on: From Relevance to Reality: Scaling Human-Centered Evaluation in the LLM Era
Tweet media one
0
2
5
@llm4eval
LLM4Eval Workshop
9 days
You are welcome to join us at Giotto Platea for the LLM4Eval workshop #SIGIR2025 .
0
1
3
@llm4eval
LLM4Eval Workshop
3 months
The deadline for submitting your LLM evaluation paper is tomorrow. Looking forward to interesting contributions!.
@llm4eval
LLM4Eval Workshop
3 months
📢 Deadline extended!.Submit to #LLM4Eval @ #SIGIR2025 by April 30 (AoE). We welcome work on LLM-based evaluation, including:.🧠 beyond query-doc.🗣️ simulating users.🧪 synthetic eval data. 🔗 #CFP #IR #LLMs #SIGIR2025.
0
0
3
@llm4eval
LLM4Eval Workshop
3 months
📢 Deadline extended!.Submit to #LLM4Eval @ #SIGIR2025 by April 30 (AoE). We welcome work on LLM-based evaluation, including:.🧠 beyond query-doc.🗣️ simulating users.🧪 synthetic eval data. 🔗 #CFP #IR #LLMs #SIGIR2025.
llm4eval.github.io
Call For Paper
@llm4eval
LLM4Eval Workshop
4 months
LLMs are reshaping how we evaluate ranked lists, conversations, and even simulate users. At #LLM4Eval @ #SIGIR2025, we’re spotlighting 3 special themes:.🧠 Beyond query-doc.🗣️ Simulating users.🧪 Synthesizing evaluation data.👇 Curious? Read on & submit by April 23!.#LLMs #IR.
1
7
9
@llm4eval
LLM4Eval Workshop
3 months
RT @maliannejadi: We will host multiple breakout discussions to pick your brains 🧠 on the future challenges and research on this topic. Let….
0
4
0
@llm4eval
LLM4Eval Workshop
4 months
🧵 Theme 3:. 🧪 Synthesizing evaluation data: When judgments are scarce, can we generate evaluation data with LLMs?. We want approaches for creating evaluation datasets in low-resource or privacy-sensitive scenarios, where collecting real data is hard, expensive, or impossible.
0
2
3
@llm4eval
LLM4Eval Workshop
4 months
🧵 Theme 2:. 🗣️ Simulating users: What if LLMs could act as realistic user proxies?. We are looking for papers on simulating user behavior, preferences, context, or personas for evaluation.
1
2
3
@llm4eval
LLM4Eval Workshop
4 months
🧵 Theme 1:. 🧠 Evaluation beyond query-doc: How can LLMs evaluate full sessions, ranked lists, or multi-turn conversations?. We welcome work that goes beyond traditional Qrels and explores richer evaluation units.
1
2
2
@llm4eval
LLM4Eval Workshop
4 months
In this series, we move BEYOND search relevance to tackle: .✨ Personalized search evaluation .✨ Conversational interfaces .✨ Holistic assessment of integrated experiences .✨ Bias & fairness in LLM evaluations.
1
3
2
@llm4eval
LLM4Eval Workshop
4 months
📢 Call for Papers: LLM4EVAL @ SIGIR '25 - The 3rd Workshop on Large Language Models for Evaluation in IR! @SIGIRConf . 🗓️ Submission: April 23, 2025 (AoE).🔗 Full CFP: 🌐 Workshop:
llm4eval.github.io
The Third Workshop on Large Language Models for Evaluation in IR (LLM4Eval @ SIGIR 2025)
1
8
13
@llm4eval
LLM4Eval Workshop
4 months
RT @TechAtBloomberg: In today's 2nd Workshop on #LLMs for Evaluation in #InformationRetrieval (#LLM4Eval) at #WSDM2025 (09:15 CET), @edgarm….
0
4
0
@llm4eval
LLM4Eval Workshop
7 months
RT @srahmanidashti: The workshop will focus on models, techniques, data collections, and methodologies for information retrieval evaluation….
0
2
0
@llm4eval
LLM4Eval Workshop
7 months
0
1
0
@llm4eval
LLM4Eval Workshop
7 months
RT @srahmanidashti: The call for papers is now open for the second workshop of #LLM4Eval:. 💡 See CFP: 🗓 Submission….
0
3
0
@llm4eval
LLM4Eval Workshop
7 months
RT @srahmanidashti: 🔊 The second workshop on Large Language Models for Evaluation in IR (#LLM4Eval), co-located with #WSDM2025, is now acce….
0
6
0
@llm4eval
LLM4Eval Workshop
11 months
RT @srahmanidashti: 📢 Thrilled to announce that, "SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval" is publicly availab….
Tweet card summary image
arxiv.org
Large-scale test collections play a crucial role in Information Retrieval (IR) research. However, according to the Cranfield paradigm and the research into publicly available datasets, the...
0
15
0
@llm4eval
LLM4Eval Workshop
1 year
RT @guglielm0f: Great panel discussion with @ian_soboroff, @lauradietz99, @claclarke, Michael Ekstrand moderated by @UnderdogGeek at @llm4e….
0
6
0