So excited to share that (my first paper ever😎) our paper "Estimating Query Performance Through Rich Contextualized Query Representations" has been accepted to @ecir2024 w/ @NegarEmpr , Maryam Khodabakhsh and @ebrahim_bagheri. #ECIR2024

I thought writing the thesis was the hard part…

Had an amazing time at @COLM_conf 🍁! Great talks, inspiring research, and wonderful conversations with the NLP community.

This paper is an extension of our ECIR 2024 work: “Estimating Query Performance Through Rich Contextualized Query Representations” ✨ If you missed it, you can read it here 👉

QSD-QPP isn’t limited to one setting! It works for both: 🔹Pre-retrieval (QSD-QPP_Pre): lightweight, interpolates from nearby queries. 🔹Post-retrieval (QSD-QPP_Post): enriches predictions by nearby queries and their performance.

Excited to share that our paper has been accepted at ACM TIST 🎉 We introduce QSD-QPP, a framework that predicts query effectiveness by leveraging distances in a neural query space. ⚙️Code is available here: https://t.co/3eDOmruUmQ 📖Paper: https://t.co/wlTsiQ82mo #TIST #QPP

Proactive Conversational Information Seeking with Large Language Models (ProActLLM) 🔗 https://t.co/7Olz1b00j6

9/ Huge thanks to my brilliant co-authors: Soroush Sadeghian, Ali Ghorbanpour, @NegarEmpr , @sara_slmt , Muhan Li, Hai Son Le, Mahdi Bashari, and @ebrahim_bagheri for making this possible 🙌 Looking forward to seeing you in Seoul! 🇰🇷 #RottenReviews #Reviewerly #CIKM2025

8/ Why it matters: ✔️ Helps venues detect low-effort reviews ✔️ Enables fairer & more transparent review evaluation ✔️ Opens the door for evidence-based peer review reform We are releasing all data, code, and models to encourage further research: 🔗

7/ Key finding #3: Fine-tuning LLaMA-3 on human-annotated data improves performance, although it still falls short compared to lightweight models. Even GPT-4o could not outperform a straightforward regression model trained on handcrafted features.

6/ Key finding #2: LLMs in both zero-shot and fine-tuned settings show limited alignment with human judgments for most review quality dimensions.

5/ Key finding #1: Simple and interpretable features like review length, semantic alignment, and readability correlate surprisingly well with human ratings.

4/ We measure review quality from three perspectives: 1️⃣ Quantifiable metrics such as length, citations, politeness, and topical alignment 2️⃣ Human expert annotations 3️⃣ LLM-based structured assessments The key question: Which of these aligns best with human judgment?

3/📊 RottenReviews dataset includes: 1⃣ 55k+ reviews from NeurIPS, ICLR, F1000Research, SWJ 2⃣ 9k+ linked reviewer scholarly profiles 3⃣ Thirteen human-annotated review quality dimensions All openly available: 🔗

2/ Peer review is the backbone of science. But how can we tell if a review is high quality? 🤔 There is no standard definition. Evaluations are often subjective. Public datasets with quality labels are scarce. That is why we built RottenReviews 🍅.

1/ 🚀 Excited to share that our paper "RottenReviews: Benchmarking Review Quality with Human and LLM-Based Judgments" has been accepted at #CIKM2025 Applied Research Track! 🎉 We explore what makes a peer review good and why LLMs still have a long way to go in evaluating them.🧵

Two acceptances in two days. What a week it’s been 😌 Details coming up…

🎉 We now present the list of #CIKM2025 accepted Workshops! ✨ 14 incredible workshops will take place on November 14 in Seoul! Check them out at https://t.co/fP3UtbJqaa Stay tuned! More info to come!

Organized by: @ShubhamC526 , @wangxieric , @imsure318 , @sadjadeb , @zhaochun_ren , @debforit , Gareth Jones, Emine Yilmaz, @HamedZamani 📩 Questions? Feel free to reach out! #ProActLLM #ConversationalAI

Bring us your: 🤖 Novel proactive AI behaviors & algorithms 🧠 User modeling & context understanding 🔧 Technical foundations for LLM-powered systems 📊 Evaluation metrics & methodologies 🌍 Real-world applications & demos ⚖️ Responsible AI approaches

🧠💭 Tired of AI that just waits for you to ask? What if your assistant could read your mind, anticipate your next move and offer what you need? ProActLLM is calling for researchers who want to transform conversational AI from reactive question-answerers into proactive partners.