Rotem Dror
@DrorRotem
Followers
320
Following
543
Media
3
Statuses
199
Asst. Prof. of Natural Language Processing @UofHaifa
Joined November 2018
A strange trend I've noticed at #ACL2025 is that people are hesitant to reach out to papers/"academic products" authors. This is unfortunate for both parties! A simple email can save a lot of time to the sender, but is also one of my favorite kind of email as the receiver!
0
1
33
Tomorrow morning (9AM๐
) I'll be giving a keynote talk at LAW (linguistic annotation workshop) at #ACL2025 on annotations in the era of LLMs - see you there!!๐๐
0
3
17
The Alternative Annotator Test (alt-test) is a new statistical procedure proposed in our ACL 2025 paper! ๐ฆ๐น๐ฆ๐น @DrorRotem @roireichart The goal? To help justify using LLMs over humans. If the LLM passes the test, its annotations can be trusted ๐ https://t.co/Y2l8Eyv2mY
arxiv.org
The "LLM-as-an-annotator" and "LLM-as-a-judge" paradigms employ Large Language Models (LLMs) as annotators, judges, and evaluators in tasks traditionally performed by humans. LLM annotations are...
1
1
11
Everyone uses LLMs to annotate data or evaluate models in their research. But how can we convince others (readers, collaborators, reviewers!!!) that LLMs are reliable? ๐ค Hereโs a simple (and low-effort) solution: show the LLM is a *comparable alternative annotator* โ
3
21
69
๐ฅืืกืงืืจืืช ืืืฉืืืืช ืืืจืื ื-X๐ฅ ๐งต ืืืืืจ ืืืืื ืฉื ืืืืง: 25.06.25 The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs ืืืืจ ๐ฎ๐ฑ ืชืคื ืืช ืืขื ืืื ืช ืืชืจืืฉืช ืืชืงืืคื ืืืืจืื ื ืืขืืื ืฉื ืืขืจืืช ืืืฆืืขื ืืืืืื. ืื ืื ื ืืืจ ืื ืฉืืืืื ืจืง
3
7
24
Preferences drive modern LLM research and development: from model alignment to evaluation. But how well do we understand them? Excited to share our new preprint: Multi-domain Explainability of Preferences https://t.co/XuoN5Mz246
@roireichart @LiatEinDor ๐งต๐ 1/11
2
19
35
This is your chance to uncover revolutionary research, forge invaluable connections, and become part of the AI innovation wave. Conference dates:ย May 25-27, 2025 ๐ทย Information and registration:
lnkd.in
This link will take you to a page thatโs not on LinkedIn
0
0
0
Keynote speakers: . Hod Lipson, Mechanical Engineering, Columbia University, USA Prof. Mor Naaman, Information, Cornell Tech, USA Prof. Tanya Berger-Wolf, Computer Science & Engineering, Ohio State University, USA
1
0
0
Explore AI's transformative power across diverse fields, including robotics, biomedicine, environmental science, social sciences, and the humanities.
1
0
0
Witness cutting-edge research by the University of Haifa's brightest minds, including pioneering work by our researchers and graduate students. Be inspired by world-renowned keynote speakers and gain practical knowledge from leading industry talks.
1
0
0
Join us at the University of Haifa for the HiAI Conference, a dynamic event at the forefront of Artificial Intelligence. Immerse yourself in groundbreaking discoveries, gain exclusive insights into the latest advancements, and navigate the future of AI with leading experts.
1
0
1
In our new preprint, @roireichart ,@DrorRotem and I propose a new statistical procedure: The Alternative Annotator Test (alt-test) The goal? To help researchers justify using LLMs over humansโif the LLM passes, its annotations can be confidently trusted๐ https://t.co/Y2l8EyvAcw
arxiv.org
The "LLM-as-an-annotator" and "LLM-as-a-judge" paradigms employ Large Language Models (LLMs) as annotators, judges, and evaluators in tasks traditionally performed by humans. LLM annotations are...
1
3
13
Check out our new pre-print on statistically sound methodology to verify the quality of LLM-as-a-judge annotation. @NitCal @DrorRotem
In our new preprint, @roireichart ,@DrorRotem and I propose a new statistical procedure: The Alternative Annotator Test (alt-test) The goal? To help researchers justify using LLMs over humansโif the LLM passes, its annotations can be confidently trusted๐ https://t.co/Y2l8EyvAcw
0
2
8
A few months ago I told you that I'm working on something awesome and I need some datasets...well here is the first output of that effort and I sincerely think it's amazing ๐คฉ check it out! @NitCal @roireichart #llm-as-a-judge, #nlpevaluation
Do you use LLM-as-a-judge or LLM annotations in your research? Thereโs a growing trend of replacing human annotators with LLMs in researchโthey're fast, cheap, and require less effort. But can we trust them?๐ค Well, we need a rigorous procedure to answer this. ๐จNew preprint๐
0
7
23
Do you think LLMs could win a Nobel Prize one day? ๐ค Can NLP predict heroin addiction outcomes, uncover suicide risks, or simulate brain activity? ๐ง @lotemi_peled, @roireichart, and I wrote a blog post about NLP for Human-Centric Sciences ๐ค๐ฉโ๐ฌ ๐๐๐ https://t.co/gMCVN5iYy0
0
8
17
For a new and exciting research about data we need your help! ๐ค๐๐ป If your dataset meets these criteria, we would love to cite your research and analyze the data - exciting results are guaranteed:) #DataScience #NLP #vision @DMLRJournal
I am looking for NLP/CV datasets that: 1. include annotator IDs 2. have multiple annotators per item 3. feature interesting tasks (including generation) Any suggestions?
2
3
11
I am looking for NLP/CV datasets that: 1. include annotator IDs 2. have multiple annotators per item 3. feature interesting tasks (including generation) Any suggestions?
5
3
11
Keynote speech is supposed to be technically sound and enlightening. For students who feel discouraged / unwelcome by any speech - simply ignore, especially when you actually know so much more about whatโs happening than the speakers.
3
7
94
Excited to have presented our TACL paper about the sensitivity of LLMs to prompt paraphrases at #ACL2024! ๐โจ @guymkaplan, @malk_dan, @DrorRotem, @HyadataLab, @GabiStanovsky
https://t.co/oLM47z6BkT
6
5
65
Submission deadline extended- August 23rd! Looking forward to reading your works.
Reminder - *Call for Papers* for The NLP4Science workshop, to be held in EMNLP 2023 in Miami, Florida ๐ช Looking forward to your submissions utilizing NLP for exploration of the human mind (in fields like psychology, psychiatry, neuroscience, and economic behavior) as well as
0
2
1