Amelia Dai
@ameliadai_
Followers
10
Following
10
Media
0
Statuses
3
Our recent research on LLM news prediction is featured on NYU CDS blog. @ameliadai_ @agentic_ai_lab
New research by CDS MS student Amelia (Hui) Dai, PhD student Ryan Teehan (@rteehas), and Asst. Prof. Mengye Ren (@mengyer) shows that models’ accuracy on current events drops 20% over time—even when given the source articles. Presented at #NeurIPS2024. https://t.co/qAkHtzKLQu
0
4
11
many thanks to my wonderful supervisors @mengyer and @rteehas! and check out @agentic_ai_lab for more info :)
0
0
0
🔍Excited to share our new work on LLM continuous evaluation benchmark for news event forecasting! 📉See how it uncovers model performance degradation & why continuous model updates are critical! - Website & Dataset: https://t.co/LdfmK6PsBP - Paper:
agenticlearning.ai
Daily Oracle: a continuous evaluation benchmark using automatically generated QA pairs from daily news to assess how the future prediction capabilities of LLMs evolve over time
Will LLMs ever get out-dated? Can LLMs predict the future? Today, we release Daily Oracle, a daily news QA benchmark testing LLM’s temporal generalization and forecasting capability. 🧵
1
0
3