⏲️ 🎓 Only a few days left to apply to do a PhD in Natural Language Processing with @atanasovapepa and me @CopeNLU @AiCentreDK @DIKU_Institut #NLProc #ML #XAI #phdpositions #academicjobs

🛣️ Our method can be used to study epistemic diversity for any arbitrary topics, downstream tasks, and real-world use cases with open-ended LLM outputs. This allows researchers to answer research questions about which, whose, and how much knowledge LLMs are representing. 9/9

📏 To measure diversity we use a statistically grounded measure commonly used to measure species diversity in ecology, in order to fairly compare the relative diversity of models in different settings. 8/9

🪛 Approach: we propose a new methodology which includes sampling plain text LLM outputs with 200 prompt variations from real chats across 155 topics, decomposing into individual claims, and clustering those claims based on entailment. 7/9

🌍 There are gaps in country specific knowledge. When matching claims to English & local language Wikipedia, no local language is significantly more represented than English, and English language knowledge is significantly more represented for 5 out of 8 countries. 6/9

🏗️ Model size has an unintuitive negative impact on diversity; smaller models tend to be more diverse 🔎 RAG positively impacts diversity, indicating usefulness in making LLM outputs more diverse. However, RAG gains are not equal across topics about different countries. 5/9

📈 Knowledge in LLMs across 3 of 4 model families has *expanded* since 2023 ✅ ; however, their absolute diversity is quite low compared to a very modest traditional search baseline 👎 4/9

👍 To assess this risk, we set out to measure to what extent LLMs are homogenous in terms of the *real-world claims* they generate. We perform a large study across 27 LLMs, 2 generation settings, with different model versions and sizes. In a nutshell, our findings are: 3/9

🤔 Many use LLMs despite their outputs not being very diverse. What does this mean for the future of knowledge? Many speculate that overreliance on LLMs will lead to knowledge collapse, where the diversity of human knowledge is narrowed by a reliance on homogenous LLMs. 2/9

Which, whose, and how much knowledge do LLMs represent? We're excited to share our preprint answering these questions. 📄Paper: https://t.co/amPblCS1zr 💻Code: https://t.co/6vef7LxJec @dustinbwright @_srishtiyadav @maria_antoniak @PeterEbertChri1 @IAugenstein @AiCentreDK 1/9

Available #NLProc PhD positions: - Explainable NLU, start in Spring 2026 https://t.co/8ZnV6B4U1y - Trustworthy NLP, main supervisor: @atanasovapepa, start in Spring 2026 https://t.co/9xiEZByM56 - Open-topic: express interest via ELLIS, start in Autumn 2026

Looking for PhD opportunites in #NLProc #XAI? We @copenlu @AiCentreDK @DIKU_Institut @atanasovapepa are hiring for a start in Spring or Autumn 2026. 📆 Application deadline: 31 October 2025 ℹ️ Details: https://t.co/rxkZVEP6fs 👀 Reasons to apply:

Excited to see @ymwangv first paper accepted at EMNLP 2025! 🥳 Our framework SR-NLE addresses a critical trustworthiness issue -- making LLM explanations reflect their actual reasoning. #EMNLP2025 #XAI #NLP #LLM

- Fully funded PhD fellowship on Explainable NLU: apply by 31 October 2025, start in Spring 2026: https://t.co/zCcfJSus5W - Open-topic PhD positions: express your interest through ELLIS by 31 October 2025, start in Autumn 2026: https://t.co/f63KBIZWY3 #NLProc #XAI

📣 Looking for PhD opportunities in Natural Language Processing? Our group has several openings for a start in Spring or Autumn 2026 -- apply by 31 October: https://t.co/rxkZVEP6fs #NLProc #XAI @atanasovapepa @CopeNLU @AiCentreDK @DIKU_Institut

Happy to share that our work on multi-modal framing analysis of news articles was accepted to #EMNLP2025! Understanding news is especially important in today's environment and it's imperative to take a holistic look at it. Looking forward to presenting it in Suzhou!

🚀 Excited to share our new preprint: BiasGym: Fantastic LLM Biases and How to Find (and Remove) Them 📄 Read the paper: https://t.co/S0O5jaZbgQ

🙋 How do Large Language Models internally process cultural knowledge? 🌐 Happy to share our new preprint "Entangled in Representations: Mechanistic Investigation of Cultural Biases in Large Language Models" 📃 Paper: https://t.co/Gt14mjMiQg

Thanks for everyone who came to our #acl2025nlp presentations (even the informal ones). Hope to connect again at #emnlp2025! #NLProc

Meet @IAugenstein, Prof at @uni_copenhagen 🇩🇰, ELLIS Scholar, and member of @elliscph & ELLIS Program #NLProc. Her advice to young women in science: “Success isn't just about tech skills. It's also about managing your career.” #WomenInELLIS