Excited to share that our paper has been accepted to ACL 2024 (Findings)! Big thank you to my collaborators @vlachos_nlp and @michael_sejr!. Can LLMs help you when writing scientific papers?. We introduce automated focused feedback generation for scientific writing assistance. 🧵

RT @matt_tehenan: Just released a new paper !. We show that LLMs encode the relative position of objects in a 3D subspace in their weights.….

RT @nedjmaou: Check out our new preprint on analysing the research framings of #NLProc papers (w/ a focus on automated fact-checking, hate….

RT @michael_sejr: Understanding the direction a field moves is crucial for anticipating problems and spotting gaps. But it's hard work - re….

We applied our system to recent AFC papers and compared trends to past work 📊. 🔍 Vague framings are on the rise .🧪 More work framed as Scientific Curiosity (exploring models or data without deployment).🙋‍♂️ Growing focus on assisting fact-checkers while full automation declines

How well can we automatically classify the purpose of NLP papers? 📄. Our pipeline achieves: .💡 Significant gains over a baseline LLM via ranked framing explanations .📊 High accuracy in extracting epistemic elements .🎯 86% precision in detecting vague research framings.

To test generalization beyond automated fact-checking (for which we had prior annotations), we annotated 49 highly cited hate speech papers with epistemic elements + research framings. 🧵 Result? ~65% of analyzed papers had vague framings—often omitting users or downstream use

Manual analysis of research framing is insightful, but slow. To scale it up, we built an automated 3-stage system: .1️⃣ Extracts epistemic elements .2️⃣ Links them via semi-automatically inferred logical rules .3️⃣ Refines framing predictions using an LLM with context + examples

To support cross-domain analysis, we define a general schema of research framings. Each framing captures how epistemic elements are connected (or not), which helps distinguish clear vs. vague framings across NLP domains. Examples from hate speech & fact-checking 👇

NLP papers vary in how clearly they explain an artefact’s real-world purpose. We capture this via epistemic elements: .👤 Who's involved .🛠️ What it does & how it's used .🎯 Why it matters. Research framings link these. When links are vague or missing, they risk misuse (e.g. 👇)

📄 Paper: .💻 Code and annotated data: (coming soon). Big thank you to my collaborators @vlachos_nlp, @michael_sejr, @nedjmaou!.

What are NLP papers really saying about the purpose and use of their models/datasets? 🤔. Who are they for? What problems do they solve? How are they used?. We built a framework + tool to:.(1) analyze framing trends across papers.(2) help authors reflect on their own framing 🧵

RT @vlachos_nlp: Pleased to announce the next @FEVERworkshop at ACL2025! Regular workshop papers (ARR and direct submissions) due 15th of A….

RT @richarddm1: The golden pacifier has made it back to Cambridge! Big thank you to the #babylm and #connl teams for the award and a great….

RT @NllpWorkshop: 🥁 Drum roll. 🥁. 🏆 The Best Presentation Award goes to . 🎉 . @RamintaIeva for presenting 📄"Comparative Study of Explai….

RT @cambridgenlp: For this week's @cambridgenlp reading group, @pietro_lesci presented his (+@clara__meister, Thomas Hofmann, @vlachos_nlp,….

RT @KamiarMohaddes: 𝘽𝙄𝙂 𝙘𝙤𝙣𝙜𝙧𝙖𝙩𝙨 to our friend Geoff Hinton (alumnus of @Kings_College) who has just jointly won the @NobelPrize in #Physic….

RT @l2m2_workshop: 📣Hey #NLProc! We* are planning to organize a *ACL workshop on memorization in LLM. Goal: provide a central venue to disc….

RT @michael_sejr: Working on retrieval-augmented models with data you don't fully trust? Concerned about what your LLM might tell its users….

Big congrats to @pietro_lesci, @tpimentelms, @vlachos_nlp and the rest of the team for winning the best paper award at ACL! Incredible work!

RT @pietro_lesci: If you’re at #ACL2024, today at 14:00 pass by our poster! @tpimentelms and I will be there to talk about our paper “Causa….