I am happy to announce that our position paper "You Reap What You Sow: On the Challenges of Bias Evaluation Under Multi-Lingual Settings" has been accepted for presentation at the @BigscienceW #acl2022 ☘️ workshop! 🧵⬇️

RT @AiEleuther: Like Pythia, but 1234 isn't your favorite random seed? We retrained Pythia 9x using different random seeds to explore how s….

RT @pietro_lesci: ✈️ Headed to @iclr_conf — whether you’ll be there in person or tuning in remotely, I’d love to connect!. We’ll be present….

RT @__alucic: 🚨 PhD position alert! 🚨. I'm hiring a fully funded PhD student to work on mechanistic interpretability at @UvA_Amsterdam. If….

💬Panel discussion with Sally Haslanger and Marjolein Lanzing: A philosophical perspective on algorithmic discrimination. Is discrimination the right way to frame the issues of lang tech? Or should we answer deeper rooted questions? And how does tech fit in systems of oppression?

We also presented our own work on testing the validity and reliability of LM bias measures:. 📄Undesirable Biases in NLP: Addressing Challenges of Measurement.

🔑Keynote @ZeerakTalat: On the promise of equitable machine learning technologies. Can we create equitable ML technologies? Can statistical models faithfully express human language? Or are tokenizers "tokenizing" people—creating a Frankenstein monster of lived experiences?

📄A Capabilities Approach to Studying Bias and Harm in Language Technologies. @HellinaNigatu introduced us to the Capabilities Approach and how it can help us better understand the social impact of language technologies—with case studies of failing tech in the Majority World.

📄Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution. @florplaza22 discussed the importance of studying emotional stereotypes in LLMs, and how collaborating with philosophers benefits work on bias evaluation greatly.

🔑Keynote by John Lalor: Should Fairness be a Metric or a Model?. While fairness is often viewed as a metric, using integrated models instead can help with explaining upstream bias, predicting downstream fairness, and capturing intersectional bias.

📄A Decade of Gender Bias in Machine Translation. @Evanmassenhove: how has research on gender bias in MT developed over the years? Important issues, like non-binary gender bias, now get more attention. Yet, fundamental problems (that initially seemed trivial) remain unsolved.

📄MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs. @VeraNeplenbroek presented a multilingual extension of the BBQ bias benchmark to study bias across English, Dutch, Spanish, and Turkish. "Multilingual LLMs are not necessarily multicultural!"

🔑Keynote @dongng: When LLMs meet language variation: Taking stock and looking forward. Non-standard language is often seen as noisy/incorrect data, but this ignores the reality of language. Variation should play a larger role in LLM developments and sociolinguistics can help!

Last week, we organized the workshop "New Perspectives on Bias and Discrimination in Language Technology" @UvA_Amsterdam @AmsterdamNLP. We're looking back at two inspiring days of talks, posters, and discussions—thanks to everyone who participated!.

This is a friendly reminder that there are 7 days left for submitting your extended abstract to this workshop! . (Since the workshop is non-archival, previously published work is welcome too. So consider submitting previous/future work to join the discussion in Amsterdam!).

RT @UvA_Humanities: #CallforPapers for the workshop New Perspectives on Bias and Discrimination in Language Technology, which will discuss….

This workshop is organized by @AmsterdamNLP @UvA_Amsterdam researchers Katrin Schulz, Leendert van Maanen, @wzuidema, Dominik Bachmann, and myself. More information on the workshop can be found on the website, which will be updated regularly.

🌟The goal of this workshop is to bring together researchers from different fields to discuss the state of the art on bias measurement and mitigation in language technology and to explore new avenues of approach.

One of the central issues discussed in the context of the societal impact of language technology is that ML systems can contribute to discrimination. Despite efforts to address these issues, we are far from solving them.

We're super excited to host.@dongng, John Lalor, @ZeerakTalat, and @az_jacobs as invited speakers at this workshop! Submit an extended abstract to join the discussions; either in a 20min talk or a poster session. 📝Deadline Call for Abstracts: 15 Sep, 2024.

Working on #bias & #discrimination in #NLP? Passionate about integrating insights from other disciplines? Want to discuss current limitations of #LLM bias mitigation?.👋Join the workshop New Perspectives on Bias and Discrimination in Language Technology; 4&5 Nov in #Amsterdam!.