Introducing “Merging Text Transformers from Different Initializations” ! We design a Transformer merging algorithm in order to study the relationship between separate Transformer minima🔍 👩‍💻: https://t.co/ZdsDQRmNLQ 📝: https://t.co/w1ojnvDkVI 🧵 1/6

I'm on the job market and at #neurips2025! Looking for research roles around data for foundation models and would love to chat with folks - resume/site in my bio. I've recently worked @AIatMeta and @databricks and publish papers with my awesome collaborators @jhuclsp!

Tough week! I also got impacted less than 3 months after joining. Ironically, I just landed some new RL infra features the day before. Life moves on. My past work spans RL, PEFT, Quantization, and Multimodal LLMs. If your team is working on these areas, I’d love to connect.

Considering a PhD in NLP/Speech? 🤔 Need guidance with your application materials? @jhuclsp is offering a student-run application mentoring program for prospective applicants from underrepresented backgrounds. 📝 Learn more & apply: https://t.co/9NjuZ02wy1 📅 Deadline: Nov 20

For #WorldHealthDay, a new study by Hopkins researchers including @suchisaria, @DrewPrinster, & more finds that doctors’ diagnostic performance and trust in #AI advice depends on just how exactly the AI assistant explains itself. Learn more:

📢 Want to host MASC 2025? The 12th Mid-Atlantic Student Colloquium is a one day event bringing together students, faculty and researchers from universities/industry in the Mid-Atlantic. Please submit this very short form if you are interested in hosting! Deadline January 6th

🚨 I am on the faculty job market this year 🚨 I will be presenting at #NeurIPS2024 and am happy to chat in-person or digitally! I work on developing AI agents that can collaborate and communicate robustly with us and each other. My work covers 3 key problems👇 1⃣ Multi-agent +

I'm on the job market! Please reach out if you are looking to hire someone to work on - RLHF - Efficiency - MoE/Modular models - Synthetic Data - Test time compute - other phases of pre/post-training. If you are not hiring then I would appreciate a retweet! More details👇

Considering a PhD in NLP/Speech? Need guidance with your application materials? 🤔 The Center for Language and Speech Processing @ JohnsHopkins is offering a student-run mentoring program for prospective applicants! 🙌 📝 Apply here: https://t.co/SUIqSBFXNF 📅 Deadline: Nov 17

🚨 MTMA 2024 Announcement 🚨 @jhuclsp is hosting Machine Translation Marathon in the Americas Jul 29-Aug 2 in Baltimore. What’s MTMA? week-long MT hackathon--a chance to get together with researchers of all levels and work on open-source, collaborative projects.

This work serves a step toward better understanding the loss landscape of this popular model. There is much more to do! For more details, check out our preprint/code above. This project was part of my internship with my wonderful mentor @melbayad at @AIatMeta 😁 🧵 6/6

We also find that our approach to merging attention sublayers achieves the lowest loss barrier compared to several baselines, and our method is grounded in correlation patterns between models' attention features! 🧵 5/6

We test our merging method on the MultiBERTs models and find that we can reduce the loss barrier between these models when considering these permutation equivalences. These minima are less sharp and isolated than we think!💡 🧵 4/6

We consider different approaches to merge more complex portions of the architecture, like Multi-Headed Attention. We propose a simple method to permute both attention heads as well as their individual features. 🧵 3/6

When studying the loss landscape relationships between different models, we should consider their equivalence classes—like the set of possible model feature permutations. Permuting transformer features is non-trivial, and we address this directly with our method: 🧵 2/6

Reminder: We have a fantastic PhD fellowship program for students from HBCUs or Minority Serving Institutions. Deadline is Dec 1. Spread the word!

It is ✨grad app season✨ and the JHU CLSP is providing a student-run application mentoring program for those looking for more guidance with their application materials. The link to apply is here: https://t.co/rzDnSLDulG and applications for our program are due by November 19.

Excited to share that Johns Hopkins researchers are authors on 20 publications set to appear in EMNLP 2023!! 🎉 🎉 Congrats to all, including our esteemed external collaborators! @emnlpmeeting @HopkinsEngineer @JohnsHopkins Here is a thread in no particular order:

Can I trust this AI/ML prediction? A good uncertainty quantification (UQ) method could answer this with a predictive confidence interval. But in practice, 2 key challenges limit real-world UQ. Our #ICML2023 Oral “JAWS-X” tackles these challenges! 🧵w/ @suchisaria @anqi_liu33 1/n

Presenting two posters from @jhuclsp today at #ACL2023!