The USC NLP faculty is 2x of its size in two years! We're thrilled to have @DaniYogatama and @swabhz joining us this Fall (@jieyuzhao11 starting in Fall'23), along with five other CS new hires! 🙌 🔖 https://t.co/9H3M06SUYq

Thrilled for the Best Paper Award runner-up at #NAACL2025! 🥳 Even when answers are incorrect, people may rely more on LLMs if they use warm and emphatic expressions! We analyze the risks of human over-reliance on LLM expressions of uncertainty: https://t.co/E6289Kx5vV w/

Proud of my student @huihan_li and intern Arnav presenting their #ICLR2025 work on attributing culture-conditioned generation to LLM’s training corpora. Fun time meeting many friends. Ping me if you want to chat about model security, interpretability and human-LM interaction!

Heading to #EMNLP2024, down to chat! Excited to present our work (Wed 10:30am) on systematic data generation in long-tail (low confidence) distribution for more challenging evaluation. 🧵👇 📰: https://t.co/NuTrjE1Znz 💻: https://t.co/Vyi9gyPPGa 🔖: https://t.co/AmmpZJjg2G

Proud moment seeing our CEO & Co-Founder @xiangrenNLP alongside his @nlp_usc students at @aclmeeting. Supporting the next generation of thought leaders in AI is exactly what drives us forward.

Join us at the co-located <AI / ALL> summit on Aug 15, with the social party in the evening! https://t.co/jDqtdpjDr7 co-hosted with @SCB10X_OFFICIAL @SambaNovaAI sponsored by @awscloud participated by folks @AIatMeta @google @CohereForAI @togethercompute

Find us at the posters! Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs w/ @siyuan___wang @YejinChoinka et al Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty w/ @KaitlynZhou, @MaartenSap et al.

Arriving in Bangkok for @aclmeeting! 😃 Will be sharing our recent work on logical scaffolding, model uncertainty expression & multi-hop entailment inference w/ folks @nlp_usc + @KaitlynZhou +friends @allen_ai I'm also helping on the <AI / ALL> summit w/ @SaharaLabsAI 👇👇

Excited to see everyone soon at #acl2024 in Bangkok! I'll be presenting our work, Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty https://t.co/3lROnoBecm Poster session 3 on Aug 12 at 16:00! W/ @MaartenSap @JenaHwang2 @xiangrenNLP

Introducing 𝗟𝗶𝗳𝗲𝗹𝗼𝗻𝗴 𝗜𝗖𝗟 and 𝗧𝗮𝘀𝗸 𝗛𝗮𝘆𝘀𝘁𝗮𝗰𝗸, a new approach for evaluating long-context LMs, featuring ever-changing task streams that controllably fill the context window, and NIAH-style visualization for easy diagnosis. 📜 https://t.co/QW0Az5Fv8A 🧵

Congratulations to the GDM @GoogleDeepMind team on their best paper award at #ICML2024 & Appreciate @afedercooper's shout out to our concurrent paper 🙌 If you are into the topic of recovering model info through just its output logits, check out our paper led by @mattf1n too!

🧐LMs forget upstream knowledge when continuously fine-tuned. When fine-tuned on new data, can we forecast what upstream examples will be forgotten? 🥳Excited to share our #ICML Spotlight paper on forecasting example forgetting! 🔗Project page: https://t.co/1az8DPFz3X

New paper 🚨 Looking for a strong, open-sourced entailment-verification model to verify your model generations for consistency? ✅ You can now use the 🤗model https://t.co/5Gau4pA7IE for this! Our FlanT5-xxl finetuned model can predict entailment errors better than GPT3.5 and

Wanna know gpt-3.5-turbo's embed size? We find a way to extract info from LLM APIs and estimate gpt-3.5-turbo’s embed size to be 4096. With the same trick we also develop 25x faster logprob extraction, audits for LLM APIs, and more! 📄 https://t.co/NdYU8ZhuVH Here’s how 1/🧵

Absolutely thrilled to receive this honor. Rarely for a researcher could have their first PhD publication win a Test of Time Award (for 10 years of its cumulative impact). I’m super grateful for the chance to collaborate with Xiao on this fun project — turns out to be a

To detect if your data was used for LLM pretraining, consider using data watermarks: https://t.co/hn2YuueDN8 Detection can be framed as hypothesis testing (statistical guarantees!), if you contributed multiple training documents and watermarked them before public release. 🧵

Arrived at NOLA for #NeurIPS2023🔥 Exciting time to chat about limits/science of LLMs, “slow” reasoning & explainability. Join our posters for a fun disucssion🍻 Ads: USC CS is hiring tenured track AI faculty + USC NLP is looking for strong PhD students. Talk to us!

How good are LMs at inductive reasoning? How are their behaviors similar to/contrasted with those of humans? We study these via iterative hypothesis refinement. We observe that LMs are phenomenal hypothesis proposers, but they also behave as puzzling inductive reasoners: (1/n)

Throwback to when @xiangrenNLP and our lab made our wishlist and dream research directions to discuss in our lab meeting — very helpful in contextualising our work in the age of LLMs!! 🙌🏼 @nlp_usc is such a great place to do research 🫶

We're excited to attend #SocalNLP today! ICYMI, sunny southern California is a fantastic place to do #NLProc, come check out what USC NLP [ https://t.co/Z16jR4liAP] has been working on lately! And did we say we're hiring PhD students this fall? 🌴🏖️☀️

Feeling hard generating challenging evaluation data for LLMs? Check our work👇! Introducing LINK🔗, the first framework for systematically generating data in the long-tail distribution, guided by symbolic rules https://t.co/NuTrjE1Znz w/@nlp_usc @ai2_mosaic 🧵⬇️ #NLProc [1/n]