7/ Very grateful to amazing co-authors! Jing Huang @ZhengxuanZenWu @davidbau @shi_weiyan.

6/ 💡We have also found representations of harmfulness may vary across different risk categories. Additionally, adversarial finetuning has minimum influence on the internal belief of harmfulness. Read our full paper:

5/ ⚔️ We propose Latent Guard: a safeguard model that uses internal beliefs to detect unsafe inputs or over-refusal cases.

4/ 🔓 We also find jailbreak methods can suppress refusal, but LLMs may still internally know the input is harmful.

3/ 🧪 Causal evidence: We extract a “harmfulness” direction and a “refusal” direction. We design a reply inversion task where steering with these two directions leads to opposite results!.Harmfulness direction: it will make LLMs interpret benign prompts as harmful.Refusal

2/🔗Paper: 💻Code: .📚Project Page: 📊We focus on two token positions t_inst and t_post-inst. Through clustering, we show:. — Harmfulness decides the clustering of instructions at t_inst. — Refusal decides

1/ 🚨New Paper 🚨.LLMs are trained to refuse harmful instructions, but internally, do they see harmfulness and refusal as the same? .⚔️We find causal evidence that 👈”LLMs encode harmfulness and refusal separately” 👉. ✂️LLMs may know a prompt is harmful internally yet still

RT @shi_weiyan: 🤔Long-horizon tasks: How to train LLMs for the marathon?🌀. Submit anything on 🔁"Multi-turn Interactions in LLMs"🔁 to our @N….

RT @dawnsongtweets: 1/ 🔥 AI agents are reaching a breakthrough moment in cybersecurity. In our latest work:. 🔓 CyberGym: AI agents discov….

What types of exemplar CoTs are better for In-Context Learning?.Our #EMNLP paper shows that an LLM usually prefers its own generated CoTs as demonstrations for ICL. 📅I will present this paper in person on Wednesday 4pm at Poster Session E (Jasmine). Come visit our poster!

RT @RubyFreax: ❓ How do you solve grasping problems when your target object is completely out of sight?. 🚀 Excited to share our latest rese….

RT @simon_ycl: ❗Are We Truly Achieving Multilingualism in LLMs or Just Relying on Translation?❗. Need multilingual instruction data and ben….

Our work has been accepted by #EMNLP2024 Findings ! So thankful for my wonderful co-authors !!! All my three projects during my Master's study @UMassAmherst now have happy endings!

RT @simon_ycl: ☕We release the paper, “Diversity and Conquer: Diversity-Centric Data Selection with Iterative Refinement”. 👉In this paper,….

RT @mengyer: 🚨 New Research Alert!.People have found safety training of LLMs can be easily undone through finetuning. How can we ensure saf….

RT @DimaKrotov: In our #NeurIPS2023 paper Energy Transformer we propose a network that unifies three promising ideas in AI: Transformers, E….

ok, this has been studied.

Is it possible to find a metric to locate or explain what training data LLMs generalize from to the test cases? Really interested in how LLMs can magically answer users’ diverse questions. Or it’s simply a result of almost exhaustive training data.

I will have a poster presentation at ICML TEACH workshop (7/29). The paper is an extension of this tweet 😆that interprets ICL as retrieving from associative memory.

RT @Andrew_Akbashev: In your application letter for #PhD / postdoc, NEVER ever say:. "Hi prof"."Hello"."Dear Professor"."Greetings of the d….