🔥 Thrilled to share our new paper accepted at TMLR: "Operationalizing a Threat Model for Red-Teaming LLMs" .🎯 Goes beyond prompt jailbreaks to systematize ALL attack vectors across LLM lifecycle .🛡️ First comprehensive taxonomy based on entry points (training → deployment)

I had the same question. Shouldn't we also be testing how well an LLM can abstain from proving a wrong theorem? Paper Idea: Corrupt existing theorems and see how often SOTA LLMs end up proving it. 💡.

🧵[3/3] Given how heavily the enterprise is investing in filter-based approaches, this leaves me concerned if our current bets on guardrails are actually aligned with the deeper computational realities this paper suggests. Overall, it's an intriguing and somewhat unsettling point.

🧵[2/3] But I'm not fully clear how this intuition translates into the strong computational claim of fundamental impossibility. It feels like there's an implicit analogy to complexity theory here, but the connection seems subtle and not fully spelled out. Why should.

🧵[1/3] Came across this provocative paper arguing for the impossibility of reliably aligning LLMs via external filters and guardrails. The core idea seems to be: there's a fundamental "efficiency gap" between increasingly sophisticated models and simpler

🔍🚨 A quick post. Recent findings suggest quantized models are surprisingly more prone to jailbreak and fault injection attacks. Based on our earlier threat modeling framework for LLM red-teaming, I would characterize it as a novel side-channel vulnerability in LLMs:.

💡 Ready to operationalize these insights? We've got you covered: .📜 ArXiv: .🎧 Deep dive podcast: 💻 Awesome resource hub: .✨ Full systematization of knowledge for H3LLM applications [🧵3/3].

🚀 Key insights that'll change how you think about LLM security: .🔍 Side channels & model inversion attacks are vastly underexplored vs prompt jailbreaks .🤝 Manual + automated red-teaming = superior vulnerability discovery .🎯 Domain-specific risk taxonomies >.

RT @_onionesque: That watermarking should interfere with alignment in LLMs should be fairly obvious. Here we present experiments studying t….

RT @upperwal: Let’s not miss out on any Indian language!.This is a great opportunity to have your language represented in one of the larges….

Watermarking breaks AI alignment 🚨.Solution: generate multiple responses, pick the best one 🎯."Watermarking Degrades Alignment in Language Models" 📄 #AIResearch #AISafety.

RT @hhsun1: Realistic adversarial testing of Computer-Use Agents (CUAs) to identify their vulnerabilities and make them safer and more secu….

RT @_AndrewZhao: if submitting to @NeurIPSConf, DONT forget to add this at the END. Defend against AI reviewers & lost in the middle:. \tex….

Shout-out to Hinton for saying it out loud.

When I was in undergrad a lot of people did a lot of competitive programming. Leetcode was non-existent back then but there was a certain thrill associated with -in the words of Donald Knuth "eke out the last bit of performance" from a machine. I wonder, how will the culture of.

This is an interesting take.

RT @davidstutz92: Writing good reviews and rebuttals is a key skill in academic research. Unfortunately, this skill is rarely properly taug….

Just played around with this - it's almost too easy with a few tricks. Wondering if the examples gathered through this method could be termed genuine instances of persuasion rather than just manipulating the tradeoff between helpfulness and harmlessness in an LLM via a cleverly.

This looks super cool. Want to level up your game? Our paper is the ultimate resource to get you up to speed with LLM attack strategies. 📚🔥.Check it out here: #AI #MachineLearning #LLM #Security.