Wanna know gpt-3.5-turbo's embed size? We find a way to extract info from LLM APIs and estimate gpt-3.5-turbo’s embed size to be 4096. With the same trick we also develop 25x faster logprob extraction, audits for LLM APIs, and more!.📄 Here’s how 1/🧵

RT @murtazanazir: this was an amazing project. Matt is an absolute joy to work with with his ever extending support and his genius ideas. a….

RT @murtazanazir: excited to finally share this paper. still shocked that this works so well! this was a fun project with matt, @jxmnop, @s….

RT @aryaman2020: @mattf1n @BrihiJ @jxmnop @swabhz @xiangrenNLP Super cool work and a fire team.

The project was led by @murtazanazir, an independent researcher with serious engineering chops. It's his first paper. He's a joy to work with and is applying to PhDs. Hire him!. It's great to finally collab with @jxmnop, and a big thanks to @swabhz and @xiangrenNLP for advising.

Our technical insight is that logprob vectors can be linearly encoded as a much smaller vector. We make prompt stealing both *more accurate* and *cheaper*, by compactly encoding logprob outputs over multiple generation steps, resulting in massive gains over previous SoTA methods.

We noticed that existing methods don't fully use LLM outputs:.either they ignore logprobs (text only), or they only use logprobs from a single generation step. The problem is that next-token logprobs are big--the size of the entire LLM vocabulary *for each generation step*.

@jxmnop @swabhz @xiangrenNLP When interacting with an AI model via an API, the API provider may secretly change your prompt or inject a system message before feeding it to the model. Prompt stealing--also known as LM inversion--tries to reverse engineer the prompt that produced a particular LM output.

I didn't believe when I first saw, but:.We trained a prompt stealing model that gets >3x SoTA accuracy. The secret is representing LLM outputs *correctly*. 🚲 Demo/blog: 📄: 🤖: 🧑‍💻:

Check out the full thread on bsky: Shout out to my collaborators at Meta @uralik1, Dan Bikel, @barlas_berkeley, @ccsasuke, @aasishp, and a big thanks to the authors of the open-source library @fairseq2 which made SFT and DPO training a piece of cake!.

🧵 Adapting your LLM for new tasks is dangerous! A bad training set degrades models by encouraging hallucinations and other misbehavior. Our paper remedies this for RAG training by replacing gold responses with self-generated demonstrations. Check it out:

RT @RobertTLange: Loving the #NeurIPS2024 'Beyond Decoding: Meta-Generation Algorithms for LLMs' workshop ❤️ by @wellecks @mattf1n @hailey….

I didn’t realize when making these diagrams that my Taylor example would be so timely 😂.

RT @wellecks: We're incredibly honored to have an amazing group of panelists: @agarwl_ , @polynoamial , @BeidiChen, @nouhadziri, @j_foerst….

RT @wellecks: Curious about inference-time scaling, the #1 trending topic in LLMs?. Come to our NeurIPS tutorial: Beyond Decoding: Meta-Gen….

RT @wellecks: Excited to give a NeurIPS tutorial on LLM inference strategies, inference-time scaling laws & more with @mattf1n and @haileys….

RT @jaspreetranjit_: Thank you so much @SpecNews1SoCal @jaskang21 for featuring our work on OATH-Frames: Characterizing Online Attitudes to….

RT @xiangrenNLP: Arrived in Philadelphia for the very 1st @COLM_conf! Excited to catch up w/ everyone & happy to chat about faculty/phd pos….

RT @harsh3vedi: I had a fantastic time visiting USC and talking about 🌎AppWorld ( last Friday!! Thank you, @swabhz,….

Just landed in Philly for @COLM_conf where I’ll be presenting my work on extracting secrets from LLM APIs at the Wednesday afternoon poster sesh. Please reach out if you wanna hang and talk about sneaky LLM API hacks, accountability, and the geometry of LLM representations!.

RT @xiangrenNLP: Congratulations to the GDM @GoogleDeepMind team on their best paper award at #ICML2024 & Appreciate @afedercooper's shout….