https://t.co/x6AnS2cnTm

🚨 Before Sam puts personalized ads in your AI chats… Take our 5 min survey & discover what LLMs actually know about you! 🤖💡 Your responses will help build better AI privacy safeguards.

The next EuroLLM model is out 🎉 We support all the 🇪🇺 EU languages (+ more), but now in a 9B size (base and instruct). We are not done yet; stay tuned for more 👀

These papers appeared the same day on arxiv and present nearly the same method, wild! Cache & Distil: Optimising API Calls to LLMs https://t.co/8RbuMgTjlL Cache me if you Can: an Online Cost-aware Teacher-Student Framework to Reduce the Calls to LLMs https://t.co/NqWo3jKV3I

We release the code to encourage more work on optimising LLM API calls https://t.co/oYMddc8EgS

This problem, deciding whether to call the LLM, shares some similarities with Active Learning. We benchmark and analyse classic Active Learning criteria, finding that some of them can be useful to save LLM calls.

In this work, we introduce the 'neural caching' problem: given a request from a user, we obtain the answer using a student model. Then, a policy decides whether to call the LLM to correct the annotation under budget-constrained conditions.

Using GPT-4 but the calls are expensive? Distilling your past queries into a student model may help. Introducing: 'Cache & Distil: Optimising API Calls to Large Language Models'