How many times should I say it?. The future will be smaller, highly specialized models that excel in one domain. Specially for services like ChatGPT.

Join millions who have switched to Grok.

Paper: Code:

It can match GPT-5-medium’s performance at 27% less cost, and achieved 90% of its performance at 63% less cost. Additionally, always gave the best accuracy for any given cost and the lowest cost for any given accuracy, outperforming every single model tested. 5/5.

For a new query during inference, it embeds, finds nearest clusters, and routes to the model with the best pre-computed score. It suggests that the future of LLMs may lie in orchestrating multiple specialized cost-effective models instead of one big general purpose model. 4/5.

To train the system, they build clusters of queries, then evaluate each model’s accuracy (performance) and cost (efficiency) within each cluster. From this, they compute normalized "performance–efficiency scores" per model–cluster pair. 3/5.

Instead of sending queries to just one of two models, Avengers-Pro embeds queries, clusters them by semantic similarity, and then routes each query to the model with the best performance-efficiency score, balancing accuracy and cost through a trainable trade-off parameter. 2/5.

📝 Beyond GPT-5: Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routing. GPT-5 used test-time routing to balance efficiency and performance. This paper extends that with Avengers-Pro, which combine multiple LLMs to optimize the same trade-off. 1/5

Paper:

Interestingly, optimal positions vary, smaller models prefer ssp/esp, while very large models (like, LLaMA3-70B) often benefit from demos closer to the query (sum = start of user message). The optimal demo placement is task- and model-dependent. 5/5.

Placing demos early in the prompt (ssp = start of system prompt, or esp = end of system prompt) usually gives more accurate and stable results than later positions. Late placement (eum = end of user message) makes models unstable, flipping many answers and hurting accuracy. 4/5.

They kept the demo content the same and only changed where it appeared in the prompt, then tested this across many tasks using models from 1.5B to 72B parameters. They measured accuracy gains, prediction flips, and confirmed the results with statistical significance tests. 3/5.

A demo is an example input–output pair placed in the prompt to show the model how to solve a task. (In-Context Learning) This paper explores that how demo placement alone (independent of content) can change accuracy by up to 50% and flip nearly half of a model’s predictions. 2/5.

📝 Where to show Demos in Your Prompt: A Positional Bias of In-Context Learning. It's already known that the choice of demos (demonstrations) and their order affect how models perform. But, no one had really studied how the placement of demos in the prompt changes results. 1/5

Paper:

ARAG outperforms both Vanilla RAG and Recency-based ranking, with up to 42% NDCG@5 and 35% Hit@5 gains in Clothing, and strong improvements in Electronics (~38%) and Home (~26%). Also, because it produces reasoning traces, it offers more transparent recommendations. 6/6.

They showed that each agent contributes to performance: the User Understanding Agent boosts context relevance, the Context Summary Agent especially helps in style-driven domains, and the full ARAG setup delivers the best overall results. 5/6.

The authors built ARAG as a blackboard-style multi-agent system, where agents share information through structured memory. The process works in four steps: first, a standard RAG retrieves candidate items; then the UUA summarizes preferences while the NLI Agent checks alignment;.

They introduced four LLM (GPT-3.5) agents. A User Understanding Agent (UUA): summarizes preferences, an NLI Agent: checks alignment of items, a Context Summary Agent (CSA): condenses relevant evidence, and an Item Ranker Agent (IRA): produces the final personalized list. 3/6.

It extends standard Retrieval-Augmented Generation (RAG) by introducing a multi-agent collaboration mechanism that allows LLMs to better capture user preferences (both long-term and session-level) and generate context-aware, personalized recommendations. 2/6.