Want to watch token use and costs live? Test them out with the newest release of JrDev!.pip install jrdev

Doing some evals of AI models on some instruction routing prompts. Qwen3 8B outperforming R1! Results so far:.Claude Opus 4: 3/3.Gemini 2.5 Pro: 3/3.Claude Sonnet 4: 2/3.Qwen3 8B: 2/3.DeepSeek V3: 2/3.o4 Mini High: 2/3.Gemini 2.5 Flash: 1/3.DeepSeek R1: 1/3.

Be careful when you star an old repo you worked on, someone might be watching😆.

Slowly growing those repo stars ⭐️

What's the most valuable metric of activity on a GitHub repo? Unique cloners, stars, or something else?.

I asked the mysterious Cypher Alpha to "update the readme"

I am actually seeing some decent results when retrying the same task and different tasks. There is a possiblity its a little inconsistent, hence Alpha phase?.

Testing the mysterious new stealth model "Cypher Alpha". Fails to follow the same instructions that are easy for Gemini-2.5-Pro & Flash, Claude, and GPT to follow. This is a little bit more intelligent than Gemini-2.5-Flash-Lite.

Working on rigging up an intent interpreter agent to all the commands in JrDev. Just type and it will figure out what to do. This will make it so you can be lazier. ie copy and paste all the text for a new model and just say "add this model".

Since posting this a few hours ago, it looks like the price has dropped to a super competitive rate. $0.25 per million tokens for input and $1/m for output. I am definitely going to start using this model a lot.

I love the Inception: Mercury model so far. It is crazy fast, reliable, and smart. The downside is the high cost $10 per m tokens, and the small context window of 32k. Both limit the usability for me. Hopefully the next iteration will solve both issues.

Out of all the tests, it looks like OpenCode was the only tool to completely solve the lint errors. It was quite expensive though!.JrDev did a decent job, especially considering its low cost. Claude Code disappointed me with the worst results, and tried to disable lint checks.

Out of curiosity, I tried JrDev + Gemini 2.5 Pro.Solved 24/25 lint errors.Speed: 6m 27s.Cost: $0.47.

I tried this on OpenCode with Claude Sonnet 4 and then again with Gemini 2.5 Pro. Both models solved 25/25 lint errors.Claude Cost: $9.17 (wow thats crazy).Gemini Cost: $3.38 (still very high!).Claude Speed: 6m 0s.Gemini Speed: 8m 0s

Onto OpenCode:.This is a seriously cool tool, and open source too. It just underwent a full rewrite, and the source code is clean and professional. OpenCode uses the same "surgical" approach as Claude. OpenCode will run terminal commands as part of its agentic loop.

Results:.Overall this run produced the better results than Claude Code, took 70% less time, and was 80% cheaper. 24/25 lint errors solved + created 1 new error (trailing whitespace).Cost: $0.24.Speed: 2m 30s

JrDev (Using Claude Sonnet 4):.All analysis and coding is done by Sonnet 4 here, Gemini 2.5 Flash is solely used for one call saying "are you sure you have the context you need". Its a cheap, quick model, and this step increases result quality.

This allows the AI model to do its thing, and cuts out all of the unnecessary tool calling. JrDev and OpenCode give flexibility to use pretty much any provider and model. JrDev has a little more flexibility in that you can use assign models to specific parts of the pipeline.

Next Up JrDev:.This is the developer assistant I have created over the last few months. JrDev does not take the "surgical" approach, rather it tells the AI model to make the changes and output the entire file with the changes incorporated.