introducing qqWen: our fully open-sourced project (code+weights+data+detailed technical report) for full-stack finetuning (pretrain+SFT+RL) a series of models (1.5b, 3b, 7b, 14b & 32b) for a niche financial programming language called Q All details below!

sticking w bf16, wouldn’t wanna make things too easy

Met a founder in SF who's crushing it $1 million MRR after just 3 months But he had a nagging problem "Our customer's data just feels so vulnerable. And the government doesn't even care to put serious guidelines in place that tell us how to store and manage our data." He

everyone thinks their own numbering system is base 10

me when i come across an exceptionally well-crafted personal website

We're scaling our Open-Source Environments Program As part of this, we're committing hundreds of thousands of $ in bounties and looking for partners who want to join our mission to accelerate open superintelligence Join us in building the global hub for environments and evals

app idea: fine tune a vision llm on pictures of car damage and the fair repair price get a quick fair estimate maybe just sells leads to auto body shops

my per token credit tweak for grpo might actually be working 🥲

the related works section went from being my least favorite part to write to one of my favorites

repo: https://t.co/tsSFY8mqYo link to the old blog with more details about pretraining:

update 1 year later: i've now added a GRPO trainer to hill climb telling the scariest stories! 👻 after grpo, my 1.5B model (fully trained from scratch pretrain, midtrain, rl) went from losing every time to beating gpt4.1-nano at scary stories in ~25% of head-to-head matchups.

next step is to try smaller MoE models and to build more Q environments

* the pretraining data is all idiomatic Q - and of course our RL env just enforces the model writes correct Q - the pythonic bias came from SFT on Q that was translated from python unfortunately its still a little pythonic - I think its just the nature of the leet code problems

Update to this - we have trained a 72B model the most exciting part is the pretrain accuracy is high enough that we can skip SFT and just use RL - meaning less pythonic Q! * interestingly not much performance gain over 32B (actually worse pass@1) - but with higher pass@N's it

GitHub:

releasing my grpo v2 repo: nano-grpo-reasoning-gym two big changes (1) this one entirely implements the grpo training stack from just pytorch/very simple python code - but is now extended to use vLLM, the liger kernel and other optimizations that make it much quicker to train

okay embarrassingly late to the party but uv is amazing

sigh

does backtracking help the model think or is it just more likely in human text when someone backtracks they get the correct answer is that the same thing

“if this sigmoid trend continues well reach 2 by mid 2026!”

getting a citi bike membership has turned the worst part of my week into one of best parts