Cheap Model Training Is Getting Embarrassingly Good

Someone fine-tuned a 4B parameter model to produce query plans 81% faster than Postgres for a total cost of around $1,200 in compute and API fees, renting two H100s from Lambda for about 95 hours. The model was trained using reinforcement learning with a custom reward signal based on actual query execution time. The thread lit up with skepticism about correctness guarantees, which is fair, but nobody disputed the cost or the performance numbers.

This is the pattern that keeps repeating: tasks that looked like they required massive scale are getting solved by tiny models with clever training signals and domain-specific data. The 4B model beating Postgres is not magic, it is just a model that was trained on the right thing with the right feedback loop.

Xiaomi also posted a live post-training dashboard for their Mimo 2.6 model, showing that two-thirds of training data is source code. The transparency is unusual and interesting. The comment about other labs not doing this raised the obvious question: what are they hiding, or what competitive advantage does opacity give them?