Small open models are closing the gap on frontier coding
Memdoor showed a team of five Qwen3.8-27B models beating Fable 5 on LiveCodeBench Hard. Commenters wanted it re-run with qwen 3.8-flash-next, and one noted that Bonsai 2 quantisations get Qwen 27B to about 50 tokens per second on a reasonably priced gaming machine. In the Ask HN on replacing Claude or GPT with a local model, one user said Qwen 3.6 27B dense feels roughly like Claude Haiku 4.5 and sometimes Sonnet. Others reported DeepSeek V4 Flash at 160 tok/s on two RTX Pro 6000 cards, and the slower tok/s of Gemma 4 on an M4 was the main complaint.
The pattern here: quality is no longer the blocker for local models, throughput and tooling are. Ensembling several cheap models is a new lever, and people are already pricing it against API bills. Nobody claims a clean win yet, but the conversation has moved from "can it work" to "which setup".
The counterpoint is that benchmark wins like LiveCodeBench Hard are narrow, and several commenters said real daily coding still favors the hosted models. The 'enterprise tooling for picking and running a model' gap is also real.
So what?
If your product's margin depends on frontier API pricing, you now have a credible fallback to price against. Build your stack so the model is swappable, and test open-weight models on your own tasks instead of trusting leaderboards.