AI inference costs collapsing, open source closing the gap

The GLM 5.2 thread is the clearest statement yet of something builders have been watching for months: open source model quality is catching up to frontier models, and inference costs are falling fast enough that the economics of paying for closed-model APIs look increasingly hard to justify. One commenter put it plainly, saying this is the least understood upcoming shift in AI economics, then noted it gets discussed on HN every single day. The small AI models thread reinforced this from the other direction, showing real deployment of smaller, offline models in low-connectivity environments.

The pattern: quality is good enough, cost is falling, and the remaining argument for expensive closed models is shrinking to very specific capability gaps. The Ternlight thread, a 7MB embedding model running in the browser via WASM, shows how far the compression of AI capability has come. Someone built a working sentence encoder that runs on-device in under 30 seconds, distilled from MiniLM with ternary quantization.

The counterpoint in the threads is speed. Commenters noted that cheaper inference has not yet translated to faster speeds at lower tiers, and that the gap between what you can run locally and what you can run in the cloud on latency-sensitive workloads is still real. But that gap is narrowing too.