Tiny local models versus cloud APIs: the real tradeoffs

Two threads collided on this today. The Cactus Needle 3 thread claimed 8-29MB automation models can match DeepSeek V4 Flash, and got immediately pushed back: the use cases were fictional, the claims felt like the growing pile of dubious tiny-model-beats-big-model announcements that are making it hard to spot real innovation. Separately, the 'has anyone replaced Claude/GPT with a local model for daily coding' thread showed real practitioners comparing results. Qwen 3 27b dense getting compared to Claude Haiku 4.5, DeepSeek V4 Flash running on two RTX Pro 6000 Blackwells at 160 tok/s, people noting that the gap is closing but the tooling around local models is still rough.

The pattern: cloud AI pricing has dropped low enough that the economic case for local models is narrowing, but privacy and latency remain real reasons to run locally. The Cactus Needle thread's pushback on dubious benchmark claims is important signal. The community is getting skeptical of 'tiny model beats everything' announcements precisely because there have been too many of them.

The cache-to-cache LLM communication paper (letting models talk in 'Neuralese' rather than natural language) adds another dimension: the architecture of AI systems is changing underneath founders who are building on top of specific model behaviors.