Open-weight DeepSeek gets a serving stack; US labs' moat gets questioned

DeepInfer, an inference framework from lightseekorg, claims high-throughput, low-latency large-scale serving for DeepSeek V4.1 Flash. The top reaction called it "a nuclear bomb" and "Kami Kaze" for US frontier labs, arguing there is no path to trillion-dollar valuations when open weights are this good, followed by a sarcastic "Ban all open weight models!"

The hardware side is moving in the same direction. Google open-sourced TorchTPU, a native PyTorch backend for TPUs, and a commenter pointed out Amazon is doing the same for Trainium with torch-neuronx. In the "has anyone replaced Claude/GPT locally" thread, one user runs DeepSeek V4 Flash on 2x RTX Pro 6000 Blackwell at 160 tok/s, and another says Qwen 3.6 27B is about Haiku 4.5 level. Others still find the tokens per second too low on an M4.

The counterpoint is audio. Voxtral 3 is closed, and commenters say speech models remain the weak spot for local deployment. The pattern: serving software, not model weights, is where the open-model race is now being won.