Local LLMs: people want to try before they buy hardware
Show HN: Llmspeed lets you feel how fast a local model runs before you spend on a machine. Commenters promptly found holes. One said their M2 Ultra 128GB runs nearly twice as fast as the reported M3 Ultra 512GB figure. Others said the harness and library versions change the numbers, and asked for older cards like the RX580. The DeepSeek Flash thread added the other side of the same problem: one commenter called running it locally "very costly." In the Ask HN about replacing Claude or GPT with a local model, one person runs DeepSeek V4 Flash on 2x RTX Pro 6000 Blackwell at 160 tok/s. Another says Qwen 3.6 27B dense is about Haiku 4.5 level.
The pattern: local coding models are good enough for some people, but buying hardware is still a guess. Throughput depends on the whole stack, and estimates are unreliable. People want benchmarks of real setups, not spec sheets.
The counterpoint is also in the thread. Someone tried Gemma 4 on an M4 and found tokens per second far below the cloud, and enterprise tooling for picking and running models barely exists.
So what?
There is room for honest, reproducible benchmarking and model-picking tools for local inference. If you sell to developers, assume a growing minority will ask whether your product works with a local or open-weight backend.