Local LLMs for coding: real usage, real limits
A thread asking whether anyone has replaced Claude or GPT with a local model for daily coding produced detailed, specific answers rather than the usual hype. People running DeepSeek V4 Flash on dual RTX Pro 6000 Blackwell cards get 160 tok/s but note it's a reasoning model with different latency characteristics. Someone on an M4 Mac found Gemma 4 noticeably slower than cloud. The general consensus: local models are close but not there yet for interactive coding at cloud speeds.
The honest summary from the thread: for batch workloads and overnight tasks, local models work. For interactive coding with tight latency expectations, cloud still wins on speed. The gap is closing but not closed.
The Jev model kept coming up across multiple threads today as a reference point for speed, which suggests fast inference is becoming a distinct product category rather than just a feature.
So what?
If you're building a coding assistant or dev tool, the local model option is real enough that you need a position on it. Users who care about privacy or cost will ask. But the current performance gap means you can still differentiate on latency if you run in the cloud and optimize for it.