Local LLMs are actually good now, tooling is the bottleneck

Two threads today converged on the same finding: local models like Qwen 3 27B are genuinely impressive, but most users never see their real capability because the surrounding tooling is broken. The 'Why your local LLM feels dumber than it is' thread makes the case that the biggest culprit is chat templates. Many GGUF builds drop the template from metadata, and runtimes like Ollama silently fall back to a generic ChatML format that mismatches the model's training, degrading output quality in ways that look like the model being stupid.

Separately, someone ran Qwen 3 27B on a dual Arc Pro B70 setup and had it complete a real reverse-engineering task in 30 minutes, including recovering a key, passing signature checks, and catching a secondary integrity hash. The comments lit up asking about the harness setup, which is telling: the capability is there, but reproducing results requires nontrivial scaffolding that most people don't have.

The 'What Is a Harness?' thread adds context. Harnesses (the wrapper infrastructure that connects LLMs to tools, memory, and tasks) are being described as the next layer of the stack, the way electronics followed electricity. The counterpoint in the comments is healthy skepticism: things marketed as agents are often just deterministic code with an LLM bolted on.