Local LLM Inference Security Is an Unsolved Problem

A thread on whether LLMs could exploit their own inference engines to take over host machines got serious engagement. The framing in the original piece was that the harness should enforce security. The HN response was direct: that is completely wrong. Security belongs in the isolation layer, meaning a VM or at minimum a container, not in the harness logic.

Several commenters flagged that LMStudio, Ollama, and Darkbloom all run unsandboxed on macOS, partly because proper sandboxing would require Mac App Store distribution, which imposes constraints these tools do not want. The thread on Headlong, a Bash-based agent harness, surfaced the same tension: a commenter noted it has zero data isolation even between legitimate users, let alone adversarial ones.

This is a concrete infrastructure gap. Builders are shipping local inference setups and agent harnesses without treating the model as an untrusted process. The attack surface is real and the tooling to contain it is not standard practice yet.