Local AI inference keeps punching past its weight class

Three separate threads today all point at the same thing: the gap between what you can run locally and what requires a cloud API is shrinking fast. MiniMax H3 ships with open weights, 2K video, and native audio, with a memory footprint reduced 66% down to 42.5 GB via quantization. Someone posted a Show HN running an 80B Qwen model in 4.3 GB of RAM on a Mac. Another Show HN lets you fine-tune an 8B model on a 4 GB laptop GPU with a single CLI command.

The pattern here: every few weeks, a capability that felt like it required serious cloud infrastructure becomes something you can run on consumer hardware. The Qwen thread got pushback about decode speeds and SSD wear, which are real problems, but the counterargument in the comments was equally real: this is how progress happens, people keep iterating on it.

The nuance worth noting is that raw capability and practical usability are still far apart. Prefill bottlenecks, slow decode times, and drive wear are genuine issues. But the direction of travel is clear, and founders building on hosted APIs should be thinking about what happens when their moat is 'we run this for you' and users can run it themselves.