Local and Edge AI Is Moving Faster Than Expected
Two projects landed on the same day that would have seemed implausible a year ago. Maple-Preview is running a 20B mixture-of-experts model at 120 tokens per second on an iPhone. A separate project trained a 319K-parameter language model entirely on an $8 ESP32-S3 microcontroller, with training taking two days on the device itself.
These are not production-ready products. The Maple-Preview model hallucinates aggressively and relies on search tools to paper over it. The ESP32 model speaks Klingon and is explicitly not a chatbot. But the trajectory is the signal, not the current state.
Commenters are already asking about clusters of ESP32s and wondering about interconnects. The iPhone demo is drawing comparisons to what early GPU compute looked like before it became infrastructure. The HN shorthand was 'edge is edging closer,' which is an accurate if understated read.
So what?
Within 18 to 24 months, meaningful inference on consumer hardware will be normal. If your product's value proposition depends on cloud-only AI, you should be stress-testing that assumption now. Founders building privacy-sensitive applications, offline tools, or anything in markets with poor connectivity have a genuine architectural opportunity to design for local-first AI from the start.