On-device AI models: real utility, real platform limits
Desert Ant Labs shipped local, fast models designed to run on-device, and the HN thread surfaced a recurring frustration: most of the interesting small models are iOS-only, with no Node package or cross-platform path. Developers building for web or server contexts cannot use them. One commenter wanted a model that could process PDFs into a JSON schema, which is a very concrete and common workflow.
The pattern: on-device AI is maturing as a category but the platform fragmentation is a real barrier. Apple Silicon makes on-device inference genuinely fast, but the ecosystem is still Apple-first in a way that excludes large chunks of the developer population.
This connects to the iPhone Duo thread, where a commenter noted that Apple's eSIM-only decision ignores most of the world outside the US. Apple is building for a specific customer and the rest of the world is working around it.
So what?
If you are evaluating on-device AI for your product, do not assume iOS availability means cross-platform availability. The gap between 'runs on Apple Silicon' and 'ships as a usable library' is still significant. The most useful on-device models right now are the ones with clean APIs, not the ones with the best benchmarks.