Benchmarks do not measure the job, and everyone knows it

A study headline, "90% of US working hours are spent on tasks not covered well by AI benchmarks" (50121179), drew the sharpest comments of the day. One said most work lacks the tight feedback loop needed for training and agentic workflows, which is why there is no jobpocalypse. Another said about 80% of their software engineering job is fighting disagreements between stakeholders. A third noted that much of what people do only becomes visible once they are not there to do it.

The same doubt shows up in the StarCraft thread (50118688), where Gemini 4 Argon plays the game. Commenters were shocked a model passes 50% against a built-in bot, but asked how contamination from pre-training data is ruled out, and why scores drop to single digits against stronger opponents. Even the naming fight, with people refusing to call anything "Super Intelligence" (50130570) and a Butlerian Jihad link (50127274), is about distrust of the claims made for the technology.

The pattern: capability claims are running ahead of any measure that matches real work. Founders hear the benchmark story, then their customers show them the messy reality of ambiguous goals, politics and unstated context.