Nobody trusts coding benchmarks anymore

MaxxBench is pitched as a skeptical reply to a DeepSWE score ("Nice DeepSWE score. But is it true?"), and the thread treats it half as a joke and half as a real complaint. One commenter says real programming is full of these small, annoying changes. Another says there is always a huge gap between benchmarks and real-world ability, and calls both deepseek 4.1 flash and haiku 5.5 "fast garbage producers" for their use case.

The same distrust shows up in the local-model threads. People test on their own tasks, like transposing a bit in a scalar function, and in the harness thread, where one person's own nightly runs contradict how the tools are marketed.

The pattern: published scores no longer persuade practitioners, and personal evals are becoming the default way to pick a model.