Inference hardware: squeezing GPUs, wafer-scale racks, and narrow models
GPU-unlock claims to raise FP/BF16 instruction issue rates on Nvidia 3000-series cards by exploiting firmware. One commenter confirmed the results. The caveats are real: the card runs hotter, the best payoff needs extra work on some cards, and many workloads, training in particular, are memory-bandwidth bound anyway. Cerebras announced trillion-parameter model deployments. Commenters did the math on a 59-wafer pipeline: about 20 kW per wafer, roughly 1.18 MW total, and about 4.1 J/token at 1,500 tokens per second. They also asked what failure looks like when 'the model is the rack' and one bad wafer takes it down.
A third data point: a 10-year-old CNN beat frontier LLMs at skin cancer diagnosis. Practitioners said it was an unfair test, a fine-tuned CNN against general models on a tiny dataset, and compared it to a calculator beating an LLM at multiplication.
The pattern: people are hunting for efficiency from every angle, either squeezing hidden performance from old silicon, buying a new architecture, or using a narrower tool. The consistent critique is that headline numbers hide the constraint that actually binds, whether memory bandwidth, failure domains or dataset size.
So what?
Before you chase a hardware speedup, find out whether your workload is compute-bound or memory-bound. For product work, a small specialized model is often the right answer where a general LLM is overkill, and it is cheaper to run. Treat vendor benchmarks as marketing until someone shows the failure case.