AI system prompt bloat and token costs are becoming a real tax
A thread analyzing Claude's system prompts noted they've grown from around 300 words in early versions to over 3,000 words in the latest Opus release. One commenter asked the obvious question: why not bake this into the model weights rather than paying for these tokens on every API call? The Anthropic watermarking thread added another dimension, with commenters pointing out that if Anthropic mandates watermarks in generated text via terms of service, you effectively don't fully own your output.
The Qwen 3.8/27B thread connected directly to this: the 27B model's tendency to overthink simple tasks creates a 7-8x speed penalty compared to the smaller MoE model. For agent workloads where every tool call costs tokens and latency, this isn't a minor annoyance but an operating cost multiplier. One commenter put it plainly: 'For agents, token efficiency is an operating cost. I'd rather have a terse model that escalates hard cases than one that overthinks every tool call.'
The credit resale economy thread added another layer: a whole gray market exists for cheap API tokens, with services like Yunwu.ai offering deeply discounted access, raising obvious questions about where those tokens come from and who's reading your traces.
So what?
If you're building on top of frontier models, the cost structure is less stable than it looks. System prompt bloat, reasoning overhead, and watermarking constraints are all variables that model providers control and can change. Architects building agent pipelines should pressure-test their token budgets against worst-case verbosity, not average-case, and should have a clear answer for what happens when a model provider changes its terms around output ownership.
Read these
Claude: System Prompts
Qwen 3.8 27B is excellent, but it defaults to overthinking things
Anthropic's 'watermark' text adulteration in Claude is a perversion of writing
The AI Credit Resale Economy