Agentic AI Context Management Is the Real Engineering Problem

Two threads converged on the same bottleneck in production AI systems. A paper on 'Agentic Context Management' argued that memory and cost in multi-step AI agents are architecture problems, not prompt engineering problems. Separately, a post on RAG prompted the observation that most 'document question-answering' systems are just search with extra steps, and the hard part is not the embeddings, it is getting the right context into the model at the right time.

The through-line: as people move from single-shot LLM calls to agents that run for minutes or hours, the context window becomes the scarce resource that determines both quality and cost. What goes in, when, and at what granularity is now a real system design question. The commenter who summarized it as 'most LLM problems are context problems' got a lot of agreement.

The counterpoint worth tracking: several people pushed back on the framing that RAG is complex. For simple document retrieval, it really is just embeddings plus a vector search. The complexity explodes when you have agents that need to remember state across many steps, call tools, and recover from errors. These are different problems, and conflating them leads to over-engineered solutions for simple cases.