Each time I observe an agent making a call to a standard API, I notice the same pattern: it asks a short question and then receives a very large response.

If you ask an agent to find out whether a meeting slot is free, a typical calendar API won't simply reply with 'yes' or 'no'; instead it returns the complete event object, including the organizer's metadata, time zone strings, recurrence rules, the list of attendees, colour codes, and conferencing links, even though none of these were requested. The agent then has to go through each part of this information one token at a time in order to pick out the two facts it actually needed. When you consider that this has to be done for every tool call during a long agent session, you can see why such sessions become expensive so quickly and lose context well before the task has actually been completed.

The point I'm returning to is that the whole data layer of the internet, every API, each tool response, and each scraped page was created with humans in mind. It is wordy and repetitive, and it includes labels because it's intended for people who skim and then forget. Agents don't skim; they have to pay for every token, whether they need it or not. In my view, it is this mismatch, not the level of model intelligence, that is the real bottleneck preventing a great deal of agentic work from progressing. As I always do, I'd like to make it clear that this is merely my speculation and not something I've built or proven at scale.

I looked to see if anyone had actually taken measurements of this rather than merely complaining about it, and the figures proved out to be larger than I had expected.

Cloudflare’s own Code Mode work is the clearest data point. Their full API surface- over 2,500 endpoints would cost about 1.17 million tokens if exposed as a standard set of tool definitions. More than the context window of most frontier models, just to describe what’s available before the agent has done anything. By collapsing it into two meta-tools, the agent can call one to search for what it needs and another to execute code against it, which provides the same coverage in roughly 1,000 tokens. A 99.9% reduction, and the footprint stays flat no matter how many endpoints the API actually has.

Anthropic has also developed a similar approach from a different perspective. In the article describing code execution using MCP, they reconstructed a workflow which linked Google Drive to Salesforce, rather than loading all the tool schemas in advance and sending each intermediate result back through the model, having the agent write code that would run the entire pipeline within a sandbox and only return the final result. This reduced token usage from 150,000 to 2,000, a decrease of 98.7 per cent for the same task.

In addition, Anthropic's context editing and memory tools, which automatically clear outdated tool output as a long session progresses, reduced token usage by 84% in a 100-turn evaluation and enabled the agent to complete tasks that would have failed outright if it had run out of space.

These aren't just toy benchmarks; they are actual figures obtained independently by two companies dealing with the same issue and arriving at a similar solution.

What I notice is that all three of them adopt the same fundamental approach: rather than supplying the agent with raw data and hoping it correctly filters that data in its mind, let it write some code, run that code in a sandbox environment, and then only bring the actual result, not the raw data from which it was extracted, back into the model's context.

It's not a brand-new concept. The idea stems from a 2024 paper called CodeAct, which found that allowing agents to use rather than tools results in success rates as much as 20% higher across 17 different models. Two years on, that academic finding is now being implemented in the production systems of two of the largest platforms developing agent technology. It's rather uncommon to see something like that occur in this area, an idea moving from a paper to actual production systems so quickly.

My honest guess is that this will become the standard architecture, not just a special technique in which agents are given a small, stable surface to work with rather than having to deal with thousands of tool schemas. Any content that would overload their context is filtered on the spot by code they themselves wrote, against the actual form the raw data takes. Rather than relying on a human to maintain fragile extraction rules that break whenever the upstream API changes its structure, the agent adapts its extraction logic each time, in response to the data as it currently stands.

I don’t believe this is given for free, and I’d like to make that clear rather than pass it over.

As soon as an agent is allowed to write and carry out its own code on live data, you're moving towards what Simon Willison refers to as the "lethal trifecta": an agent that has access to private data, is exposed to content which it did not produce itself, and has a means of sending information out. It is possible to deal with any two of these three elements individually, but all three together form a truly hazardous situation, and merely sandboxing it is necessary yet by itself is not enough; the credentials still have to remain out of the generated code's structural access, not just be gently requested not to leak.

If I take that idea seriously, and I do, the difficult aspect was not 'having the agent write the extraction code', since that merely accounts for the easy 20 per cent; the real difficulty lies in making it safe from the very structure itself, so that the generated code is structurally incapable of ever coming across a raw API key, not just that it is told to act in a certain way.

The approach is correct, and the figures provided by Cloudflare and Anthropic are sufficient to show that this isn't simply a theoretical idea. The only thing I still lack, however, is proof that a general form of this concept- something not dependent on the API of a single company- works when applied to the truly messy and inconsistent tools that most real agent workflows actually deal with, rather than to the well-designed API wrappers created by companies that fully understand what they're doing.

Well, as usual, I'm making a harness to properly test this rather than simply believing the story, since it's so compelling. I'm going to use actual tool calls and real, messy responses, comparing them against the raw baseline to check whether modifying the data this way actually works when tested outside someone else's benchmark.