Tuning an API for AI agents: what the test agents taught us

Retavon's API has about 500 operations and 700 schemas. Its OpenAPI document is several megabytes. An AI assistant that knows only the base URL cannot read that, and until this month nothing in the document told it where to start.

This post is what we changed to make the API readable by an agent, how we tested it with agents that knew nothing else, and what those agents got wrong because of us. It is in two parts, because the work was. The first round, measured on the build of 24 October 2026, made the document readable. The second, measured on 27 October, asked a harder question: is the API ready to be the whole of an MCP server? The numbers are our own measurements on those builds. Where the two rounds measured the same thing, the later number stands.

The limits that matter are the agent's, not the model's

A model's context window is not the limit an agent hits first. The tools it reads through are. For Claude Code, the assistant we tested with first:

• One tool result is capped at about 25,000 tokens, roughly 100,000 characters of JSON.
• Shell output is cut off at about 30,000 characters, and most agents read an API with curl in a shell.
• Text search works by the line, so a document on one line is one enormous match.

We first sized every read against the 100,000 character limit. The agents used curl and grep, so the limit that mattered was 30,000 characters and a line break. In the second round it turned out to be smaller still: the agents showed themselves only the first few thousand characters of any read.

Split the document into reads an agent can take in

The whole document is still published at /openapi.json for code generators. Agents start at GET /, which points to a map. In the first round the map was every tag with its description and every operation as one line of operation id, method, path and summary: 57,000 characters, made to be searched rather than read whole. From there:

• /openapi/tags/{slug}: one area's operations, to choose between them.
• /openapi/operations/{id}: one operation with every schema its request uses, to call it.
• /openapi/schemas/{name}: one schema, for reading a result.

Each read holds what its step needs and names the rest. An operation's read carried its request schemas whole and listed its result schemas by name and address, because the response itself says what to call next. With the cuts below, that took the largest operation read from 140,000 characters to 26,000.

The rest came from not saying things twice:

• Regex beside a format. Our schema library writes a pattern next to every uuid, date and date-time field. The format already says it in one word, and the patterns ran to hundreds of characters on a couple of thousand fields.
• Shared lists inlined everywhere. The list of permissions, screens and reports was repeated in every schema that used it. Each is now a named schema, referenced.
• Update inputs repeating create inputs. The policy update repeated every policy's description, 46,000 characters. Its fields now say "As Policy's field" unless they add something.
• Every error code's meaning on every operation. Each error response now lists its codes and leaves their meanings to the error schema, said once.

The compact document fell from 3.86 million characters to 2.96 million, 23% smaller, with nothing removed that a caller acts on.

Served one entry a line (below), the median operation read was 10,000 characters and the largest 34,000. A test holds each kind of read under a budget, so a schema that grows past what one step needs fails the build.

One entry a line

Our first map was compact JSON: 53,000 characters on one line. Agents told us, in their own words, that grep "gave nothing useful until I parsed it", and that the shell showed only the first part. Every one of the four fell back to writing a script to parse it.

Serving the reads indented by one space puts every operation on its own line. The map grew 7%. In the next round every agent found its operations with grep for a word of the job: deposit, bounce, receiv.

Tag reads had a different problem. Half of each was error responses, which nobody choosing an operation needs. Leaving them to the operation's own read took the largest tag from 57,000 characters to 37,000 before indenting.

List rows, not records

A list that returns whole records outgrows a tool result quickly. Our appointment list returned what each delivery hands over, its photos, sign-off, parts and next actions: 3,900 characters a row, 146,000 for a page of 50. Rows are now a summary of 1,300 characters, and the single read holds the rest. Sales order rows went from 1,600 characters to 1,000, a page of 50 from 80,000 to 49,000.

Three reports defaulted to 500 or 1,000 rows with no way past the limit. One of them was 27,000 characters for 29 orders. They now default to 50 and take an offset, and the limit parameter says that a large page can outgrow what an AI client reads in one result.

An empty answer is not an answer

A key with no roles and no person behind it could call the sales report and get a valid, empty result: zero rows for a week in which the owner's key showed 129 orders. The agent holding it only found out why by running the same report with the owner's key and comparing. A person seeing an empty table wonders whether they can see it; an agent given nothing to compare with takes it as fact.

Every list or report made only of sales and money now refuses that key with a 403 that says why and what to do: issue a key with a role at the stores it should read. The refusal's next points at the operation that issues keys. In the next round, the agent holding that key wrote the owner a correct note naming the three permissions it needed.

Error text is instructions

An agent does what a refusal says, literally. Two examples from one afternoon:

• A wrong number costs a second refusal. A quote of $12,153.94 needed a 25% deposit, $3,038.49, before it could become an order. The agent offered $500. The refusal said $2,538.49 more was needed and to send "a deposit of at least 2538.49". A refused request keeps nothing, so the right figure was the whole $3,038.49. The agent followed the advice and was refused again. A test had pinned the wrong figure. After the fix, the next agent opened its order on the first retry.
• A mentioned shortcut gets taken. Our refusal also said that a read about one customer is answered whatever the key's stores, which is true. The agent called the per-customer read for all 196 customers to rebuild the receivables report it had been refused. The refusal now names only the fix.

Validation messages from the schema library were the same problem in miniature. "Expected string, received number" does not tell an agent that quantities are sent as decimal strings. The message now says so, with an example, and a missing idempotency key is told what the key is for.

Generated documentation drifts quietly

The agents also found two defects no test was looking for, both in the generated document:

• Fields that accept a natural key, such as a store's branch code in place of its id, had their own description replaced by "Accepts the record's id or its branch code." 167 request fields said what they accept but not what they were. An agent recording a mailed check had to guess which store to name.
• 67 decimal fields had lost the sentence saying they are sent as strings, because a field described again in place replaced it.

Each is now a convention test over the whole document. If a description can be lost by a code path, test the output, not the code path.

How we tested: agents that know only the base URL

Each test agent got a base URL, one person's API key and one job a person at a dealer would do: quote a hot tub with delivery and a deposit, find which deliveries this week are not ready, record a bounced check, raise a purchase order. They could use curl and nothing else, and were not allowed to read the source. Each reported every call, the size of every documentation read, every refusal with its text, and every place it guessed.

• Nine walks in two rounds, run with a smaller model, Claude Sonnet.
• Each took one to two and a half minutes and 58,000 to 78,000 tokens.
• Every agent finished its job, or, where its key was not allowed to, said exactly why. The agents reported the next lists on responses as accurate throughout; one found an extra entry it had no use for.

Alongside the agents, a simulation of people working the web app ran 1,861 actions with the books, stock and cash drawers checked after each, and 162 browser specs covered the screens the list changes touched.

Check every finding before fixing it

About half of what the agents reported was not a defect:

• One reported a field that does not exist.
• One read a correctly documented "ready by" date as wrong.
• Two problems were ours. A script measuring read sizes from the same address tripped the API's rate limit, 50 requests a second with bursts of 100, and an agent got a 429. And two agents shared one business, so one raised the deposit percentage while the other was taking a deposit.
• The simulation's five findings were all one stale step in its own script, looking for a button by a name the screen had changed the day before.

Give each test agent its own copy of the business, keep your own scripts off its address, and verify each claim against the API before writing a fix.

Three days later: is it ready to be an MCP server?

An MCP server over an API like this is a thin wrapper that derives every tool from the OpenAPI document. Then what a model pays per tool is what the document says per operation, and what it pays per answer is the response. We measured both, on the build of 27 October: 497 operations, 43 tags, 701 schemas, 4.81 MB served one entry a line.

• A tool definition proper, the description plus input schemas: median 415 tokens, 90th percentile 1,173, largest 4,802 (the policy update and the role editors). Healthy, in the range the guidance recommends.
• The read a wrapper would serve per operation: median 2,567 tokens, floor 1,679. Every one carried the 1,134-token error schema, every error status, and the document header.
• A default page of 20 orders: 6,154 tokens. Twenty appointments: 7,211. A whole order: 1,353 tokens, 82 fields.

The tool definitions were fine. The reads around them were not, and neither were the answers.

The first walk: four jobs

A Sonnet agent, four jobs: who owes money, record a check on account, tomorrow's schedule at a store, a counter sale for cash. It finished all four in 21 calls and read 275 KB to use about 25 KB. Its own list of what cost most, in order: the 60 KB map read twice; a 41 KB schema fetched to learn the field names of a 2.6 KB answer; operation reads of 15 to 20 KB "mostly repeated text"; and building a UTC window by hand for "tomorrow" from the business clock and a guess at daylight saving.

What we changed:

1. An operation's read is what calling it needs and no more: its request schemas, the names of its result's fields, and the addresses of the rest. Errors are one line; the error schema is read once per session. Median read 1,023 tokens, from 2,567.
2. Every read of a record or a page takes fields=a,b and answers with those fields alone. A page keeps its cursor and its totals. An unknown name is refused, naming the fields there are. Twenty orders cut to four fields: 480 tokens, from 6,154. Five orders to two fields: 65 tokens.
3. Two kinds of schema noise removed: JavaScript's safe-integer bounds on 403 integers, and nullable ids written as a two-way choice that repeated "Unique identifier (UUID v7...)" under the field's own description, 766 times. The document fell to 4.62 MB while gaining a parameter on 192 reads.
4. Writes that move money without an idempotency key say why a repeat is safe, so a wrapper can mark the tool idempotent honestly.
5. The appointment list takes a date at the appointment's own location, so "tomorrow" is one parameter, and appointments carry their location's time zone.
6. GET /v1 answers the service index like GET /, because that is where a client told "the base URL is /v1" looks.

The same agent on the same four jobs then read 125 KB over 16 calls, and the schedule job took two calls instead of four.

The second walk: discovery, with a smaller model

Six jobs across six parts of the business: stock and unit cost at a store, a customer's next visit, sales against last month by store, capture a walk-in lead, open and late purchase orders, a customer note. The same prompt went to Claude Sonnet and to Claude Haiku, because the smaller model is the one a big read hurts.

Every answer given was correct. The waste was all in discovery, and two things stood out:

• Both agents could take in only about 6,000 characters of a read at a time through their shell. Both read the 14 KB map twice. Sonnet, seeing only its first tags, went to the flat catalog and read all 51 KB of it to grep it.
• Haiku went tag by tag as designed, and the tag reads were 8 to 16 KB each because they carried every operation's whole description.

What we changed:

1. The map is the tags alone, each with what it covers, its count and its address. 14 KB.
2. The catalog of every operation in a line lives at /openapi/operations, and ?q=word makes the server grep it. A search for stock returns 3.5 KB instead of 51 KB.
3. A tag's read is each operation in a few lines: summary, opening sentences, the permission every call needs, what it moves, whether a repeat is safe, and the address of its own read. Median 5 KB, from 20 KB.
4. The first call says where the map is, so the chain does not end there.
5. One list description points at the sibling an agent reached for second.

The same six jobs again

What this says, bluntly:

• The ceiling moved and the floor did not. No read is now larger than the 14 KB map, and Haiku's total fell 38%. But the bytes an agent spends on discovery per job held at 15 to 20 KB, because the map and tag reads replaced the catalog rather than removing a hop.
• Haiku lost a job to its call budget, not to a wrong turn. On the first job it spent ten calls on the map (twice, its own doing), a tag, a sibling tag, the locations list and two stock reads, and stopped one call short of asking for the cost fields it had already seen named. Completion for a small model is a budget question as much as a discovery one.
• Both agents still read the map twice, because each shows itself only the first few thousand characters of a read. Both read the identity call first because the service index said to, and both said it helped none of the six jobs. The service index now begins at the map and makes identity a read for when a job needs it.
• Eleven of twelve first picks were right. The one miss: "open and late purchase orders" is a report under Reports, and the Purchasing tag did not say so. The purchase order list now points at the report.
• Report reads named rows and totals but not the row's fields, so both agents called first and read the fields off the answer. An operation's read now names fields one level down: each row of a report, each record on a page, each object a field holds.

Two things the walks asked for are not done, with why. A filter for "a salesperson at a store" on the staff list: both agents inferred it from job titles and role names, and the honest version, a filter by permission held at a location, is a day's work. And a natural key for payment methods: names are not unique per business, so it stays a lookup.

Then the MCP server, and what its walks found

With the reads sized, the MCP server was what we expected it to be: a wrapper that reads the document in pieces and calls one operation by id, naming no operation itself, so it cannot drift from the document. Two Haiku agents, an owner and a salesperson, did their jobs through its tools, and the log of every tool call found the server's own waste: a call made right after the model's own read of the operation read it again, 10 to 27 KB, to learn its method and path. The server now learns an operation's place from the read the model already made. The owner's second walk found an answer no agent should get: a month's journal export answered with every entry's id, 311 KB. The batch is now its count, and its entries are a page at a time behind their own read.

One tool for every call was wrong too. Every client we tried, Claude's, ChatGPT's and Codex, confirms a call by the tool's annotations, so one tool for reads and writes had every lookup confirmed as if it moved money. Reads are one tool, marked read-only, and writes another.

What we would tell anyone tuning an API for agents

1. Measure against the tool the agent reads through. For a shell, that is a few thousand characters and a line break, not the model's context window.
2. Measure the read, not the schema. A tool definition of 400 tokens was riding inside a 2,600-token read because of what came along with it.
3. Give agents a map that fits in one read, then one read per step. Small reads beat one big read even at more calls: a lesser agent reads the first few thousand characters of a large read and acts on those.
4. In a read for calling an operation, include what the request needs and name the rest. Field names are enough to read an answer; agents fetched 40 KB schemas to learn 40 names.
5. Let the caller cut the answer. fields= turned a 6,000-token page into 500 and let the agent ask for what its question needed. List rows small; put the whole record behind its own read.
6. Grep on the server. Telling an agent to grep a catalog means it downloads the catalog.
7. Refuse when a caller can see nothing, and say how to get access. Do not answer with an empty list.
8. Write every refusal as an instruction that will be executed: the exact figure to send next, and no shortcuts. Refusals are the best documentation an agent reads, because it reads them at the moment it needs them. Every guessed parameter or field in every walk came back with the real names, and every agent recovered.
9. Natural keys everywhere a person quotes a number.
10. Test the generated document's output with conventions, not just the code that writes it.
11. Test with agents that know only the base URL, and with the smaller model. Sonnet worked around every gap; Haiku showed where they were.
12. Check each finding against the API before you fix it.