Designing APIs for Agents

Just did this, found result fairly interesting, I reject most of its objections on the basis of bad code. When I say Im pro explicitness I'm also pro comment, and pro separating the "this is core fact" and "this is configuration that i don't care too much about". Will use this test for future writing.

Re: default values are bad

One of my recent projects has contributable code (via extensions) with a central settings management (json file). What I do is take all the defaults contributed by the extensions (namespaced by extension name) and materialize them to the settings file as the initial values to each setting. When an agent wants to edit the settings, it already knows the entirety of the setting surface area and thier values. It never has to go digging to find default values nested somewhere in the extension code or documentation. Also, the code that reads the extensions into the framework (for runtime execution) only reads the materialized values, it doesn't have a concept of a default value past the point of materialization. Defaults only exist in the extension registration/mounting part of the lifecycle (so they can materialize if missing from the settings file).

One upside of this is as new settings are created (from new features being developed) your configuration gets notified via the new fields being materialized into your settings upon startup.

AFAIK this is not at all a new practice. I used to see TOML files with default values in commented out lines all the time.

The downside is that the agent is not aware of which settings are being overridden from the default (like a sparse settings file would provide), but I don't know how much semantic value that offers in most cases (other than maybe debugging?).

I'm not even taking a side on this one this is just a pattern I decided to take for this use case.

I had to deal with exactly this issue with one of my recent oss projects so I can share a few things on the topic.

1. Text is the default interface

i.e. the api must be text based first but it should allow to fallback to structured output by using the accept header.

That being said, it is not wrong to introduce other means to return json by using ?format=json etc.

2. Make it grepable

Basically surface as much useful information in a single line so that the agent can grep and slice.

3. Identifiers must be short

i.e. short enough to describe in 5 tokens but not too short to introduce collisions or confusion.

Otherwise you could be wasting a lot of token for nothing. However, adding prefixes helps like cus_abc123, token_xxx, etc. The prefix can help with lookup, error correction and deduplication.

4. Surface information that is likely to be used by the agent

i.e. if the agent is asking for a list of resources, don't just return the list but also some additional information that might help the agent understand better the context around the resource.

Without this a single task could take a lot more steps simply because the agent needs to run its own loop - it is slow and expensive.

5. Add bulk operations

It is a lot easier to insert 10 records in one request then performing 10 separate requests.

6. Error messages should be descriptive

Ensure that error message point to actual docs and manuals that can be read by the agent so that it can troubleshoot on its own. Also return hints.

I've added most of these at https://github.com/crmkit/crmkit which is highly experimental CRM I tried to specifically design for AI agents. We use it internally for a few projects and it is not perfect but I think it might be on the right direction.

I hope this helps.

Having just joined Twilio (thanks for buying us) and designing some new APIs right now, it warmed my heart to see messages.create(), written over 15 years ago, show up as an example here :)

True. But it has no idea that it has no idea, so it might be able to look back at the session trace and pattern match its way to actionable feedback?