Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If an AI agent can't figure out how your API works, neither can your users

stytch.com

Post reply on HN
Print view View original

41–50 of 55 posts

Re: If an AI agent can't figure out how your API works, neither can your users

#41

Post by thechao »

dramdass868 wrote: post #17

It’s a great forcing function for: * Simplicity of input knobs - way too many APIs are unapproachable with the number and complexity of inputs * Complete documentation - if you don’t document a parameter or endpoint, expect an AI agent is never going to use it (or at least use it the way you want it to) especially in multi-agentic systems where your tool needs to be chosen by the LLM * Clear, descriptive API outputs…

Todd Veldhuizen — of C++ template metaprogramming (in)fame(y) — had a paper called "Parsimony Principles in Software Programming" or something like that. His argument was that libraries should be built in decomposable layers. The bottom layer should be all of the basic utilities that can be cobbled together to build the thing you want. On top of that should be convenience layers that follow "standard happy paths". The unbreakable rule of the upper layers was that they could only be written to the public API of the lower layers, and they should try to expose their internals (given appropriate invariants) as much as possible. Such libraries then come "knob free" for people who just need to "knock stuff together"; but, if you had to dig down, there was a discipline and a public API you could use for the parts that the higher levels were built from.

I mean — maybe in some glorious alternate timeline, but not ours, I guess?

Re: If an AI agent can't figure out how your API works, neither can your users

#42

Post by eddd-ddde »

ModernMech wrote: post #38
neves wrote: post #13

In one of my first uses of a LLM for helping me code, was to use a not popular Python library API. It completely hallucinated the API calls, but what was interesting is that the invented API was much more pythonic than the real one. It created a better API. Now I'm waiting for the change to use LLMs for creating an API for a package of mine. It will be averaged from all other apis and won't have unexpected calls.

> It created a better API. Did it though? It didn't create an API, it created the appearance of an API. The reality of the API, something the library author had to wrestle with and the LLM didn't, is probably much more complex and nuanced than the LLM is hallucinating. Maybe the API it's hinting at would be better if made real. But it pains me you're telling us the LLM, a tool known for making things up and being wro…

And API is just a set of functions and types that determine an interface. I fully believe an LLM is better at creating APIs than most programmers.

Re: If an AI agent can't figure out how your API works, neither can your users

#43

Post by liampulles »

exabrial wrote: post #30

We used to have things called WSDLs and XSD schemas that made it _extraordinarily_ easy to make remote calls. Granted, a bunch of ding dongs never loaded their own WSDL to look at it, creating a bad rap. We do have: * WADLs: https://en.wikipedia.org/wiki/Web_Application_Description_La... * JSON Schema: https://json-schema.org/learn/miscellaneous-examples And when they are available they're incredible, but nobody uses…

I think one can define a good API or a bad API using WSDLs (as with most commonly used schemas), but I have to say my experience with consuming WSDLs has universally been the latter.

"My theory" is that the ease at which one can turn a function into an exposed, documented API is inversely proportional to the likelihood of it being a quality API. I think automagic annotations which turn functions into JSON APIs obey the same principle, for what its worth.

Re: If an AI agent can't figure out how your API works, neither can your users

#44

Post by ModernMech »

eddd-ddde wrote: post #42
ModernMech wrote: post #38

Earlier quoted context omitted.

> It created a better API. Did it though? It didn't create an API, it created the appearance of an API. The reality of the API, something the library author had to wrestle with and the LLM didn't, is probably much more complex and nuanced than the LLM is hallucinating. Maybe the API it's hinting at would be better if made real. But it pains me you're telling us the LLM, a tool known for making things up and being wro…

And API is just a set of functions and types that determine an interface. I fully believe an LLM is better at creating APIs than most programmers.

> And API is just a set of functions and types that determine an interface.

That's the kind of thinking that leads to janky APIs. When you say "just" you're doing the same thing the LLM does - you're removing all the nuance and complexity from the activity.

For example, your concept of an API as just a set of functions does not consider how the API changes over time. Library authors who take this into account will have a better time evolving the library API. Library authors who don't might write themselves into a corner, which might force some sort of API version schism which causes half the API to be nice while the other half has questionable decisions, causing perpetual confusion and frustration with users for decades.

The LLM hallucinating some nice looking function calls doesn't really take any of that into account.

Re: If an AI agent can't figure out how your API works, neither can your users

#45

Post by datpuz »

diggan wrote: post #6
datpuz wrote: post #4

AI agents can't even do very basic things without causing absolute mayhem. Do you really think it makes sense for your basic assumption to be that your user is dumber than a token generator?

> Do you really think it makes sense for your basic assumption to be that your user is dumber than a token generator? Have you seen the average user in action? I'm fairly sure that's true at least on average. Even putting huge red warnings like "This action is irreversible" for some things will lead to user reaching out to you saying they didn't see it.

The average user of a developer API is quite a bit smarter than the average user of, say, a mobile game

Re: If an AI agent can't figure out how your API works, neither can your users

#46

Post by exabrial »

marcosdumay wrote: post #37
exabrial wrote: post #30

We used to have things called WSDLs and XSD schemas that made it _extraordinarily_ easy to make remote calls. Granted, a bunch of ding dongs never loaded their own WSDL to look at it, creating a bad rap. We do have: * WADLs: https://en.wikipedia.org/wiki/Web_Application_Description_La... * JSON Schema: https://json-schema.org/learn/miscellaneous-examples And when they are available they're incredible, but nobody uses…

If you ever tried to read an actual WSDL file, you'd have noticed it was not great. Yes, having an API type declaration is really important. And yes, somehow a lot of people just don't use those things. But the WSDL was one of the worst standards for that in all times. (And also, it inherited all of the shitness from XML, even the allowing non-deterministic processing and side effects while reading the file.) Anyway,…

> Granted, a bunch of ding dongs never loaded their own WSDL to look at it

...

Re: If an AI agent can't figure out how your API works, neither can your users

#47

Post by stronglikedan »

jelambs wrote: post #2

Thanks for sharing! Would love to hear how others are thinking about this problem.

I mean, that's kinda why we're all here, so...

EDIT: Coincidentally, it just dawned on me that I'm very likely replying to an AI that knows how to use the HN API.

Re: If an AI agent can't figure out how your API works, neither can your users

#48

Post by john-h-k »

lesser23 wrote: post #18
pcwelder wrote: post #7

I believe this applies to all AI use cases to varying degrees. If AI can't use X, then there is something wrong with X. X in { website, codebase, function, language, library, mcp, ...}

“AI” can’t use X so we have to dumb it down to the point a next token predictor can figure it out. Every day it seems like we are using spicy autocomplete as a measure of understandability which seems entirely silly to me. My own employer has ascribed some sort of spiritual status to prompts. The difference between prompting an LLM and a seance with psychedelics is getting smaller and smaller. The next AI winter is g…

> a next token predictor can figure it out

Describing LLMs as "next token predictors" is disingenuous and wrong

Re: If an AI agent can't figure out how your API works, neither can your users

#49

Post by poly2it »

john-h-k wrote: post #48
lesser23 wrote: post #18

Earlier quoted context omitted.

“AI” can’t use X so we have to dumb it down to the point a next token predictor can figure it out. Every day it seems like we are using spicy autocomplete as a measure of understandability which seems entirely silly to me. My own employer has ascribed some sort of spiritual status to prompts. The difference between prompting an LLM and a seance with psychedelics is getting smaller and smaller. The next AI winter is g…

> a next token predictor can figure it out Describing LLMs as "next token predictors" is disingenuous and wrong

How so? Autoregressive LLMs are quite literally "next token predictors", just very sophisticated ones.

Re: If an AI agent can't figure out how your API works, neither can your users

#50

Post by poly2it »

stronglikedan wrote: post #47
jelambs wrote: post #2

Thanks for sharing! Would love to hear how others are thinking about this problem.

I mean, that's kinda why we're all here, so... EDIT: Coincidentally, it just dawned on me that I'm very likely replying to an AI that knows how to use the HN API.

I don't think they are based on their previous activity. Seems a bit to sporadic and tempered.
Post reply on HN