Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If an AI agent can't figure out how your API works, neither can your users

stytch.com

Post reply on HN
Print view View original

51–55 of 55 posts

Re: If an AI agent can't figure out how your API works, neither can your users

#51

Post by palmotea »

ModernMech wrote: post #44
eddd-ddde wrote: post #42

Earlier quoted context omitted.

And API is just a set of functions and types that determine an interface. I fully believe an LLM is better at creating APIs than most programmers.

> And API is just a set of functions and types that determine an interface. That's the kind of thinking that leads to janky APIs. When you say "just" you're doing the same thing the LLM does - you're removing all the nuance and complexity from the activity. For example, your concept of an API as just a set of functions does not consider how the API changes over time. Library authors who take this into account will ha…

>> And API is just a set of functions and types that determine an interface.

> That's the kind of thinking that leads to janky APIs. When you say "just" you're doing the same thing the LLM does - you're removing all the nuance and complexity from the activity.

I wonder if that kind of thinking is a component of the difference between LLM fans and LLM skeptics. If the programmer who is not sensitive to "nuance and complexity" gets something lacking it from an LLM, they're happy today. A programmer who is sensitive to "nuance and complexity" would be unhappy with the same response.

Re: If an AI agent can't figure out how your API works, neither can your users

#52

Post by nativeit »

lesser23 wrote: post #18
pcwelder wrote: post #7

I believe this applies to all AI use cases to varying degrees. If AI can't use X, then there is something wrong with X. X in { website, codebase, function, language, library, mcp, ...}

“AI” can’t use X so we have to dumb it down to the point a next token predictor can figure it out. Every day it seems like we are using spicy autocomplete as a measure of understandability which seems entirely silly to me. My own employer has ascribed some sort of spiritual status to prompts. The difference between prompting an LLM and a seance with psychedelics is getting smaller and smaller. The next AI winter is g…

Your description of your employer struck a chord that's been resonating in me for the last several months. I'm legitimately concerned about the knowledge gap with regards to how LLMs work, and a new generation of cults using them as quasi-deities (in both good/bad faith, as it were).

Re: If an AI agent can't figure out how your API works, neither can your users

#53

Post by exabrial »

liampulles wrote: post #43
exabrial wrote: post #30

We used to have things called WSDLs and XSD schemas that made it _extraordinarily_ easy to make remote calls. Granted, a bunch of ding dongs never loaded their own WSDL to look at it, creating a bad rap. We do have: * WADLs: https://en.wikipedia.org/wiki/Web_Application_Description_La... * JSON Schema: https://json-schema.org/learn/miscellaneous-examples And when they are available they're incredible, but nobody uses…

I think one can define a good API or a bad API using WSDLs (as with most commonly used schemas), but I have to say my experience with consuming WSDLs has universally been the latter. "My theory" is that the ease at which one can turn a function into an exposed, documented API is inversely proportional to the likelihood of it being a quality API. I think automagic annotations which turn functions into JSON APIs obey t…

> that the ease at which one can turn a function into an exposed, documented API is inversely proportional to the likelihood of it being a quality API.

I guess we'll just call this "liampulles postulate of apis"

Re: If an AI agent can't figure out how your API works, neither can your users

#54

Post by syklemil »

lesser23 wrote: post #18
pcwelder wrote: post #7

I believe this applies to all AI use cases to varying degrees. If AI can't use X, then there is something wrong with X. X in { website, codebase, function, language, library, mcp, ...}

“AI” can’t use X so we have to dumb it down to the point a next token predictor can figure it out. Every day it seems like we are using spicy autocomplete as a measure of understandability which seems entirely silly to me. My own employer has ascribed some sort of spiritual status to prompts. The difference between prompting an LLM and a seance with psychedelics is getting smaller and smaller. The next AI winter is g…

> My own employer has ascribed some sort of spiritual status to prompts.

You may want to get them away from the prompts asap. They might be headed down the route to heavy spiritual delusions: https://www.rollingstone.com/culture/culture-features/ai-spi...

It seems that just like how some people predisposed to psychosis should stay away from certain recreational drugs, some people should stay away from LLMs.

Re: If an AI agent can't figure out how your API works, neither can your users

#55

Post by neves »

ModernMech wrote: post #38
neves wrote: post #13

In one of my first uses of a LLM for helping me code, was to use a not popular Python library API. It completely hallucinated the API calls, but what was interesting is that the invented API was much more pythonic than the real one. It created a better API. Now I'm waiting for the change to use LLMs for creating an API for a package of mine. It will be averaged from all other apis and won't have unexpected calls.

> It created a better API. Did it though? It didn't create an API, it created the appearance of an API. The reality of the API, something the library author had to wrestle with and the LLM didn't, is probably much more complex and nuanced than the LLM is hallucinating. Maybe the API it's hinting at would be better if made real. But it pains me you're telling us the LLM, a tool known for making things up and being wro…

I'm not saying it can do better than a good developer, but that it knows the "average" way to do it.

There's one thing called the "curse of knowledge". You don't know how it is to don't know something. The LLMs will spit an "average" way to do it, without breaking user expectations.

I've seen a lot of badly developed apis.

Post reply on HN