Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Designing APIs for Agents

freestyle.sh

Post reply on HN
Print view View original

51–58 of 58 posts

Re: Designing APIs for Agents

#51

Post by grim_io »

benswerd wrote: post #11
CoolestBeans wrote: post #8

Earlier quoted context omitted.

Because the defaults aren't just for convenience, the API designer is also making the parameters they think should be used the most have the least resistance. Good example is runtime parameters like in the JVM. You shouldn't start with having to tune your JVM, you probably want a middle of the road place to start with even if you know you're going to tune it.

im not opposed to good defaults, i just believe they should be explicit. The AI should read the starting.md to fill in explicitly what its JVM configuration is. Then, when you want to tune in the future its clear what options are available and what specifically is changing.

I can't parse this. If you explicitly define recommended values, those are not what I consider "defaults".

Re: Designing APIs for Agents

#52

Post by catlifeonmars »

kolinko wrote: post #48
catlifeonmars wrote: post #31

Earlier quoted context omitted.

You won’t get grounded results by asking for feedback. The LLM doesn’t know its own internal state so while you may get a plausible and contextually significant answer, it isn’t going to be an accurate one.

Are you talking from experience or from theory? I have a reflections mechanism, which launches right after the session ends (when it’s still in cache, so it’s almost free) and it catches a ton of problems with tool/skill/system design, all the time. Low level stuff can be fixed almost automatically (e.g. agent funbled 10 turns bc tool’s docs were wrong), high level too but human gating is useful. If you try doing it…

Theory. I guess my question is: why do you think it works so well. What’s the mechanism behind it?

It feels like it would be relatively random across the range of plausible results, but it sounds like you are finding otherwise.

Re: Designing APIs for Agents

#53

Post by kolinko »

catlifeonmars wrote: post #52
kolinko wrote: post #48

Earlier quoted context omitted.

Are you talking from experience or from theory? I have a reflections mechanism, which launches right after the session ends (when it’s still in cache, so it’s almost free) and it catches a ton of problems with tool/skill/system design, all the time. Low level stuff can be fixed almost automatically (e.g. agent funbled 10 turns bc tool’s docs were wrong), high level too but human gating is useful. If you try doing it…

Theory. I guess my question is: why do you think it works so well. What’s the mechanism behind it? It feels like it would be relatively random across the range of plausible results, but it sounds like you are finding otherwise.

Why would it be random? Perhaps we’re talking about two different things?

I’m talking about asking an agent at the end of the session how well the session went - what tools it found useful, what it found confusing and whatnot. Agents do gladly complain about doing pointless work or fighting with the harness :)

Re: Designing APIs for Agents

#54

Post by hankbond »

hdjrudni wrote: post #46
hankbond wrote: post #24

Re: default values are bad One of my recent projects has contributable code (via extensions) with a central settings management (json file). What I do is take all the defaults contributed by the extensions (namespaced by extension name) and materialize them to the settings file as the initial values to each setting. When an agent wants to edit the settings, it already knows the entirety of the setting surface area an…

I did that too but not for the sake of the LLM, just for myself. Massively helps discoverability. It also protects from backwards-compat if you ever want to change a default without breaking existing users, knowing that whatever they had before has already been saved into their settings so some setting doesn't randomly change on them.

That was also my reasoning. I'm a fan for abstraction and nesting for DRY reasons but always find it mentally taxing to dive in and out of "functions". So while the code is organized this way to create the settings, they can be read flat all at once. Want to reset the value to the default but forgot what it was? Delete the entry and it'll reset on boot.

Re: Designing APIs for Agents

#55

Post by catlifeonmars »

kolinko wrote: post #53
catlifeonmars wrote: post #52

Earlier quoted context omitted.

Theory. I guess my question is: why do you think it works so well. What’s the mechanism behind it? It feels like it would be relatively random across the range of plausible results, but it sounds like you are finding otherwise.

Why would it be random? Perhaps we’re talking about two different things? I’m talking about asking an agent at the end of the session how well the session went - what tools it found useful, what it found confusing and whatnot. Agents do gladly complain about doing pointless work or fighting with the harness :)

Let me put it this way. How does the model know what it finds confusing?

I’m sure it works great fwiw, also the use of a hot cache is clever.

Re: Designing APIs for Agents

#57

Post by pjm331 »

what wrote: post #27
pjm331 wrote: post #14

Earlier quoted context omitted.

Yeah i have a similar setup at the harness level - a “devlog” at the end of every session about the experience developing, what could have been better, what was confusing And also files issues for blockers I’ve absolutely caught things and made improvements just from skimming them occasionally - they are particularly useful when you get a PR that makes you scratch your head But I’m definitely not taking full advantag…

Why are you asking an LLM what could have been better or was confusing? It literally has no idea.

Listen friend I’m also in the camp of LLMs are just math and data and don’t actually “have an idea” about anything but that doesn’t make them any less accurate/effective at generating tokens

I’m not sure why this prompt is different than any other in that regard

Post reply on HN