Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You should write an agent

fly.io

Post reply on HN
Print view View original

361–370 of 409 posts

Re: You should write an agent

#361

Post by SJC_Hacker »

recursive wrote: post #359
criddell wrote: post #354

Earlier quoted context omitted.

> Personally I’d absolutely buy an LLM in a box In a box? I want one in a unit with arms and legs and cameras and microphones so I can have it do useful things for me around my home.

You're an optimist I see. I wouldn't allow that in my house until I have some kind of strong and comprehensible evidence that it won't murder me in my sleep.

A silly scenario. LLMs don’t have independent will. They are action / response.

If home robot assistants become feasible, they would have similar limitations

Re: You should write an agent

#362

Post by ericmcer »

coffeecoders wrote: post #138
hshdhdhehd wrote: post #114

There is a lot of stuff I should do. From making my own CPU from a breadboard of nand gates to building a CDN in Rust. But aint got time for all the things. That said I built an LLM following Karpathy's tutorial. So I think it aims good to dabble a bit.

Yeah, it’s a never-ending curve. I built an 8-bit computer on breadboards once, then went down the rabbit hole of flight training for a PPL. Every time I think I’m "done," the finish line moves a few miles further. Guess we nerds are never happy.

Seriously I feel like it's self-sabotage sometimes at work. Just fixing the thing getting tests to pass isn't enough. Until I fully have a mental model of what is happening I can't move on.

Re: You should write an agent

#363

Post by losvedir »

I appreciate the goal of demystifying agents by writing one yourself, but for me the key part is still a little obscured by using OpenAI APIs in the examples. A lot of the magic has to do with tool calls, which the API helpfully wraps for you, with a format for defining tools and parsed responses helpfully telling you the tools it wants to call.

I kind of am missing the bridge between that, and the fundamental knowledge that everything is token based in and out.

Is it fair to say that the tool abstraction the library provides you is essentially some niceties around a prompt something like "Defined below are certain 'tools' you can use to gather data or perform actions. If you want to use one, please return the tool call you want and it's arguments, delimited before and after with '###', and stop. I will invoke the tool call and then reply with the output delimited by '==='".

Basically, telling the model how to use tools, earlier in the context window. I already don't totally understand how a model knows when to stop generating tokens, but presumably those instructions will get it to output the request for a tool call in a certain way and stop. Then the agent harness knows to look for those delimiters and extract out the tool call to execute, and then add to the context with the response so the LLM keeps going.

Is that basically it? Or is there more magic there? Are the tool call instructions in some sort of permanent context, or could the interaction demonstrated in a fine tuning step, and inferred by the model and just in its weights?

Re: You should write an agent

#364

Post by richardlblair »

andai wrote: post #197
ericd wrote: post #41

Absolutely, especially the part about just rolling your own alternative to Claude Code - build your own lightsaber. Having your coding agent improve itself is a pretty magical experience. And then you can trivially swap in whatever model you want (Cerebras is crazy fast, for example, which makes a big difference for these many-turn tool call conversations with big lumps of context, though gpt-oss 120b is obviously no…

What are you using for transcription? I tried Whisper, but it's slow and not great. I tried the gpt audio models, but they're trained to refuse to transcribe things. I tried Google's models and they were terrible. I ended up using one of Mistral's models, which is alright and very fast except sometimes it will respond to the text instead of transcribing it. So I'll occasionally end up with pages of LLM rambling paste…

The new Qwen model is supposed to be very good.

Honestly, I've gotten really far simply by transcribing audio with whisper, having a cheap model clean up the output to make it make sense (especially in a coding context), and copying the result to the clipboard. My goal is less about speed and more about not touching the keyboard, though.

Re: You should write an agent

#365

Post by richardlblair »

MinimalAction wrote: post #336

Do we need an agent? I get the point of this post: have fun building one because it's easy. But every time I see one of these takes, I keep wondering why do we encourage a tool that would potentially replace us. Why help it build better that could eventually take away what was fun and sustainable income-wise?

I've been building tools for stuff I don't want to do. Any task where I need to take some amount of data, structured or unstructured, and need a specific outcome is perfect. That way I can spend more time on the thing I do want to do (including building these little tools).

Re: You should write an agent

#366

Post by JoshMandel »

losvedir wrote: post #363

I appreciate the goal of demystifying agents by writing one yourself, but for me the key part is still a little obscured by using OpenAI APIs in the examples. A lot of the magic has to do with tool calls, which the API helpfully wraps for you, with a format for defining tools and parsed responses helpfully telling you the tools it wants to call. I kind of am missing the bridge between that, and the fundamental knowle…

I think that it's basically fair and I often write simple agents using exactly the technique that you describe. I typically provide a TypeScript interface for the available tools and just ask the model to respond with a JSON block and it works fine.

That said, it is worth understanding that the current generation of models is extensively RL-trained on how to make tool calls... so they may in fact be better at issuing tool calls in the specific format that their training has focused on (using specific internal tokens to demarcate and indicate when a tool call begins/ends, etc). Intuitively, there's probably a lot of transfer learning between this format and any ad-hoc format that you might request inline your prompt.

There may be recent literature quantifying the performance gap here. And certainly if you're doing anything performance-sensitive you will want to characterize this for your use case, with benchmarks. But conceptually, I think your model is spot on.

Re: You should write an agent

#367

Post by deadbabe »

The more I use agents, the more I find agents to be pointless, any tasks an agent performs regularly in high volume should be turned into classical deterministic code.

The number one feature of agents is to be disambiguation for tool selectors and pretty printers.

Re: You should write an agent

#369

Post by nextaccountic »

SJC_Hacker wrote: post #361
recursive wrote: post #359

Earlier quoted context omitted.

You're an optimist I see. I wouldn't allow that in my house until I have some kind of strong and comprehensible evidence that it won't murder me in my sleep.

A silly scenario. LLMs don’t have independent will. They are action / response. If home robot assistants become feasible, they would have similar limitations

An agent is a higher level thing that could run as a daemon

Re: You should write an agent

#370

Post by fryz »

losvedir wrote: post #363

I appreciate the goal of demystifying agents by writing one yourself, but for me the key part is still a little obscured by using OpenAI APIs in the examples. A lot of the magic has to do with tool calls, which the API helpfully wraps for you, with a format for defining tools and parsed responses helpfully telling you the tools it wants to call. I kind of am missing the bridge between that, and the fundamental knowle…

The "magic" is done via the JSON schemas that are passed in along with the definition of the tool.

Structured Output APIs (inc. the Tool API) take the schema and build a Context-free Grammar, which is then used during generation to mask which tokens can be output.

I found https://openai.com/index/introducing-structured-outputs-in-t... (have to scroll down a bit to the "under the hood" section) and https://www.leewayhertz.com/structured-outputs-in-llms/#cons... to be pretty good resources

Post reply on HN