Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You should write an agent

fly.io

Post reply on HN
Print view View original

291–300 of 409 posts

Re: You should write an agent

#291

Post by michaelanckaert »

aaronblohowiak wrote: post #214

THEY SEND THE WHOLE CONTEXT EVERY TIME? Man that seems... not great. sometimes it will go off and spin on something.. seems like it would be a LOT better to roll back than to send a corrective message. hmmm...... this article is nerd-sniping on a massive scale ;D

Sending the whole context on each user message is essentially what the model remembers of this conversation. ie: it is entirely stateless.

I've written some agents that have their context altered by another llm to get it back on track. Let's say the agent is going off rails, then a supervisor agent will spot this and remove messages from the context where it went off rails, or alter those with correct information. Really fun stuff but yeah, we're essentially still inventing this as we go along.

Re: You should write an agent

#292

Post by raymond_goo »

andai wrote: post #197
ericd wrote: post #41

Absolutely, especially the part about just rolling your own alternative to Claude Code - build your own lightsaber. Having your coding agent improve itself is a pretty magical experience. And then you can trivially swap in whatever model you want (Cerebras is crazy fast, for example, which makes a big difference for these many-turn tool call conversations with big lumps of context, though gpt-oss 120b is obviously no…

What are you using for transcription? I tried Whisper, but it's slow and not great. I tried the gpt audio models, but they're trained to refuse to transcribe things. I tried Google's models and they were terrible. I ended up using one of Mistral's models, which is alright and very fast except sometimes it will respond to the text instead of transcribing it. So I'll occasionally end up with pages of LLM rambling paste…

https://github.com/rhulha/Speech2Speech

https://github.com/rhulha/EchoMate

Re: You should write an agent

#293

Post by svnt »

Workaccount2 wrote: post #109
hoppp wrote: post #105

Earlier quoted context omitted.

When comparing the cost of an H100 GPU per hour and calculating cost of tokens, it seems the OpenAI offering for the latest model is 5 times cheaper than renting the hardware. OpenAI balance sheet also shows an $11 billion loss . I can't see any profit on anything they create. The product is good but it relies on investors fueling the AI bubble.

https://martinalderson.com/posts/are-openai-and-anthropic-re... All the labs are going hard on training and new GPUs. If we ever level off, they probably will be immensely profitable. Inference is cheap, training is expensive.

To do this analysis on an hourly retail cost and an open weight model and infer anything about the situation at OpenAI or Anthropic is quite a reach.

For one (basic) thing, they buy and own their hardware, and have to size their resources for peak demand. For another, Deepseek R1 does not come close to matching claude performance in many real tasks.

Re: You should write an agent

#294

Post by ljm »

singularity2001 wrote: post #246
dave1010uk wrote: post #80

Two years ago I wrote an agent in 25 lines of PHP [0]. It was surprisingly effective, even back then before tool calling was a thing and you had to coax the LLM into returning structured output. I think it even worked with GPT-3.5 for trivial things. In my mind LLMs are just UNIX strong manipulation tools like `sed` or `awk`: you give them an input and command and they give you an output. This is especially true if y…

what's the point of specialized agents when you just have one universal agent that can do anything e.g. Claude

Composing multiple smaller agents allows you to build more complex pipelines, which is a lot easier than getting a single monolithic agent to switch between contexts for different tasks. I also get some insight into how the agent performs (e.g via langfuse) because it’s less of a black box.

To use an example: I could write an elaborate prompt to fetch requirements, browse a website, generate E2E test cases, and compile a report, and Claude could run it all to some degree of success. But I could also break it down into four specialised agents, with their own context windows, and make them good at their individual tasks.

Re: You should write an agent

#295

Post by a-french-anon »

tptacek wrote: post #49
jacquesm wrote: post #47

Earlier quoted context omitted.

Interesting. You have to wonder if all the tools that is based on would have been written in the first place if that kind of thing had been possible all along. Who needs 'grep' when you can write a prompt?

My long running joke is that the actual good `jq` is just the LLM interface that generates `jq` queries; 'simonw actually went and built that.

Tried gron (https://github.com/tomnomnom/gron) a bit? If you know your UNIX, I think it can replace jq in a lot of cases. And when it can't, well, you can reach for Python, I guess.

Re: You should write an agent

#296

Post by GardenLetter27 »

ericd wrote: post #41

Absolutely, especially the part about just rolling your own alternative to Claude Code - build your own lightsaber. Having your coding agent improve itself is a pretty magical experience. And then you can trivially swap in whatever model you want (Cerebras is crazy fast, for example, which makes a big difference for these many-turn tool call conversations with big lumps of context, though gpt-oss 120b is obviously no…

But it's way more expensive since most providers won't give you prompt caching?

Re: You should write an agent

#297

Post by worldsayshi »

cornel_io wrote: post #270
worldsayshi wrote: post #265

I feel like one small piece is missing to call it an agent? The ability to iterate in multiple steps until it feels like it's "done". What is the canonical way to do that? I suspect that implementing that in the wrong way could make it spiral.

When a tool call completes the result is sent back to the LLM to decide what to do next, that's where it can decide to go do other stuff before returning a final answer. Sometimes people use structured outputs or tool calls to explicitly have the LLM decide when it's done, or allow it to send intermediate messages for logging to the user. But the simple loop there lets the LLM do plenty of it has good tools.

So it returns a tool call for "continue" every time it wants to continue working? Do people implement this in different ways? It would be nice what method it has been trained on if any.

Re: You should write an agent

#298

Post by otsaloma »

Agreed! It's easy understand "LLM with tools in a loop" at a high-level, but once you actually design the architecture and implement the code in full, you'll have proper understanding of how it all fits and works together.

I did the same exercise. My implementation is at around 300 lines with two tools: web search and web page fetch with a command line chat interface and Python package. And it could have been a lot less lines if I didn't want to write a usable, extensible package interface.

As the agent setup itself is simple, majority of the work to make this useful would in the tools themselves and context management for the tools.

Re: You should write an agent

#299

Post by svnt »

furyofantares wrote: post #123
lmm wrote: post #107

Earlier quoted context omitted.

> But I don't think any are operating inference at a loss, I think their margins are actually rather large. Citation needed. I haven't seen any of them claim to have even positive gross margins to shareholders/investors, which surely they would do if they did.

https://officechai.com/ai/each-individual-ai-model-can-alrea...

> “if you consider each model to be a company, the model that was trained in 2023 was profitable. You paid $100 million, and then it made $200 million of revenue. There’s some cost to inference with the model, but let’s just assume, in this cartoonish cartoon example, that even if you add those two up, you’re kind of in a good state. So, if every model was a company, the model, in this example, profitable,” he added.

“What’s going on is that while you’re reaping the benefits from one company, you’re founding another company that’s much more expensive and requires much more upfront R&D investment. The way this is going to shake out is that it’s going to keep going up until the numbers get very large, and the models can’t get larger, and then there will be a large, very profitable business. Or at some point, the models will stop getting better, and there will perhaps be some overhang — we spent some money, and we didn’t get anything for it — and then the business returns to whatever scale it’s at,” he said.

This take from Amodei is hilarious but explains so much.

Post reply on HN