Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You should write an agent

fly.io

Post reply on HN
Print view View original

271–280 of 409 posts

Re: You should write an agent

#271

Post by robot-wrangler »

simonw wrote: post #259
robot-wrangler wrote: post #256

Earlier quoted context omitted.

The expectation that reasonable people have isn't fully local claude code, that's a strawman. But it's also not ping tools or the simple weather agent that tutorials like to use. It's somewhere in between, isn't that obvious? If you're into evangelism, acknowledging this and actually taking a measured stance would help prevent light skeptics from turning into complete AI-deniers. If you mislead people about one thing…

I don't think I was being misleading here. https://fly.io/blog/everyone-write-an-agent/ is a tutorial about writing a simple "agent" - aka a thing that uses an LLM to call tools in a loop - that can make a simple tool call. The complaint I was responding to here was that there's no point trying this if you don't want to be hooked on expensive APIs. I think this is one of the areas where the existence of tiny but capa…

I think it is misleading to suggest today that tool-calling for nontrivial stuff really works with local models. It just works in demos because those tools always accept one or two arguments, usually string literals or numbers. In the real world functions take more complex arguments, many arguments, or take a single argument that's an object with multiple attributes, etc. You can begin to work around this stuff by passing function signatures, typing details, and JSON-schemas to set expectations in context, but local models tend to fail at handling this kind of stuff long before you ever hit limits in the context window. There's a reason demos are always using 1 string literal like hostname, or 2 floats like lat/long. It's normal that passing a dictionary with a few strict requirements might need 300 retries instead of 3 to get a tool call that's syntactically correct and properly passed arguments. Actually `ping --help` for me shows like 20 options, and for any attempt to 1:1 map things with more args I think you'd start to see breakdown pretty quickly.

Zooming in on the details is fun but doesn't change the shape of what I was saying before. No need to muddy the water; very very simple stuff still requires very big local hardware or a SOTA model.

Re: You should write an agent

#272

Post by greenfish6 »

andai wrote: post #197
ericd wrote: post #41

Absolutely, especially the part about just rolling your own alternative to Claude Code - build your own lightsaber. Having your coding agent improve itself is a pretty magical experience. And then you can trivially swap in whatever model you want (Cerebras is crazy fast, for example, which makes a big difference for these many-turn tool call conversations with big lumps of context, though gpt-oss 120b is obviously no…

What are you using for transcription? I tried Whisper, but it's slow and not great. I tried the gpt audio models, but they're trained to refuse to transcribe things. I tried Google's models and they were terrible. I ended up using one of Mistral's models, which is alright and very fast except sometimes it will respond to the text instead of transcribing it. So I'll occasionally end up with pages of LLM rambling paste…

I use Willow AI, which I think is pretty good

Re: You should write an agent

#273

Post by jillesvangurp »

hoppp wrote: post #93

I should? what problems can I solve, that can be only done with an agent? As long as every AI provider is operating at a loss starting a sustainably monetizable project doesn't feel that realistic.

You are asking the wrong questions. You should be asking what the problems are that you can still solve better and cheaper than an agent? Because anything else, you are probably doing it wrong (the slow and expensive way). That's not long term sustainable. It helps if you know how agents work and as the article argues, there isn't a whole lot to that.

Re: You should write an agent

#275

Post by utopiah »

chemotaxis wrote: post #13
riskable wrote: post #11

It's interesting how much this makes you want to write Unix-style tools that do one thing and only one thing really well. Not just because it makes coding an agent simpler, but because it's much more secure!

You could even imagine a world in which we create an entire suite of deterministic, limited-purpose tools and then expose it directly to humans!

Hmmm but how would you name that? Agent skills? Meta cognition agentic tooling? Intelligence driven self improving partial building blocks?

Oh... oh I know how about... UNIX Philosophy? No... no that'd never work.

/s

Re: You should write an agent

#276

Post by cantor_S_drug »

oooyay wrote: post #7

Heh, the bit about context engineering is palpable. I'm writing a personal assistant which, imo, is distinct from an agent in that it has a lot of capabilities a regular agent wouldn't necessarily need such as memory, task tracking, broad solutioning capabilities, etc... I ended up writing agents that talk to other agents which have MCP prompts, resources, and tools to guide them as general problem solvers. The first…

https://github.com/mem0ai/mem0?tab=readme-ov-file

Is this useful for you?

Re: You should write an agent

#277

Post by globular-toast »

The formatting of the code is messed up on my phone. I was looking at the first bit thinking `call` was a function returning `None`. I thought initially it was doing some clever functional programming stuff but, no, just a linebreak that shouldn't be there.

Re: You should write an agent

#278

Post by cantor_S_drug »

ht96 wrote: post #110
furyofantares wrote: post #95

Earlier quoted context omitted.

That's why you build extensive tooling to run your change hundreds of times in parallel against the context you're trying to fix, and then re-run hundreds of past scenarios in parallel to verify none of them breaks.

Do you use a tool for this? Is there some sort of tool which collects evals from live inferences (especially those which fail)

https://x.com/rerundotio/status/1968806896959402144

This is a use of Rerun that I haven't seen before!

This is pretty fascinating!!!

Typically people use Rerun to visualize robotics data - if I'm following along correctly... what's fascinating here is that Adam for his master's thesis is using Rerun to visualize Agent (like ... software / LLM Agent) state.

Interesting use of Rerun!

https://github.com/gustofied/P2Engine

Re: You should write an agent

#279

Post by awayto »

worldsayshi wrote: post #267
awayto wrote: post #67

Earlier quoted context omitted.

You can build your agent into a docker image then easily limit both networking and file system scope. docker run -it --rm \ -e SOME_API_KEY="$(SOME_API_KEY)" \ -v "$(shell pwd):/app" \ /dev/null | awk '{printf " --add-host=llm-provider.com:%s", $$0}') \ Probably could be a bit cleaner, but it worked for me.

Putting it inside docker is probably fine for most use cases but it's generally not considered to be a safe sandbox AFAIK. A docker container shares kernel with the host OS which widens the attack surface. If you want your agent to pull untrusted code from the internet and go wild while you're doing other stuff it might not be a good choice.

Could you point to some resources which talk about how docker isn't considered a safe sandbox given the network and file system restrictions I mentioned?

I understand the sharing of kernel, while I might not be aware of all of the implications. I.e. if you have some local access or other sophisticated knowledge of the network/box docker is running on, then sure you could do some damage.

But I think the chances of a whitelisted llm endpoint returning some nefarious code which could compromise the system is actually zero. We're not talking about untrusted code from the internet. These models are pretty constrained.

Re: You should write an agent

#280

Post by cantor_S_drug »

aaronblohowiak wrote: post #214

THEY SEND THE WHOLE CONTEXT EVERY TIME? Man that seems... not great. sometimes it will go off and spin on something.. seems like it would be a LOT better to roll back than to send a corrective message. hmmm...... this article is nerd-sniping on a massive scale ;D

There is context caching in many models. It is less expensive if you enable that.
Post reply on HN