231–240 of 334 posts
I understand the need for memory footprint in some situations, but what's the point of seeking performance for a software that mostly calls LLMs and waits?
Before I tried coding agents my guess would have been: none. But seeing how slow claude code and copilot cli are and how much ram they use I'm flabbergasted. If you have long running sessions they can both take tens pf gigabytes of ram and feel quite sluggish.
Earlier quoted context omitted.
latency and throughput (when with Java the system is crying for more memory while it's chilling in the Rust case)
What's the latency difference between a long running process issuing a network call in Java vs rust? This is such a short time that it is completely overshadowed by noise (OS doing something else, what other software is running etc) As for throughput: you have 1-2 requests going at a time, the next one waiting for the reply. What throughput are we talking about? That's like speeding to the post office and expecting y…
Anyways, consider how higher memory usage can affect the systems performance dramatically once the system needs to start swapping memory to disk signficantly
Earlier quoted context omitted.
Aren't skills fairly easy to share, and can contain more than one file?
Prompts as well... he might be on to something here, can't say as I didn't try it yet Skills are just prompts
Prompts are just text templates entered by the user, and the user must specifically know when to and remember to invoke them. If you’re just using skills as if they are the same as prompts, you’re totally missing out on the entire benefit that skills provide!
I understand the need for memory footprint in some situations, but what's the point of seeking performance for a software that mostly calls LLMs and waits?
I didn't see anyone mention this, but I think having a single binary is much nicer than having a JS (or Python) program sprawled all over your system.
Earlier quoted context omitted.
> simpler alternative to Skills this concerns me. Skills are already just about the simplest possible thing; they're just prompts, in a directory!
Skills are notably more complex than that. They require metadata (which the model is given and uses to determine whether or not to load the main file), are intended to be loaded via a tool call, contain extra resources (also loaded by tool calls), etc. In contrast, with this system the harness doesn't need a tool to load the stored prompts, the prompts don't need to include metadata to allow for runtime discovery, et…
Skills are already dead-simple and this prompt system doesn’t at all tackle the same problem.
Earlier quoted context omitted.
But that‘s not an issue with the coding agent. It’s the model that doesn’t follow the instructions. Given how an LLM works, you can never be sure it will always work. LLMs are not deterministic.
Isn't a harness supposed to guide and steer yhe coding agent?
The rules that are fed into the AI are not unbreakable laws to the AI. We should always remember that.
Sandboxing should be the default. Rather than routinely allowing unsandboxed access, one should be able to configure the sandbox to allow exactly what is needed
That's hard. For example, I've been unable to give wayland access to agents inside the sandbox (there's a special flag in bubblewrap to mount /dev/dri in a way you can make use of it, but you also must give access to the wayland socket, and maybe other things). So I think that maybe harnesses should invest in more sandboxing resources
I understand the need for memory footprint in some situations, but what's the point of seeking performance for a software that mostly calls LLMs and waits?
Before I tried coding agents my guess would have been: none. But seeing how slow claude code and copilot cli are and how much ram they use I'm flabbergasted. If you have long running sessions they can both take tens pf gigabytes of ram and feel quite sluggish.
Earlier quoted context omitted.
Compared to what LLMs actually consume, your agent makes zero difference
Why would anyone compare a cloud LLMs power usage when one doesn't pay for it? Local power consumption is important for those.
The key thing with pi is that it can extend itself. How does that work when it’s written in rust?