221–230 of 334 posts
I understand the need for memory footprint in some situations, but what's the point of seeking performance for a software that mostly calls LLMs and waits?
At first I also thought that it would be just call and wait, but a lot of work is done locally (any tool calls).
nobody actually cares about rust, let alone likes it
Earlier quoted context omitted.
Before I tried coding agents my guess would have been: none. But seeing how slow claude code and copilot cli are and how much ram they use I'm flabbergasted. If you have long running sessions they can both take tens pf gigabytes of ram and feel quite sluggish.
huh. my evidence with codex hasn’t been so bad. and tbh why would i discourage anyone from coding. hack away mr hacker. your solution will either sink or swim
Earlier quoted context omitted.
- Reduce the footprint on the planet - prolonged life of hardware - less electricity - less expensive hardware
Compared to what LLMs actually consume, your agent makes zero difference
It says inspired by Pi, but I don't see any extension/plugin possibilities. The best feature of Pi is that an extension can hook anywhere and completely change the behavior. It also allows two extensions to stack on the same hook where there are no conflicts. I believe Pi extensibility is the most important feature, exactly as how it was important for WordPress. WordPress won because anyone could install it and add t…
Earlier quoted context omitted.
No because it means people will use Rust for the wrong reasons. Systems programming is only a tiny fraction of code out there. Approaching every problem as a systems programming problem is a massive waste of resources and intellect.
For small to medium projects, an LLM can write functional (if not well crafted) Rust. Considering how easy this is now, why choose a heavier, slower and less typesafe language?
Earlier quoted context omitted.
I see spreading Rust as an overall good thing, because it changes benchmark on how software should feel in terms of performance, stability, memory footprint. So even if it doesn't create tangible advantage in a particular use case - its still good for the whole industry.
I haven't used Rust extensively but my feeling is, if you change the design (which inevitably happens in many early stage projects), the refactoring takes more time due to borrow-checker semantics. Although I am far from a representative sample and could well have been using it wrong
Earlier quoted context omitted.
How is it any faster than something written in say, Java?
latency and throughput (when with Java the system is crying for more memory while it's chilling in the Rust case)
As for throughput: you have 1-2 requests going at a time, the next one waiting for the reply. What throughput are we talking about?
That's like speeding to the post office and expecting your letter to get to the recipient faster.
Earlier quoted context omitted.
I actually though about this issue, but while Pi can have this script-like environment thanks to the fact that it's based on an interpreted language (TypeScript), Rust has its own limitation as a compiled language. I decided to allow for customization in a different way: 1. The prompt library (~/.config/hypernova/prompts/) acts as a simpler alternative to Skills, with the built-in prompts that should replace superpow…
Ok, what about having tools be discoverable from the environment, similar to how $PATH works in POSIX? There could be an env var $AGENT_TOOLS, a string of paths delimited by `:` and tools would be discovered as some specific format of file. Maybe a JSON that contains tool name, list of parameters and the command to run it. This is essentially decoupling tools from the agent, allowing more customization and per-projec…
The approach mostly being communicating over json rpc which has become the standard for MCP so it makes it more approachable to agent developers.
Obviously its very much NOT mcp, its a low level events based rpc system for registering capabilities and extending low level primitives of the agnet itself not the model
Earlier quoted context omitted.
How is it any faster than something written in say, Java?
latency and throughput (when with Java the system is crying for more memory while it's chilling in the Rust case)
Next to none of them will get you nearly as good cold start times as of native app, if using free java.
There was GraalVM and its ecosystem which included Java Native Image - first thing I’d evaluate if thought about non-server side, performant Java application.
But it all had been sadly swept away by Oracle from free tier.