271–280 of 334 posts
Earlier quoted context omitted.
Because people are crazy, usage of npm for installing binaries is quite common unfortunately.
So …. do I understand it right? openai, one of the hottest companies on the planet right now, with very deep pockets, distribute their official rust cli via the … public npm repo?
There is also homebrew install.
We don’t trust llm execution- so we add user approvals. But task decomposition calls for co-recursion between code and prompts. This means that the approvals should be evocable at any depth. I think we need some kind of protocol for that (à la the Cubes OS protocols for cut and paste between vms). Maybe a workaround could be to use bubblewrap of the scripts ther recursively call the llm (and run the agent in yolo ins…
Earlier quoted context omitted.
"{Feature} is the whole point of {more complex technology}" is an objection that can very often be raised. That doesn't mean that giving up features in exchange for simplicity is always the wrong call. And there's also advantages to having the user drive what instructions go into the prompt instead of the harness/model.
This is tangential to the point. It’s often great to have a simpler version of a solution, even if it eschews some features. But this isn’t that. OP claims that the prompt system is an “alternative” to skills, but it isn’t. It isn’t solving the same problem that skills solve at all. It’s like saying that a bicycle is a simpler alternative to a lawnmower because they both have wheels. Prompts are a feature that are si…
Earlier quoted context omitted.
For small to medium projects, an LLM can write functional (if not well crafted) Rust. Considering how easy this is now, why choose a heavier, slower and less typesafe language?
Could choose a similar weight, similar speed, equal or more typesafe language though :)
Earlier quoted context omitted.
Pretty sure we're talking about the output text, not the tensors.
These LLM replies are really getting annoying.
I guess get better at detecting LLMs instead of accusing everything of being an LLM reply?
Earlier quoted context omitted.
Could choose a similar weight, similar speed, equal or more typesafe language though :)
Ada? Other than c and c++ everything else benchmarks 2-4 times slower than rust for compute bound tasks, even after jit warmup. I'm up for ada though, especially with an llm where I don't have to type all that verbose syntax.
Lots of options with no jit or warmup
I (somewhat jokingly) wrote one recently too... https://github.com/pnegahdar/nano in under 200 lines. Repl, sessions, non-interactive, approvals, etc The smarter the models get the less the harnesses matter (outside of devx). Maybe one day I'll run it through swebech.
Earlier quoted context omitted.
If you cannot write a simple Java agent without consuming so much RAM that your system is swapping then that really says more about the developer than anything. Java is used in plenty of embedded systems and other memory constrained environments. Yes, it’s not going to perform well compared with Rust, but that doesn’t mean it’s an Electron-equivalent bloated clusterfuck of an ecosystem that’s going to eat all your sy…
> so much 1) the agent is probably not the only thing running on the system, so more is just worse generally 2) I am fine if a developer needs Rust or similar to write a resource efficient app. I wonder what the developer could achieve when he put the optimization effort into the Rust app instead.
People are so narrow minded about programming on this forum. They talk as if only Rust fills the void between unsafe C and node.js behemoths. But the reality is there are a plethora of other good languages out there too.
I understand the need for memory footprint in some situations, but what's the point of seeking performance for a software that mostly calls LLMs and waits?
e.g. opencode right now uses ~80% of my CPU. At first I also thought that it would be just call and wait, but a lot of work is done locally (any tool calls).
And in my experience is not that much faster to start than more complex software like Visual Studio Code.
Earlier quoted context omitted.
What's the latency difference between a long running process issuing a network call in Java vs rust? This is such a short time that it is completely overshadowed by noise (OS doing something else, what other software is running etc) As for throughput: you have 1-2 requests going at a time, the next one waiting for the reply. What throughput are we talking about? That's like speeding to the post office and expecting y…
you seem to specifically aim at the current example, but mine wasn't Anyways, consider how higher memory usage can affect the systems performance dramatically once the system needs to start swapping memory to disk signficantly