141–150 of 334 posts
Hmm, Claude Code and Opencode work fine for me. It's a bit amusing that coding agents rely on drawing 1000W+ and using 2TB+ of memory in a datacenter to run, yet people really focus on the last few watts and few hundred megabytes of memory on their laptop (which get dwarfed by the energy cost of compiling their code anyways). But I suppose making them a bit faster and lighter wouldn't hurt.
Earlier quoted context omitted.
It has nothing to do with local RAM usage. But a million tokens of LLM context is decidedly not 5mb. The rough estimate is 2 * L * H_kv * D * bytes per element Where: * L = number of layers * H_kv = # of KV heads * D = head dimension * factor of 2 = keys + values The dominant factor here is typically 2 * H_kv * D since it’s usually at least 2048 bytes. Per token. For Llama3 7B youre looking at 128gib if you’re contex…
Pretty sure we're talking about the output text, not the tensors.
Maybe a workaround could be to use bubblewrap of the scripts ther recursively call the llm (and run the agent in yolo inside the wrap).
I had Claude Code build me one of these as well, though I added Dirac's line hashing for edits etc. Also used Rust, and I had this idea that I should use plugins so it can self-edit by implementing in hooks but in the end, I just have it create exhaust information about improvements into a separate file and just update the source code and recompile. The source code is in a fixed place so it can just rewrite and build…
Some interesting features I add on top of being lightweight are the prompts library, Git worktrees integration and Ralph Wiggum loops integrations.
Earlier quoted context omitted.
https://www.jetbrains.com/junie/
Junie does not support BYOK inside the IDE
> Use a JetBrains AI subscription or connect your preferred provider with Bring Your Own Key (BYOK).
New to this. but whats the benefit over models like Claude code ?
Avoid lock in to stack from one provider (things like a harness that only works with models from one provider and so on).
Use local models (a couple of them do work a bit now, if you have 20Gb video RAM), which saves money and is more private, and works offline.
Can improve the harness, fix bugs in it, make it compatible with different systems and techniques.
This game happens every time in new cycles of developer technology. The good bet historically has always been to use open source - there's a reason most developer tooling just pre-AI revolution was open source (even things like Java and .NET which used to be proprietary).
New to this. but whats the benefit over models like Claude code ?
We don’t trust llm execution- so we add user approvals. But task decomposition calls for co-recursion between code and prompts. This means that the approvals should be evocable at any depth. I think we need some kind of protocol for that (à la the Cubes OS protocols for cut and paste between vms). Maybe a workaround could be to use bubblewrap of the scripts ther recursively call the llm (and run the agent in yolo ins…