Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Prime Agent: A self-improving RLM agent

primeintellect.ai

Post reply on HN
Print view View original

71–80 of 81 posts

Re: Prime Agent: A self-improving RLM agent

#71

Post by efskap »

tosh wrote: post #35
embedding-shape wrote: post #6

LLM-generated code that seemingly went without much review or design is always such an interesting dive into just how bloated you can make code. In this repository, multiple files are close to 10K LOC, one file contains a switch statement that has so many case statements it spans more than 1000 lines, and lots of other fun stuff. I guess it depends on the model you're trying to use, but seems most of them prefer smal…

here is a take on a smol agent ("smol") - 21 lines of Go - no 3rd party dependencies https://github.com/smol-env/smol easier to add and customize stuff when you start from a small base think of it as your starter dough

I dig the spirit, but if I'm meant to understand it in an afternoon, why is it codegolfed to such an extent? It's like reading the output of a minifier, not idiomatic Go.

Re: Prime Agent: A self-improving RLM agent

#72

Post by kooi »

nullbio wrote: post #40
embedding-shape wrote: post #6

LLM-generated code that seemingly went without much review or design is always such an interesting dive into just how bloated you can make code. In this repository, multiple files are close to 10K LOC, one file contains a switch statement that has so many case statements it spans more than 1000 lines, and lots of other fun stuff. I guess it depends on the model you're trying to use, but seems most of them prefer smal…

How on Earth do we solve this bloat and death-by-a-thousand-cuts issue with frontier LLMs? Are there any actual solutions or attempts at solutions to this problem that I can try? Any tools or frameworks? I've tried re-architecture skills, dedicated cleanup sessions, and a bunch of other stuff, but nothing really works well.

Add Sentrux to the LLM loop. https://github.com/sentrux/sentrux

It's an offline coding grader. It works well: You ask for a new module, LLM starts spitting out bloated crap, the code score goes down. LLM keep looping until code score is back up.

Not a substitute for human code review, but keeps things within tight guard rails.

Re: Prime Agent: A self-improving RLM agent

#73

Post by KingMob »

kooi wrote: post #72
nullbio wrote: post #40

Earlier quoted context omitted.

How on Earth do we solve this bloat and death-by-a-thousand-cuts issue with frontier LLMs? Are there any actual solutions or attempts at solutions to this problem that I can try? Any tools or frameworks? I've tried re-architecture skills, dedicated cleanup sessions, and a bunch of other stuff, but nothing really works well.

Add Sentrux to the LLM loop. https://github.com/sentrux/sentrux It's an offline coding grader. It works well: You ask for a new module, LLM starts spitting out bloated crap, the code score goes down. LLM keep looping until code score is back up. Not a substitute for human code review, but keeps things within tight guard rails.

Are you the author? The font size and contrast is kind of poor.

Re: Prime Agent: A self-improving RLM agent

#74

Post by tosh »

evanjrowley wrote: post #68
vivzkestrel wrote: post #49

Earlier quoted context omitted.

got one in typescript? or node.js with javascript?

The stuff 99% of agents are coded in? Please tell me this was sarcasm.

i'm also not a fan of javascript and typescript (let alone the ecosystem they come with)

but the runtimes are everywhere

and a smol agent /w no 3rd party dependencies is probably a good option to have in a pinch

Re: Prime Agent: A self-improving RLM agent

#75

Post by nullbio »

kooi wrote: post #72
nullbio wrote: post #40

Earlier quoted context omitted.

How on Earth do we solve this bloat and death-by-a-thousand-cuts issue with frontier LLMs? Are there any actual solutions or attempts at solutions to this problem that I can try? Any tools or frameworks? I've tried re-architecture skills, dedicated cleanup sessions, and a bunch of other stuff, but nothing really works well.

Add Sentrux to the LLM loop. https://github.com/sentrux/sentrux It's an offline coding grader. It works well: You ask for a new module, LLM starts spitting out bloated crap, the code score goes down. LLM keep looping until code score is back up. Not a substitute for human code review, but keeps things within tight guard rails.

Have you done any benchmarks on this approach?

I wonder if it just ends up in reward-hacking, or if it actually substantially changes the quality of the code.

Are these static analysis metrics enough on their own to cause a substantial improvement?

I tried setting some constraints as part of my system instruction prompt to prevent God file creation, and it did end up breaking files up, but it would often just create a new type of mess and do it needlessly.

Without the model doing additional reasoning and search for alternative approaches that were better fitted on top as it was doing this file splitting, it didn't actually seem to improve quality at all.

So I'm curious to know if this would just result in more of that, or if it would actually result in different choices. Because there's more to code maintenance than just splitting things up into modules, etc. Like the main part of it that I suspect will not be captured is the part of thinking about the big picture, brainstorming multiple architectural approaches, and reasoning through the best one.

Re: Prime Agent: A self-improving RLM agent

#76

Post by IanCal »

Jordan-117 wrote: post #55
embedding-shape wrote: post #52

Earlier quoted context omitted.

> the underage incest [...] Especially weird since the latter is portrayed like some kind of innocent ideal. (Potential spoilers to the story ahead) Are you talking specifically about the "re-population" stuff from the ending? It was many years I last read it, but I seem to recall that there was hesitation and stuff involved, together with "This is literally the only way to re-populate since we're just two people" ba…

Yes, the final chapter. Aside from it being biologically impossible to repopulate from a single couple, it was not really portrayed ambiguously. I don't care to reread it, but from what I remember, the main protagonist was the driving force behind it, grooms her own (single-digit-age?) daughter into the idea, who then sleeps with her own father in graphic detail. He is a little hesitant at first, but relents almost i…

— Spoilers, if you’ve not read this and yet are still reading this comment chain

I guess the grooming may start earlier, it’s not really discussed and it’s a bit ambiguous as to when that started

> but relents almost immediately,

Physically, at the time, though this is something they’ve argued about for 6 years.

It’s not a passage I particularly care for, and it could have been less explicit but then I’m mixed on how that would change the story. I’ve just finished it and it’s certainly an interesting sci-fi story.

Does it provide a strange contrast to the other things?

There’s disgusting horrors inflicted by terrible people but to the willing (but are they only willing due to what was done to them before, by PI?).

Deliberate and horrible suffering of a single person caused for minor gain.

There’s calm destruction of trillions, unfeeling, for protection out of a measure of harms and trying to protect one set. There’s the good goal or perhaps just self indulgent goal that led to that too.

There’s the deliberate killing of trillions with glee at breaking things for one persons view of humanity. There’s the deliberate doing of this by the original catalyst, but with less clear direction. Perhaps a lack of logic and more boredom?

And yet this, done for improving survival chances of a group, feels over the line. I don’t disagree that it is quite disgusting but I do find it interesting at that being my reaction given what else has been done up to this point.

Re: Prime Agent: A self-improving RLM agent

#79

Post by kooi »

nullbio wrote: post #75
kooi wrote: post #72

Earlier quoted context omitted.

Add Sentrux to the LLM loop. https://github.com/sentrux/sentrux It's an offline coding grader. It works well: You ask for a new module, LLM starts spitting out bloated crap, the code score goes down. LLM keep looping until code score is back up. Not a substitute for human code review, but keeps things within tight guard rails.

Have you done any benchmarks on this approach? I wonder if it just ends up in reward-hacking, or if it actually substantially changes the quality of the code. Are these static analysis metrics enough on their own to cause a substantial improvement? I tried setting some constraints as part of my system instruction prompt to prevent God file creation, and it did end up breaking files up, but it would often just create…

Only anecdotal benchmarks for what ever that is worth.

I notice the LLM reasoning things like: "The maintainability score went down, let me look.... I see I've duplicated existing code which already exists in this module...I see this logic could be consolidated".

I have noticed file splitting is a strategy to improve the "score".

In my experience, I've been doing more the architectural guidance. There is also a rules engine which I have not used, but looks pretty interesting:

``` [constraints] max_cycles = 0 max_coupling = "B" max_cc = 25 no_god_files = true

[[layers]] name = "core" paths = ["src/core/"] order = 0

[[layers]] name = "app" paths = ["src/app/"] order = 2

[[boundaries]] from = "src/app/" to = "src/core/internal/" reason = "App must not depend on core internals" ```

Re: Prime Agent: A self-improving RLM agent

#80

Post by UncleOxidant »

mromanuk wrote: post #66
znnajdla wrote: post #60

Very interesting idea but without any concrete examples of performance on real tasks its just a pretty idea

They achieved 95% in ARC-AGI-3. [0] 0: https://x.com/PrimeIntellect/status/2085087000764568010

With Opus 5 as the underlying model.
Post reply on HN