Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

github.com

Post reply on HN
Print view View original

171–180 of 292 posts

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#171

Post by zambelli »

digitaltrees wrote: post #170
zambelli wrote: post #161

Earlier quoted context omitted.

I think you have me confused with someone else. I haven't worked on any AI watermark removal project. What project are you referring to specifically?

You’re right. I followed another hacker news thread to the git repo of the watermark removal project and saw your name and seem to have wrongly connected you as an author of both projects. Seriously awesome concept to what you did build I will test it out. If you’re interested, I have sponsored research on AI reliability with Duke University (my graduate Alma mater) and there is an active research project this might…

Oh neat! I'm a little slammed the next couple of weeks with an existing engagement + CAIS, but happy to connect and see if timelines work out?

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#172

Post by digitaltrees »

zambelli wrote: post #171
digitaltrees wrote: post #170

Earlier quoted context omitted.

You’re right. I followed another hacker news thread to the git repo of the watermark removal project and saw your name and seem to have wrongly connected you as an author of both projects. Seriously awesome concept to what you did build I will test it out. If you’re interested, I have sponsored research on AI reliability with Duke University (my graduate Alma mater) and there is an active research project this might…

Oh neat! I'm a little slammed the next couple of weeks with an existing engagement + CAIS, but happy to connect and see if timelines work out?

https://calendly.com/ryanwmartin/open-office-hours

No matter what, keep up the good work. ;)

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#174

Post by blurbleblurble »

zambelli wrote: post #152
blurbleblurble wrote: post #136

Deja vu from the other week https://news.ycombinator.com/item?id=48051562

I think I'm aligned with the idea that some parts of some workflows are mandatory - auth, read before edit, etc. But otherwise, forge really doesn't own or opine much of the workflow. Step enforcement exists if you want it, so do prerequisites, but the idea is that those could be conditional or optional (you may never need to edit a file). The guardrails are designed to work for non deterministic flows or determinist…

In this blog post I'm reading their call for "control flow" as a generalization of exactly what your work illustrates so nicely.

The blog post doesn't say to me "we need to start encoding specifically opinionated conditional branching statements that guide the model" rather I'm hearing a call to realize the broader principles of control flow itself relevant for composing programs with LLMs.

I think your work "nudges" us in that direction.

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#176

Post by blurbleblurble »

zambelli wrote: post #139
blurbleblurble wrote: post #132

Earlier quoted context omitted.

Nice explanation, thank you. So basically the kind of thing I'd usually be doing manually with small models, over and over again, you just automate that nudging and off they go. Sometimes LLMs have seemed to me like "computer programs with inertia" and in that frame what your tool does is identify and reduce friction at key points so the wheels can keep spinning.

Yep! The big frontier models are already quite good at doing that, and they have decent harnesses. That's why Opus on Claude Code does what it does. Small models aren't there yet and they would veer off course, this just nudges them back onto the road. Whether or not they have a good sense of direction is a different question.

Really nice intuition, thank you.

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#177

Post by c7b »

Thanks for building what I'd hoped to find the time to build (and much better than what I would have made)! One question: do you think there is room for parallelization here, eg in the retry loop? Local models generally can handle a limited number (~ 2 digits) of concurrent requests pretty well, even on consumer hardware, which can give >10x boosts in the effective number of token/s. I've been thinking for a while about workflows that could take advantage of this, and 'fix this error' could be one (if not ideal) application. Would be curious what you think.

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#178

Post by npodbielski »

lwansbrough wrote: post #96

Had a couple thoughts in this realm, and am working them into my own harness. Curious to see what others think. I'm not sure if this is generalizable, as my harness is fairly specialized: - Breaking down a problem into a planned execution, with executing agent providing the initial plan which includes explicit objectives such as which tools it calls and what it would consider to be a successful execution. - The harne…

Yes, I was thinking about the same approach because I have Strix Halo and it slows down with longer context so context with less than 50tk/s that would be huge.

Unfortunately I am caught up right now in other projects at work and otherwise and just tried few dozens of prompts to see if this is even achievable.

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#179

Post by monster_truck »

Hey this genuinely _fucks_, you're a legend. You can even get stupid good results from the 1 bit bonsai models! Plays v nice with lmstudio

It's now completely reasonable to throw a 7900XTX in a spare rig, put it in the basement, give it an absurd goal, and forget about it.

Post reply on HN