Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

github.com

Post reply on HN
Print view View original

261–270 of 292 posts

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#261

Post by monster_truck »

zambelli wrote: post #210
monster_truck wrote: post #179

Hey this genuinely _fucks_, you're a legend. You can even get stupid good results from the 1 bit bonsai models! Plays v nice with lmstudio It's now completely reasonable to throw a 7900XTX in a spare rig, put it in the basement, give it an absurd goal, and forget about it.

Thanks! Did you try it with lmstudio? I actually never tried it with that. Only published ollama, llamfile, llama.cpp native/prompt - and unofficially tested vLLM, but never lmstudio.

Yessir! Been a longtime fan of it, I've spent too many fuckin years wrangling python, especially pytorch, especially on AMD, dep issues for fun and profit... they don't get enough flowers. It's oai compat, no thorns.

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#264

Post by zambelli »

sfifs wrote: post #227

Very nice! I also saw you have a vllm branch and I validated it works on my system. There is bugfix which sent you a PR for to auto-discover served-model-name which vllm hard validates for.

Merged! Thanks for that catch. I'll try to sequence the in-flight work ASAP to get the vllm branch merged in as a whole.

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#265

Post by JKCalhoun »

1dom wrote: post #235
JKCalhoun wrote: post #207

Earlier quoted context omitted.

"I actively avoid cloud based LLMs… This means I don't really have a good grasp of SOTA LLM performance or accuracy… …but then all of your metrics and data seem to focus 100% on accuracy. You need to address speed." I wonder, if you were to use cloud-based LLMs more often, you might find that accuracy (fidelity?) is indeed more more lacking in your local models. You can always just throw hardware at your speed proble…

I agree accuracy isn't maybe the best word here, I used it as it was used in the original post, mainly a as a catchall for "everything but speed", so fidelity, perplexity, etc. I also agree that if I spent more time using cloud based LLMs, I would very much find local LLMs less capable and useful. Comparison is the thief of joy though, and I'd rather feel blissfully ignorance towards SOTA LLMs rather than a dependenc…

No, I'm a fan of local as well. For me though, there is just such a fascination that I can have something like this sitting on my own hard drive. It's okay that it's not a "frontier model".

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#266

Post by jononor »

killing_time wrote: post #162
zambelli wrote: post #70

Earlier quoted context omitted.

That is the whole challenge, actually! A new metric I'm going to dogfood into forge is ETTWS - estimated time to working solution. A simple retry loop around your whole workflow could, in some cases, be all you need . But it could mean many blind attempts to get through a workflow successfully. And hopefully there isn't a payment step partway through! The fewer hard errors nix the whole workflow, the lower your ETTWS…

Is it strange that I immediately interpreted ETTWS to be Estimated Time To William Shakespeare?

The one true AGI metric!

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#270

Post by mrothroc »

zambelli wrote: post #251
mrothroc wrote: post #236

Earlier quoted context omitted.

Yes, "guardrails" is a squishy term. But it gets clearer if you ask what transition is being guarded. Some of this is inside the model, like topic refusals. Forge sits at the tool call level. My personal workflow uses guardrails at the SDLC level: I have a standard pipeline (plan, design, code, build, test). I use gates between each stage, and the right composition leads to a much higher quality in the final product.…

Nice symmetry with tool call failures being sent to LLM that made the call without bugging the user. The artifact-generating entity gets the error back, effectively. 100% correct, and stackable. Could have topic refusal in LLM training itself, forge in tool call alter, and sdlc gates at the workflow level.

Definitely stacks. The thing that made it clear for me was being explicit about the stages, and where/what you can verify with a guardrail, or gate. I wrote up the framework I use here: https://michael.roth.rocks/research/trust-topology/

Being explicit about the space between the stages is critical, because that's your enforcement point.

Post reply on HN