Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

github.com

Post reply on HN
Print view View original

251–260 of 292 posts

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#251

Post by zambelli »

mrothroc wrote: post #236
simonw wrote: post #92

This is a neat project, but the description made me realize that I don't actually know what the term "guardrails" means. ... which lead me to realize that it's one of those terms with multiple meanings - like "agent" or even "AI" itself - but where people who use it may not be aware of how many different definitions are floating around. In this project it refers to validating tool calls - fixing invalid tool response…

Yes, "guardrails" is a squishy term. But it gets clearer if you ask what transition is being guarded. Some of this is inside the model, like topic refusals. Forge sits at the tool call level. My personal workflow uses guardrails at the SDLC level: I have a standard pipeline (plan, design, code, build, test). I use gates between each stage, and the right composition leads to a much higher quality in the final product.…

Nice symmetry with tool call failures being sent to LLM that made the call without bugging the user. The artifact-generating entity gets the error back, effectively.

100% correct, and stackable. Could have topic refusal in LLM training itself, forge in tool call alter, and sdlc gates at the workflow level.

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#252

Post by DeathArrow »

zambelli wrote: post #141
DeathArrow wrote: post #135

I'm curious if in proxy mode it works also with remote models or only with local models. Also, did someone tried it with local Qwen 3.6?

I believe there's a comment below mentioning "qwen" but not a specific version number - if you're looking for 3rd party validation. I've personally tried qwen3.6-35b-a3b, qwen3.5-35b-a3b, and qwen3.5-27b with forge (agentic coding harness built on forge workflowrunner) and it works great. Official forge eval benchmarks for that class of models is still a couple of weeks out. Proxy mode should work fine with remote mo…

Thank you for taking the time to reply to so many questions. I am really excited about this and for me and my usage seems to be one of the most important breakthroughs in AI in the last year because it makes the models better.

It would be nice if you can continue working and improving the tool and I hope other people will jump to help.

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#253

Post by DeathArrow »

zambelli wrote: post #210
monster_truck wrote: post #179

Hey this genuinely _fucks_, you're a legend. You can even get stupid good results from the 1 bit bonsai models! Plays v nice with lmstudio It's now completely reasonable to throw a 7900XTX in a spare rig, put it in the basement, give it an absurd goal, and forget about it.

Thanks! Did you try it with lmstudio? I actually never tried it with that. Only published ollama, llamfile, llama.cpp native/prompt - and unofficially tested vLLM, but never lmstudio.

Maybe you can update the guide and tell us how to use vLLM with Forge when you find some time?

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#254

Post by zambelli »

DeathArrow wrote: post #253
zambelli wrote: post #210

Earlier quoted context omitted.

Thanks! Did you try it with lmstudio? I actually never tried it with that. Only published ollama, llamfile, llama.cpp native/prompt - and unofficially tested vLLM, but never lmstudio.

Maybe you can update the guide and tell us how to use vLLM with Forge when you find some time?

[dead]

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#256

Post by zambelli »

DeathArrow wrote: post #255
somethingsome wrote: post #244

I was hoping it would work with vLLM (openai compatible) to test it, does anyone know a similar proxy for local coding models?

Check this: https://github.com/antoinezambelli/forge/tree/az/vllm

Yeah I got it working as a quick test run to confirm a model issue vs backend issue on a consumer app. It worked on my dual-5070 Ti rig, but I didn't have time to formalize all the way and merge it in. Thanks for linking it!

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#258

Post by somethingsome »

zambelli wrote: post #256
DeathArrow wrote: post #255

Earlier quoted context omitted.

Check this: https://github.com/antoinezambelli/forge/tree/az/vllm

Yeah I got it working as a quick test run to confirm a model issue vs backend issue on a consumer app. It worked on my dual-5070 Ti rig, but I didn't have time to formalize all the way and merge it in. Thanks for linking it!

Thanks, I just tried, for me it worked on 2x L40S with vLLM. I had some issues due to the model name, forge was forwarding 'default' instead of the real model name 'Qwen2.5-Coder-14B-Instruct'.

If someone else struggle on this step, I added in vLLM args: --served-model-name "Qwen2.5-Coder-14B-Instruct" --served-model-name "default"

So default becomes an alias.

I didn't yet test Forge, I was just happy that it worked at the moment ;)

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#259

Post by zambelli »

somethingsome wrote: post #258
zambelli wrote: post #256

Earlier quoted context omitted.

Yeah I got it working as a quick test run to confirm a model issue vs backend issue on a consumer app. It worked on my dual-5070 Ti rig, but I didn't have time to formalize all the way and merge it in. Thanks for linking it!

Thanks, I just tried, for me it worked on 2x L40S with vLLM. I had some issues due to the model name, forge was forwarding 'default' instead of the real model name 'Qwen2.5-Coder-14B-Instruct'. If someone else struggle on this step, I added in vLLM args: --served-model-name "Qwen2.5-Coder-14B-Instruct" --served-model-name "default" So default becomes an alias. I didn't yet test Forge, I was just happy that it worked…

Oh that's a good find, I'll book ark this for a GitHub issue.

Glad to hear it's working!

Re: Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks

#260

Post by tempoponet »

zambelli wrote: post #65
tempoponet wrote: post #60

Why this entire tool chain instead of building within something like pi code? I've been exploring this area and a project like https://github.com/itayinbarr/little-coder (not my work) lets me mix and match with my current setup or any plugins built for pi.

Mainly because I have plenty of use cases and not all of them need or want pi. Forge isn't an orchestration framework and is not coding specific, it lives one level lower - if I understand pi correctly. The proxy mode should integrate seamlessly, and the middleware guardrail mode could be lifted into pi. As for little coder, I love it! I wanted forge to be more generic than just agentic coding as there's many more ag…

Thank you for the thoughtful reply. I have some smaller 3080's I'm looking to place and this sounds like a good opportunity.
Post reply on HN