Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Foreman, a self-hosted LLM gateway for cost aware model routing

github.com

Post reply on HN
Print view View original

11–20 of 20 posts

Re: Show HN: Foreman, a self-hosted LLM gateway for cost aware model routing

#12

Post by AndrewLiu96 »

TurdF3rguson wrote: post #10
midtake wrote: post #3

Name collision with Foreman, the provisioning/patching tool. https://en.wikipedia.org/wiki/Foreman_(software)

Not to mention the George Foreman grill

That was actually one of my naming inspirations, the george foreman grill is a s tier invention

Re: Show HN: Foreman, a self-hosted LLM gateway for cost aware model routing

#16

Post by AndrewLiu96 »

anirudhak47 wrote: post #11

how does it compare to sakana?

I assume you're referencing Sakana Fugu? Similar in the sense that there's multiple models involved, but from what I understand Fugu is kind of a black box in terms of what model is used for what. This project is more deterministic, you can pick and choose which model to assign/use for certain tasks, plus this project is open source which Fugu isn't.

Re: Show HN: Foreman, a self-hosted LLM gateway for cost aware model routing

#17

Post by TurdF3rguson »

AndrewLiu96 wrote: post #12
TurdF3rguson wrote: post #10

Earlier quoted context omitted.

Not to mention the George Foreman grill

That was actually one of my naming inspirations, the george foreman grill is a s tier invention

Lol. It's an objectively bad grill.

Re: Show HN: Foreman, a self-hosted LLM gateway for cost aware model routing

#18

Post by jarodrh »

Interesting project. Most of what I read appears to deal well with a linear agent session.

How does it handle concurrent agent/subagent spawning?

Do they fall under the same sessionId (where they're potentially fighting over the warm route breaking it with every task-type change) or get their own individual sessionIds (where you must rewrite the shared prefix for each agent)?

Re: Show HN: Foreman, a self-hosted LLM gateway for cost aware model routing

#19

Post by dilushin »

AndrewLiu96 wrote: post #14
dilushin wrote: post #9

I have something simmilar brewed in my multi-agent system, showing and saving cost data on every agent flow run.

That's awesome! I'd love to check it out and give it a try

It is in my private repo, let see if I can OSS this part )
Post reply on HN