Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Openrouter Fusion API

openrouter.ai

Post reply on HN
Print view View original

31–40 of 98 posts

Re: Openrouter Fusion API

#31

Post by rusk »

I have an old, slow GPU setup that has nearly 100gb of VRAM

I had been trying to fill this up with big models but it doesn’t seem like these give a good return per Gb

I’m looking at that and wondering would I be better off running multiple such models in parallel. It would probably be a better way to load balance across SLI.

My guess is the scaling will be more “mythical man month” than “no more free lunch” - the interaction of models resembling social dynamics moreso than multi-core setups.

Given that these actors are largely homogenous in culture and incentivising, and coordination overhead is drastically reduced.

Commonly we consider optimal team size to be between 3 and 7 and Brookes’ maximum team size is around 10 or so before the system fails. It should be possible to blow way past those numbers and still experience increased gains in productivity as long as you can keep all your instances stoked.

Re: Openrouter Fusion API

#32

Post by jimmypk »

michaelbuckbee wrote: post #11

I ran a quick eval to see what this looks like qualitatively vs just calling Opus 4.7 or GPT 5.5 directly. As expected, Fusion was 7x slower and 4x the cost. This isn't a knock against it, just that it I think this places Fusion into a "use it only when you need it" category. https://3fpi5avcqq.evvl.io/

[flagged]

Re: Openrouter Fusion API

#35

Post by WhitneyLand »

dsl wrote: post #16

Heh. I built "Fusion" a few months ago as an MCP using OpenRouter. The idea was to give Claude a "panel of experts" to go talk to when it got stuck. After extensive testing and benchmarking I discovered that when you ask one model to judge another's response you don't actually get a better answer. You are just asking it "how closely does this resemble the answer you would have given me." Additional rounds and all the…

Yeah, same experience. It turned out that objectively better answers were not that easy to find plus the expense plus it’s slow.

Re: Openrouter Fusion API

#36

Post by SteveMorin »

Spent the weekend inspired by the new openrouter fusion model and wanted to see if it could run in Claude Code and if I could make it very easy for everyone else to try.

Built - claude-fusion-launcher — run Claude Code on a panel of models, not just one

Also shows cost

https://github.com/smorinlabs/claude-fusion-launcher

Re: Openrouter Fusion API

#37

Post by CrazyStat »

jiaosdjf wrote: post #28
dsl wrote: post #16

Heh. I built "Fusion" a few months ago as an MCP using OpenRouter. The idea was to give Claude a "panel of experts" to go talk to when it got stuck. After extensive testing and benchmarking I discovered that when you ask one model to judge another's response you don't actually get a better answer. You are just asking it "how closely does this resemble the answer you would have given me." Additional rounds and all the…

But.. but I told the LLM that it is an _expert_, is that worth nothing??

Make sure to remind it to make no mistakes.

Re: Openrouter Fusion API

#38

Post by comboy »

dsl wrote: post #16

Heh. I built "Fusion" a few months ago as an MCP using OpenRouter. The idea was to give Claude a "panel of experts" to go talk to when it got stuck. After extensive testing and benchmarking I discovered that when you ask one model to judge another's response you don't actually get a better answer. You are just asking it "how closely does this resemble the answer you would have given me." Additional rounds and all the…

Prompt matters. Obviously if you want another model opinion you must generate from the scratch using the same prompt and then you can try to synthesize, but working with an existing response can work if desired. I use explicit instructions to find issues with assigned severities and then these are going through the panel of judges, only issues passing certain threshold are fixed in the original response.

I'll share a revelation which vastly improved my results: tell judges to evaluate truth and usefulness/should-be-fixed axis separately. Because inevitably with a prompt that is forcing to find issues you will end up with nitpicks. Plus truth axis allows to better evaluate the issue-finder models for your use case.

That's some part of what happens when I generate explanations like this one: https://hanzirama.com/character/%E6%9D%A5#explain - at this point the site is a small side product of my LLMs-evaluation machinery.

Bonus content for patient readers: if you need top quality you will likely need to pin provider(s) on OR, :exacto is not enough to get good repeatable results especially for open-weights models.

Re: Openrouter Fusion API

#39

Post by jedisct1 »

I got significant improvement on code quality (so much that it has become a no brainer for important tasks such as planning) simply by adding the --self-review flag to swival: https://swival.dev/pages/reviews.html

Two instances of the same model, a producer and a reviewer, and the loops doesn't end until everybody's happy.

Post reply on HN