Beating GPT-5.6 Sol on retrieval with 100x cheaper open models • Hacker News Board

Re: Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

#21

Post by Foobar8568 » Wed, Aug 05, 2026, 8:33 PM UTC

aliljet wrote: post #7

There is a more serious question in here that's not being answered. How effective is the retrieval in finding buried needles in larger and larger haystacks. And there's a correlary question, how effective could you be in finding paired needles in that haystack where you need to hold a needle to unlock finding another needle.

Considering the state of the field ( RAG/retrieval/evaluation) I have 0 trust in it, even more if it's closed source with bullshit claim like that.

Everything is vibe sloped to death, and dead after a few months to a couple of years (and not hard to be 100 cheaper than GPT-5.6 sol ... DS is basically free and I guess already 100 times cheaper or more, and here another slope ).

Re: Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

#22

Post by benjiro29 » Wed, Aug 05, 2026, 8:43 PM UTC

mrinterweb wrote: post #2

There is so much opportunity for purpose built models like this. Ideally a harness should spin up a subagent to offload to targeted models for specific tasks like this. I know this is not a novel idea. Claude code does some of this by handing off the "explore" agent work to haiku. I just love seeing that specialized LLMs are being developed.

> Claude code does some of this by handing off the "explore" agent work to haiku.

That is not handing off to a specialized model, its just handing off to a lighter and interior model (compared to the parent model). That by itself can create issues like the lighter model not capturing all the data that the parent needs.

The idea is that we get specialized models that are better then general purpose models. But its rare for a specialized model to beat a strong general model.

There is a reason why we hear less about this idea of smaller expert models, because large strong models to the tasks just as good.

And if the tasks is repetitive to the point that specialization is useful, you can get into a situation that your better off having a program written for that reputative nature, then delegating to other models. And then have the main strong model, deal with the (semi)cleaned up data.

Re: Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

#23

Post by dev_l1x_be » Wed, Aug 05, 2026, 8:52 PM UTC

I am not sure about GPT-5.6. It usually 10x more verbose for no apparent reason than GPT-5.5. Maybe it is only me.

Re: Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

#24

Post by BikiniPrince » Wed, Aug 05, 2026, 9:05 PM UTC

benjiro29 wrote: post #22

mrinterweb wrote: post #2

There is so much opportunity for purpose built models like this. Ideally a harness should spin up a subagent to offload to targeted models for specific tasks like this. I know this is not a novel idea. Claude code does some of this by handing off the "explore" agent work to haiku. I just love seeing that specialized LLMs are being developed.

> Claude code does some of this by handing off the "explore" agent work to haiku. That is not handing off to a specialized model, its just handing off to a lighter and interior model (compared to the parent model). That by itself can create issues like the lighter model not capturing all the data that the parent needs. The idea is that we get specialized models that are better then general purpose models. But its rar…

You can register models with mcp. I think it’s an expensive solution, but it is available in the framework. I use a light weight bus protocol that lets agents interact and pass short messages with pointers. It’s very efficient.

Re: Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

#25

Post by BikiniPrince » Wed, Aug 05, 2026, 9:09 PM UTC

aliljet wrote: post #7

There is a more serious question in here that's not being answered. How effective is the retrieval in finding buried needles in larger and larger haystacks. And there's a correlary question, how effective could you be in finding paired needles in that haystack where you need to hold a needle to unlock finding another needle.

I use detailed project files. It has data regarding the project and subtasks as well as task status. It doesn’t depend on agent context and it’s managed to keep the agent on track. Feature creep with the new models is a very real issue. Capturing principles and how to reconcile tasks helps too. Even today it brought up a source of truth issue it had detected. There were multiple authorities born out of a patch and it used that principle to highlight and resolve the problem.

Re: Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

#26

Post by oliver236 » Wed, Aug 05, 2026, 9:30 PM UTC

mrinterweb wrote: post #2

There is so much opportunity for purpose built models like this. Ideally a harness should spin up a subagent to offload to targeted models for specific tasks like this. I know this is not a novel idea. Claude code does some of this by handing off the "explore" agent work to haiku. I just love seeing that specialized LLMs are being developed.

this is exactly what leopold talks about in situational awareness

Re: Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

#27

Post by jorl17 » Wed, Aug 05, 2026, 9:31 PM UTC

JCharante wrote: post #10

I have done my own testing and found that smaller models can beat their larger siblings on fact retrieval from documents. I haven’t investigated it in depth with a large enough dataset but my guess is that larger models overthink it while smaller ones just do it. I would like if they compared this with 5.6 Luna instead.

Have been feeling the same. There's a sweet spot that threads the needle between "too dumb to search the right thing / relay the correct results" and "too smart to just stop overthinking and just report the damn thing"

Re: Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

#28

Post by try-working » Wed, Aug 05, 2026, 9:46 PM UTC

mrinterweb wrote: post #2

There is so much opportunity for purpose built models like this. Ideally a harness should spin up a subagent to offload to targeted models for specific tasks like this. I know this is not a novel idea. Claude code does some of this by handing off the "explore" agent work to haiku. I just love seeing that specialized LLMs are being developed.

yes, and this is why we need model routing

Re: Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

#29

Post by try-working » Wed, Aug 05, 2026, 9:47 PM UTC

swiftcoder wrote: post #11

ramon156 wrote: post #5

Bit unrelated, I realized that z.ai gives you access to deepseek 4 flash. It's incredible how well it performs when given a detailed spec. I'm not sure I've seen a model one-shot like that, and I was already impressed by gemma 4's speed and efficiency.

Deepseek flash (especially after the recent update) has to be one of the most slept-on models. Price-performance is ridiculous, and its available on a number of cheap coding subscriptions

Flash is the most used model in the world since last week

Re: Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

#30

Post by nikcub » Wed, Aug 05, 2026, 9:49 PM UTC

mrinterweb wrote: post #2

There is so much opportunity for purpose built models like this. Ideally a harness should spin up a subagent to offload to targeted models for specific tasks like this. I know this is not a novel idea. Claude code does some of this by handing off the "explore" agent work to haiku. I just love seeing that specialized LLMs are being developed.

There has been an over-obsession with frontier models and benchmarks. Most of the work will be done by task specific models. You don't put Phds on the factory floor.