Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

fermisense.com

Post reply on HN
Print view View original

121–130 of 141 posts

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#121

Post by fc417fc802 »

billmalarky wrote: post #114
cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

>As does the economic picture that justified the massive infrastructure building that’s now broadly funded by a complex network of debt. I have a genuine question and I'd like to hear people's good faith thoughts on this. There's a fair case that open models are threatening to institutions who spent a lot on training proprietary SOTA models. But to my understanding, the massive investment spend (much of it debt backe…

Maybe. But you have to consider any difference in expected returns. Also account for expected efficiency and capability advances of new hardware. And perhaps most importantly, as of a few years ago the failure rate of these GPUs when run hard was in the ballpark of 10% per year (that might or might not change but regardless it definitely needs to be accounted for).

It seems exceedingly unlikely that we will see growth sufficient to justify the level of investment. But as you say that doesn't necessarily mean a worst case scenario either. An investment can be bad without being ruinous after all.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#122

Post by klibertp »

amelius wrote: post #90
Aurornis wrote: post #85

Earlier quoted context omitted.

> The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most generic use cases do benefit from a model that has been trained broadly. When you don’t know the specific use case ahead of time, you have to have world knowledge ready to go. Even when coding it’s helpful to have all that knowledge on tap so the model ca…

Can't you take a broadly trained model and throw away useless knowledge (like foreign languages, and 99% of wikipedia) and still end up with a model that is as intelligent and useful as the original but much smaller?

I suspect it's a bit like tree-shaking/dead-code elimination in an untyped, dynamic language. You can probably delete a lot of things, but each deletion has a chance to break (or silently degrade) some functionality. Good coding practices, good docs and comments, and good tests can help determine whether a given definition is safe to delete or quickly verify that the deletion was a mistake after the fact. LLMs' weights lack all of those. It's like trying to perform DCE on a binary linked from outputs of ten different compilers with different optimization levels (and the linker is outright bonkers) without running the binary once. Oh, and the binary is many tens of GB in size. Yikes, I'm getting goosebumps just imagining it!

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#123

Post by esafak »

Aurornis wrote: post #85
cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

> The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most generic use cases do benefit from a model that has been trained broadly. When you don’t know the specific use case ahead of time, you have to have world knowledge ready to go. Even when coding it’s helpful to have all that knowledge on tap so the model ca…

Furthermore, shifting the complexity to the model simplifies engineering. A lot of companies made this trade, replacing their bespoke ML models with foundation models.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#124

Post by mandeepj »

cmiles8 wrote: post #13

The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages. Most use cases are defined within constraints where costs matter a lot. As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does…

> The point that the major labs don’t seem to get is that the vast majority of use cases simply don’t need models that have 50 PhDs and can speak 12 languages.

I think that's true across the board. Along those lines, most companies all of a sudden started looking for AI researchers, instead of software engineers, which is what they still need :-).

Just like a decade-ish ago, when the FAANG interview style got openly known, then everybody started to conduct interviews in that pattern :-)

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#125

Post by geysersam »

layer8 wrote: post #75
geysersam wrote: post #52

Earlier quoted context omitted.

They do tend to overengineer. The other day 5.6 Sol generated a while loop around a uuid4 call to make sure the generated ids were unique... I wonder if that's also how I'd behave if I had a reinforcement learning harness around me that dived hard on and punished me for every small mistake.

> They do tend to overengineer. The other day 5.6 Sol generated a while loop around a uuid4 call to make sure the generated ids were unique... That’s not overengineering, it’s plain nonsensical, because presumably it doesn’t compare it to all IDs generated in the past. Which, if you wanted to do that, you’d use a database with a uniqueness constraint, in case you don’t already have that anyway. Fixing this lack of re…

The ids were used in a limited context and it compared to all the other ids generated in the same context. Yes, absolutely nonsensical.

I find it interesting that presumably the LLM was trained on millions of legit examples where people generate uuids, but still it decided to do something different and unexpected, because it "reasoned" itself into thinking it was necessary.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#126

Post by vkaku »

teodosin wrote: post #76
vkaku wrote: post #71

Earlier quoted context omitted.

Very true. In fact, there's a lot more under the surface that will come to light once more efficient tiny models come out. P.S. Building something that proves that you don't need those many params even. https://github.com/guilt/tinytot

The premise of your project seems compelling. Is it novel or building on existing work? Anything one could read or watch to get introduced to that area of research?

You would already be able to see RSI and loops being a thing.

So there's going to be more and more research coming out telling you what size of models are really needed to solve a specific task. When you combine that with RSI, the next frontier would be to get this model as close to the real world with instant data and instant reasoning, no fine tuning at all. And see how this one could iterate itself.

I'll also be trying out more tiny models and seeing if they can call each other, and be as effective; You will also be seeing me trying this, in addition to what I'd already mentioned in the other reply.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#127

Post by vkaku »

dineshdb wrote: post #87
teodosin wrote: post #76

Earlier quoted context omitted.

The premise of your project seems compelling. Is it novel or building on existing work? Anything one could read or watch to get introduced to that area of research?

I took a look at the codebase and found that this is just pattern matching. There is no any novelty here. It looks good for knowledge-base retrieval system, but other than that, this is just nothing more than a pattern-matcher. Would love to be proven wrong with future updates.

Sure. Like throwing money at problems is novel.

Why don't you go and actually build something novel? People need to understand better how current models work, and if retrieval indeed solves 99% of the problems people care about, then those other models aren't doing anything super novel either.

I'm not arguing here: The code is out there, data is out there. Do better if you know something actually better. Thanks!

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#128

Post by vkaku »

teodosin wrote: post #76
vkaku wrote: post #71

Earlier quoted context omitted.

Very true. In fact, there's a lot more under the surface that will come to light once more efficient tiny models come out. P.S. Building something that proves that you don't need those many params even. https://github.com/guilt/tinytot

The premise of your project seems compelling. Is it novel or building on existing work? Anything one could read or watch to get introduced to that area of research?

I did some research about scaling laws and what's unsolved there.

This project was built on structural sparsity, do away with any requirements to train models on largely outdated corpus of data, and only train it on reasoning logic. Because if models can use tools well and retrieve everything else, it greatly would reduce the need for large language models.

Further, when generating traces, I ran a benchmark that showed that on grounded data, n-grams and simple retrieval outperformed transformers on efficiency (See https://github.com/guilt/Transformers) and wanted to try scaling this up to much larger traces.

It worked quite well. Next improvement would be to run this with a BERT model and add more do-not-know traces and the Claude style neurotic self-doubt/self-verify traces and test it out a bit more.

If the entropy required to route people to the relevant answers is very low, full dense attention is a waste of compute. That's the theory being pressure tested here. The more I do this exercise, the more I am getting convinced that smaller models are good enough.

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#129

Post by bugglebeetle »

synergy20 wrote: post #113
bugglebeetle wrote: post #34

Earlier quoted context omitted.

Tinker is pretty cheap. Prime Intellect if you want more flexibility.

unsloth for small use cases, tinker for enterprise level use cases?

I haven’t used their platform so can’t speak to this, but super brilliant team and big fans of their work overall!

Re: A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

#130

Post by Chipshuffle »

How difficult is it to post train a model like this. I'm still imagining it being really complicated for someone that has no expert knowledge in this field. Or are there already specialized materials or offers for this that make it easy?
Post reply on HN