Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My finetuned models beat OpenAI's GPT-4

mlops.systems

Post reply on HN
Print view View original

71–80 of 98 posts

Re: My finetuned models beat OpenAI's GPT-4

#71

Post by gillesjacobs »

renegade-otter wrote: post #57
gillesjacobs wrote: post #3

This is entirely unsurprising and in-line with the finding that even small specialized models do better in information extraction and text classification. So no wonder finetuned large LMs do good too. Personally, my PhD did fine grained ACE-like event and sentiment extraction and "small" specialized finetuned transformers outperformed prompting LLMs like BERT and Roberta-large. Would love to see an inclusion of small…

The caveat here is that if you don't know how to create good specialized models - you are just wasting everyone't time and money: https://www.threads.net/@ethan_mollick/post/C46AfItO8RS?hl=e...

Exactly, BloombergGPT performed worse on financial sentiment analysis then much smaller fine-tuned Bert-based models.

For many extractive tasks BloombergGPT was quite disappointing. A 5-10% performance hit with much larger inference cost compared to smaller models is not desirable.

But the research investment for Bloomberg makes sense to take the risk: a do-it-all generative model can mean significant complexity reduction in maintenance and deployment overhead.

It didn't directly pay off for many extractive tasks, but I bet they're iterating. Bloomberg has the data moat and the business needs in their core products to make it worthwhile.

Re: My finetuned models beat OpenAI's GPT-4

#72

Post by yreg »

babelfish wrote: post #70
kcorbitt wrote: post #43

(Disclaimer: I'm the founder of OpenPipe, one of the fine-tuning services OP tried and ultimately the one that produced the highest performing model, it appears.) Data extraction is a use case that fine-tuned models are fantastic at, so I'm not surprised that OP got good results. That said, I've also found it's pretty easy to beat GPT-4 across many task types if you have a way of getting strong training data. We publ…

Is using model responses to train a new model against the ToS for the major LLM providers (OpenAI, Anthropic, etc)?

There doesn't seem to be any restriction like that in OpenAI terms.

Re: My finetuned models beat OpenAI's GPT-4

#73

Post by zepton »

yreg wrote: post #72
babelfish wrote: post #70

Earlier quoted context omitted.

Is using model responses to train a new model against the ToS for the major LLM providers (OpenAI, Anthropic, etc)?

There doesn't seem to be any restriction like that in OpenAI terms.

There is: "you may not... Use Output to develop models that compete with OpenAI"

(from https://openai.com/policies/terms-of-use/)

Re: My finetuned models beat OpenAI's GPT-4

#74

Post by yreg »

zepton wrote: post #73
yreg wrote: post #72

Earlier quoted context omitted.

There doesn't seem to be any restriction like that in OpenAI terms.

There is: "you may not... Use Output to develop models that compete with OpenAI" (from https://openai.com/policies/terms-of-use/ )

Thanks, I've missed that.

I suppose the Output could be washed by publishing it on the web and having another entity crawl it.

OpenAI doesn't treat anyone else's content any differently, acting like it's a fair game, so why should we care.

Re: My finetuned models beat OpenAI's GPT-4

#75

Post by alach11 »

mewpmewp2 wrote: post #44
Tiberium wrote: post #28

Did you release the dataset and the code for testing? It would be interesting to check how 3.5 Sonnet performs on this task.

The dataset is there: https://huggingface.co/datasets/strickvl/isafpressreleases_t... but when looking for rows where GPT-4o was deemed inaccurate then to me it seems the label was wrong or at least it wasn't possible to infer that certain label from the input text. But finetuned model was able to predict it. Which makes me wonder whether the finetuned models are poisoned with eval data... See this one: > ISAF Joint…

Props to the author for releasing the data. My instinct is also to immediately suspect data leakage. It's super easy for this to happen. For example the original dataset could contain multiple articles about the same event.

Re: My finetuned models beat OpenAI's GPT-4

#76

Post by scosman »

geokon wrote: post #31
scosman wrote: post #2

And that’s the point of fine tuning models. Still good to see someone walk through their fine tuning process, with a mix of hosted and local options.

As I understood the point was not that they fine tuned a model and it got better They use a much simpler model, fine tune it, and manage to beat a way more advanced model

That’s still the point. That model now does exactly one thing, and because of that can do better than a model 50x the size that tries to do everything. It will crush it in instruction following and consistency.

A fine tuned 500b parameter model would probably beat the fine tuned 7b model, but only by a bit (depending on task obviously). A lot of that capacity is being used for knowledge, and isn’t needed for extraction/classification tasks. Fine tuning isn’t touching most of those weights. The smaller models need to focus on more general language skills, not answering “describe the evolution of France’s economy in the 1800s”.

Re: My finetuned models beat OpenAI's GPT-4

#77

Post by scosman »

wongarsu wrote: post #40
geokon wrote: post #31

Earlier quoted context omitted.

As I understood the point was not that they fine tuned a model and it got better They use a much simpler model, fine tune it, and manage to beat a way more advanced model

When jumping from 7B parameters to 70B to 400B (or whatever GPT-4 uses) most of the additional neurons seem to go towards a better world model and better reasoning (or whatever you want to call the inference of new information from known information). There doesn't seem to be any major improvements in basic language skills past 7B, and even 1B and 3B models do pretty well on that front. In that sense it's not that su…

Re: 7b models vs gpt-3.5, I’m guessing different fine tuning parameters can account for the difference. The OpenAI fine tuning is a black box.

Re: My finetuned models beat OpenAI's GPT-4

#78

Post by babelfish »

yreg wrote: post #74
zepton wrote: post #73

Earlier quoted context omitted.

There is: "you may not... Use Output to develop models that compete with OpenAI" (from https://openai.com/policies/terms-of-use/ )

Thanks, I've missed that. I suppose the Output could be washed by publishing it on the web and having another entity crawl it. OpenAI doesn't treat anyone else's content any differently, acting like it's a fair game, so why should we care.

It seems like you do not work for OpenPipe (OP), so it probably doesn't matter for you, but it could (should) matter a whole lot for OpenPipe and/or their customers

Re: My finetuned models beat OpenAI's GPT-4

#80

Post by jrm4 »

klabb3 wrote: post #53
jrm4 wrote: post #49

At the risk of sounding like an old head; Seems to me then, priority one should be "free and open source all the models as hard as possible, so that EVERYONE can fine-tune." (This being a subset of the idea of, free / open source is generally preferable for both freedom and quality)

It seems to me this means whoever has hoarded and declared ownership of the most personal data will make the best products. Kinda like how some people liked their targeted ads because they’re more “relevant”, only now it’s not just ads but useful products. Another winner is of course platform owners like Apple and Microsoft who can scrape your data off their apps and products, even locally. This is a much bigger edge…

Your end point I think is exactly right.

I think your first one is getting downvoted hard because your first sentence is not at all how any of this works.

Sucking down personal data isn't JUST a bad idea for privacy, it's actually also bad for "making the best products," I think you're overstating the extent to which all that data that is stolen and sold to the highest bidder actually helps the company buying it?

Post reply on HN