Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Finetune GPT-Like Large Language Models on a Custom Dataset

lightning.ai

Post reply on HN
Print view View original

111–120 of 126 posts

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#111

Post by potatoman22 »

pid-1 wrote: post #26
slenocchio wrote: post #24

Can someone explain why I'd want to use fine-tuning instead of a vector database (or some other way of storing data/context)?

I've been playing with using documents as OpenAI embeddings for the past weeks and, at least for my use case, the results are meh. It seems sometimes just using context is not enough. My next step is to play with fine tunning, but I have no results to report yet.

If the documents are large, try embedding smaller portions. If there's a heavy domain vocabulary, you might need a custom model.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#112

Post by montag »

oddthink wrote: post #42
slenocchio wrote: post #24

Can someone explain why I'd want to use fine-tuning instead of a vector database (or some other way of storing data/context)?

Wouldn't a vector database just get you nearest-neighbors on the embeddings? How would that answer a generative or extractive question? I can see it might get you sentiment, but would it help with "tell me all the places that are mentioned in this review"?

Right. You feed the text chunks (from the matched embeddings) to a generative LLM to do the extractive/summarization part.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#113

Post by winddude »

sp332 wrote: post #58
moffkalast wrote: post #29

Earlier quoted context omitted.

> I can use smthg like GPT-4 to label data and then use that as a train set for my own LLM, right? Yes, almost all improved LLama models are tuned exactly that way (trained on examples of questions and answers from say GPT 4). If OpenAI stole copyrighted works to train their models it is morally fair game to do the same to them regardless of their TOS. It's not like they can prove it anyway. Plus there's the other po…

It's against the terms of service to do the generation, but the generated text is not copyrighted. Those are different things.

show me the ToS where it says that, and I still won't care, because it would absolute be legal under the same principle openAI is using for the training data as a transformative work.

FYI: here are the relevant parts from the TOS:

(iii) use output from the Services to develop models that compete with OpenAI; (iv) except as permitted through the API

sounds like you are allowed to as long as it's from the api, as this "imaginary" restriction isn't in https://openai.com/policies/api-data-usage-policies, or https://openai.com/policies/usage-policies.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#114

Post by Taek »

nico wrote: post #67

What is the main difference between training and fine tuning? Can you start with a model trained only in producing the letter a, and then fine tune it to learn b, then c, then words, sentences, etc?

For "full fine tuning", mathematically there's no difference. Fine tuning is just extending the training on new data.

What you are suggesting is called "curriculum learning", and though it hasn't been applied to LLMs yet to the best of my knowledge, it has proven to improve learning and decrease training times in other areas of ML.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#115

Post by a_bonobo »

heliophobicdude wrote: post #43
slenocchio wrote: post #24

Can someone explain why I'd want to use fine-tuning instead of a vector database (or some other way of storing data/context)?

Assuming you would want to fine-tune over a codebase or set of documents, I would argue vector databases and fine-tuning are completely different tools. I would strongly recommend against fine-tuning over a set of documents as this is a very lossy information system retrieval system. LLMs are not well suited for information retrieval like databases and search engines. The applications of fine-tuning that we are seein…

Great reply, here's an example from my own work:

I want the user to be able to ask technical questions about a set of documents, then the user should retrieve a summary-answer from those documents along with a source.

I first need to finetune GPT4 so it better understands the niche-specific technical questions, the words used, etc. I could ask the finetuned model questions, but it won't really know from where it got the information. Without finetuning the summarised answer will suffer, or it will pull out the wrong papers.

Then I need to use a vector database to store the technical papers for the model to access; now I can ask questions, get a decent answer, and will have access to the sources.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#116

Post by heliophobicdude »

a_bonobo wrote: post #115
heliophobicdude wrote: post #43

Earlier quoted context omitted.

Assuming you would want to fine-tune over a codebase or set of documents, I would argue vector databases and fine-tuning are completely different tools. I would strongly recommend against fine-tuning over a set of documents as this is a very lossy information system retrieval system. LLMs are not well suited for information retrieval like databases and search engines. The applications of fine-tuning that we are seein…

Great reply, here's an example from my own work: I want the user to be able to ask technical questions about a set of documents, then the user should retrieve a summary-answer from those documents along with a source. I first need to finetune GPT4 so it better understands the niche-specific technical questions, the words used, etc. I could ask the finetuned model questions, but it won't really know from where it got…

Ah! That makes sense! That's a neat strategy!

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#117

Post by forgingahead »

a_bonobo wrote: post #115
heliophobicdude wrote: post #43

Earlier quoted context omitted.

Assuming you would want to fine-tune over a codebase or set of documents, I would argue vector databases and fine-tuning are completely different tools. I would strongly recommend against fine-tuning over a set of documents as this is a very lossy information system retrieval system. LLMs are not well suited for information retrieval like databases and search engines. The applications of fine-tuning that we are seein…

Great reply, here's an example from my own work: I want the user to be able to ask technical questions about a set of documents, then the user should retrieve a summary-answer from those documents along with a source. I first need to finetune GPT4 so it better understands the niche-specific technical questions, the words used, etc. I could ask the finetuned model questions, but it won't really know from where it got…

Thanks (to both you and the parent) for sharing these details. So is it fair to say the following:

1. Fine-tuning bakes the knowledge into the model, but getting the "source" of an answer to a specific question becomes cagey and it is unclear if the answer is accurate or just a hallucination.

2. Therefore vector databases, which can provide context to the LLM before it answers, can solve this "citation" problem, BUT:

3. We then have limits because of the context window of the LLM to begin with.

Is that a fair understanding, or have I totally gotten this incorrect?

Edit: Or, are you saying that you both fine-tune AND also use a vector database which stores the embeddings of the dataset used to fine-tune the model?

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#118

Post by ehnto »

cameldrv wrote: post #65
sp332 wrote: post #58

Earlier quoted context omitted.

It's against the terms of service to do the generation, but the generated text is not copyrighted. Those are different things.

GPT-4 is trained on a large number of web pages, some of which will have had their own terms of service.

Is it legal for one of their computer systems to access mine without my consent, even if publicly routable via the internet?

If I found an open port on a government computer it is still illegal for me to access that isn't it? Is the difference that this is port 80/443 and happens to serve HTTP requests something that has been described in law or court?

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#119

Post by ehnto »

sirsinsalot wrote: post #30
moffkalast wrote: post #29

Earlier quoted context omitted.

> I can use smthg like GPT-4 to label data and then use that as a train set for my own LLM, right? Yes, almost all improved LLama models are tuned exactly that way (trained on examples of questions and answers from say GPT 4). If OpenAI stole copyrighted works to train their models it is morally fair game to do the same to them regardless of their TOS. It's not like they can prove it anyway. Plus there's the other po…

This ... but we all know business is corrupt. The current attempts to spur on regulation by OpenAI is moat building

We were complacent while it happened because OpenAI wasn't a business, it wasn't seen as unethical to use community work to contribute to community research. Now they're entrenched and pulled the rug out from the community, whilst also trying to shut the door on anyone else.

Just a really disappointing series of events, the money and profit were never the big issue.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#120

Post by nl »

tsunamifury wrote: post #88
fnordpiglet wrote: post #82

Earlier quoted context omitted.

This is true - afaik there’s been no specific rulings on whether training models on copyright material is a violation. But to my mind it harkens back to stuff like xerox and such where the tool itself isn’t the violating thing it’s the use of the tool. Likewise, derivative works are often largely reproductions with minor variations and are protected under fair use. A model that takes enormous amounts of data and dist…

In the history of media law I’ve seen judged lean into whatever interpretation balances the ecosystem more than what is “literally the law”. The law is meant to serve people not the other way around. I hope judges will understand the contribution and theft can’t just be “haha fuck humanity love, openAI”

I want to train my own LLM on public but copyrighted data. I think this is serving humanity (and fucking OpenAI). I also think it is ethical because there's a big difference between "learning from" and "copying".

Your proposed reading of the law means only big tech will be able to afford the license fees to train on large amounts of data.

Post reply on HN