Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Finetune GPT-Like Large Language Models on a Custom Dataset

lightning.ai

Post reply on HN
Print view View original

81–90 of 126 posts

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#81

Post by SparkyMcUnicorn »

nomagicbullet wrote: post #78

Is there are Dreambooth equivalent for fine-tuning ChatGPT as there is for Stable Diffusion? I have to imagine that if we can add custom data to a DL text-to-image model, we should be able to do the same with a text-to-text one. Edit to add: There are a number of Google Colabs for fine-tuning SD and I wonder if there are (or if it is technically feasible) to accomplish the same with other txt2txt models.

These aren't for ChatGPT, but work on LLaMA, Vicuna, etc.

https://github.com/oobabooga/text-generation-webui/blob/main...

https://github.com/zetavg/LLaMA-LoRA-Tuner

https://github.com/h2oai/h2o-llmstudio

https://github.com/rhulha/lora

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#82

Post by fnordpiglet »

tsunamifury wrote: post #68
fnordpiglet wrote: post #61

Earlier quoted context omitted.

Use of copyrighted material in such a way that it’s aggregated into statistical properties is almost certainly fair use. Use of the model to produce reproductions of copyrighted material then consuming or distributing it is almost certainly violating the copyright. But it was the facsimile of the material that’s the violation, not the abstract use of it to generate an aggregate model.

You understand these things have a very very wide interpretation scope here that has yet to be tested in court. I wouldn’t make these statements so confidently as courts tend to reinterpret the law significantly for the balance of societal factors when serious technology changes occur.

This is true - afaik there’s been no specific rulings on whether training models on copyright material is a violation. But to my mind it harkens back to stuff like xerox and such where the tool itself isn’t the violating thing it’s the use of the tool. Likewise, derivative works are often largely reproductions with minor variations and are protected under fair use. A model that takes enormous amounts of data and distills it into a tiny vector representation way below the information theoretic levels for any meaningful fidelity and mixes and overlaps data in a way that the original data isn’t plausibly stored in the model… I’m definitely not going to wager my life that’s fair use, but I would wager my company on it.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#83

Post by a5huynh »

nomagicbullet wrote: post #78

Is there are Dreambooth equivalent for fine-tuning ChatGPT as there is for Stable Diffusion? I have to imagine that if we can add custom data to a DL text-to-image model, we should be able to do the same with a text-to-text one. Edit to add: There are a number of Google Colabs for fine-tuning SD and I wonder if there are (or if it is technically feasible) to accomplish the same with other txt2txt models.

If you're running the text-generation-webui (https://github.com/oobabooga/text-generation-webui) it has the ability to train LoRAs.

It'll require a beefy GPU but I've seen some fun examples like someone training a LoRA on Skyrim books.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#84

Post by SparkyMcUnicorn »

swalsh wrote: post #80
slenocchio wrote: post #24

Can someone explain why I'd want to use fine-tuning instead of a vector database (or some other way of storing data/context)?

Fine Tuning = Output Embeddings = Input Fine-tuning is like a chef modifying a general pizza recipe to perfect a specific pizza, such as Neapolitan. This customization optimizes the result. In AI, fine-tuning adjusts a pre-existing model to perform better on a specific task. Embeddings are like categorizing ingredients based on properties. They represent inputs so that similar inputs have similar representations. For…

I like to think of an LLM as a literal human. Not sure if it's the best analogy.

Fine tuning = Adding years of experience, in a set environment. e.g. Raise them in a home that only speaks in old english, learn pig latin, send them to a bootcamp.

Embedding = Giving them a book to reference information.

Just like a human, memory might fade a bit through the years but old habits die hard. You might not perfectly recollect what you learned years ago, but you still get the general idea, and if you took a class on the referenced book you'll be better at relaying information from it.

Edit: Asked ChatGPT to create the analogy.

A language model is like an intelligent person.

- Pre-training is their broad education and general knowledge.

- Fine-tuning is their years of specialized experience in a specific field.

- Embedding is like giving them a comprehensive book on a particular subject.

Just as a person gains knowledge, expertise, and specialized resources, the language model develops its understanding and performance through pre-training, fine-tuning, and embedding.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#85

Post by cubefox »

visarga wrote: post #33
wodenokoto wrote: post #4

Earlier quoted context omitted.

It is my understanding that this is how “alignment” works. That is, openAI paid people to chat with their LLM to fine tune it and then other LLMs use chatgpt to generate training data to align their models.

There are three ways 1. make your own RLHF dataset - like OpenAI and Open Assistant 2. exfiltrate data from a bigger/better LLM - Vicuna & family 3. use your pre-trained LLM to generate RLAIF data, no leeching - ConstitutionalAI, based on a set of rules instead of labelling examples

I wonder whether these approaches fit into the above categories:

https://arxiv.org/abs/2305.13735

https://arxiv.org/abs/2305.11206

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#86

Post by svaha1728 »

cameldrv wrote: post #65
sp332 wrote: post #58

Earlier quoted context omitted.

It's against the terms of service to do the generation, but the generated text is not copyrighted. Those are different things.

GPT-4 is trained on a large number of web pages, some of which will have had their own terms of service.

Not only web sites, full books from scribd and other sources.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#88

Post by tsunamifury »

fnordpiglet wrote: post #82
tsunamifury wrote: post #68

Earlier quoted context omitted.

You understand these things have a very very wide interpretation scope here that has yet to be tested in court. I wouldn’t make these statements so confidently as courts tend to reinterpret the law significantly for the balance of societal factors when serious technology changes occur.

This is true - afaik there’s been no specific rulings on whether training models on copyright material is a violation. But to my mind it harkens back to stuff like xerox and such where the tool itself isn’t the violating thing it’s the use of the tool. Likewise, derivative works are often largely reproductions with minor variations and are protected under fair use. A model that takes enormous amounts of data and dist…

In the history of media law I’ve seen judged lean into whatever interpretation balances the ecosystem more than what is “literally the law”. The law is meant to serve people not the other way around. I hope judges will understand the contribution and theft can’t just be “haha fuck humanity love, openAI”

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#89

Post by pmoriarty »

dragonwriter wrote: post #40
pmoriarty wrote: post #7

Earlier quoted context omitted.

So what are they going to do about it?

> So what are they going to do about it? If they think they can prove you used it to develop a competing service, sue you for breaking the TOS and recover the greater of the harm it did to their business or the amount of your profits from the service that are due to the uae of GPT-4 in violation of the agreement.

Have companies managed to get awarded damages in lawsuits against their customers who merely broke their terms of service?

Is there existing case law here?

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#90

Post by pmoriarty »

dragonwriter wrote: post #41
sanxiyn wrote: post #37

Earlier quoted context omitted.

I don't think that works. LLM-generated contents are not copyrightable.

Breach of contract for violating the TOS agreed to when signinf uo for the service doesn’t depend on copyright.

Is a terms of service considered a contract?
Post reply on HN