Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Finetune GPT-Like Large Language Models on a Custom Dataset

lightning.ai

Post reply on HN
Print view View original

61–70 of 126 posts

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#61

Post by fnordpiglet »

moffkalast wrote: post #29
artembugara wrote: post #2

Have a question to the Generative AI experts here. So, I can use smthg like GPT-4 to label data and then use that as a train set for my own LLM, right? EDIT: adding this from OpenAI Restriction TOS: "(iii) use output from the Services to develop models that compete with OpenAI;"

> I can use smthg like GPT-4 to label data and then use that as a train set for my own LLM, right? Yes, almost all improved LLama models are tuned exactly that way (trained on examples of questions and answers from say GPT 4). If OpenAI stole copyrighted works to train their models it is morally fair game to do the same to them regardless of their TOS. It's not like they can prove it anyway. Plus there's the other po…

Use of copyrighted material in such a way that it’s aggregated into statistical properties is almost certainly fair use. Use of the model to produce reproductions of copyrighted material then consuming or distributing it is almost certainly violating the copyright. But it was the facsimile of the material that’s the violation, not the abstract use of it to generate an aggregate model.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#62

Post by mercurialsolo »

While the fine-tuning pipeline is fairly straightforward for tuning and building custom models, the RLHF pipeline doesn't look to be as straightforward. Creating a dataset for RLHF seems like a fairly labour intensive exercise especially if your model is tuned to do work like code generation ?

What about the Replit Ghostwriter? Did it have a RLHF phase?

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#63

Post by bagels »

ramesh1994 wrote: post #21
fallingmeat wrote: post #6

Earlier quoted context omitted.

That is against their ToS though if you use your new LLM commercially.

It prohibits anything that competes with OpenAI services i.e as long as you're not literally providing an LLM API commercially you should be fine

Does it compete with them if you stop paying for their API?

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#65

Post by cameldrv »

sp332 wrote: post #58
moffkalast wrote: post #29

Earlier quoted context omitted.

> I can use smthg like GPT-4 to label data and then use that as a train set for my own LLM, right? Yes, almost all improved LLama models are tuned exactly that way (trained on examples of questions and answers from say GPT 4). If OpenAI stole copyrighted works to train their models it is morally fair game to do the same to them regardless of their TOS. It's not like they can prove it anyway. Plus there's the other po…

It's against the terms of service to do the generation, but the generated text is not copyrighted. Those are different things.

GPT-4 is trained on a large number of web pages, some of which will have had their own terms of service.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#66

Post by [deleted] »

dragonwriter wrote: post #55
nightski wrote: post #54

Earlier quoted context omitted.

Right but cease and desist usually relates to intellectual property or copyright matters, typically not TOS violations. Please correct me if I am mistaken.

Cease and desist can be used for any issues where the person or entity issuing the C&D thinks they have a legal right that is being violated and wants to put the violator on notice in the hopes of securing a change in behavior short of legal action.

[deleted]

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#68

Post by tsunamifury »

fnordpiglet wrote: post #61
moffkalast wrote: post #29

Earlier quoted context omitted.

> I can use smthg like GPT-4 to label data and then use that as a train set for my own LLM, right? Yes, almost all improved LLama models are tuned exactly that way (trained on examples of questions and answers from say GPT 4). If OpenAI stole copyrighted works to train their models it is morally fair game to do the same to them regardless of their TOS. It's not like they can prove it anyway. Plus there's the other po…

Use of copyrighted material in such a way that it’s aggregated into statistical properties is almost certainly fair use. Use of the model to produce reproductions of copyrighted material then consuming or distributing it is almost certainly violating the copyright. But it was the facsimile of the material that’s the violation, not the abstract use of it to generate an aggregate model.

You understand these things have a very very wide interpretation scope here that has yet to be tested in court. I wouldn’t make these statements so confidently as courts tend to reinterpret the law significantly for the balance of societal factors when serious technology changes occur.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#69

Post by itake »

tsunamifury wrote: post #68
fnordpiglet wrote: post #61

Earlier quoted context omitted.

Use of copyrighted material in such a way that it’s aggregated into statistical properties is almost certainly fair use. Use of the model to produce reproductions of copyrighted material then consuming or distributing it is almost certainly violating the copyright. But it was the facsimile of the material that’s the violation, not the abstract use of it to generate an aggregate model.

You understand these things have a very very wide interpretation scope here that has yet to be tested in court. I wouldn’t make these statements so confidently as courts tend to reinterpret the law significantly for the balance of societal factors when serious technology changes occur.

AI generated work is not copyright-able. I guess the courts later could disagree though.

https://www.copyright.gov/ai/

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#70

Post by akiselev »

pid-1 wrote: post #26
slenocchio wrote: post #24

Can someone explain why I'd want to use fine-tuning instead of a vector database (or some other way of storing data/context)?

I've been playing with using documents as OpenAI embeddings for the past weeks and, at least for my use case, the results are meh. It seems sometimes just using context is not enough. My next step is to play with fine tunning, but I have no results to report yet.

Try using InstructXL for embeddings. It’s got a more complex prompt structure for generating embeddings which might be more useful
Post reply on HN