Live data from Hacker News

How to Finetune GPT-Like Large Language Models on a Custom Dataset

lightning.ai

101–110 of 126 posts

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#101

Can someone explain why I'd want to use fine-tuning instead of a vector database (or some other way of storing data/context)?

I asked ChatGPT this question, and asked it to simplify as much as possible.

Fine-tuned Models: Imagine you have a super-smart robot that can talk about anything. But you want it to be really good at talking about, say, dinosaurs. So, you teach it more about dinosaurs specifically. That's what fine-tuning is – you're teaching the robot (or model) to be really good at a specific topic.

Vector Databases and Embeddings with LLM: This might be a little tricky, but let's think of it this way. Imagine you have a huge library of books and you want to find information on a specific topic, say, ancient Egypt. Now, instead of reading every book, you have a magical index that can tell you which books talk about ancient Egypt. This index is created by magically converting each book into a "summary dot" (that's the embedding). When you ask about ancient Egypt, your question is also converted into a "summary dot". Then, the magical index finds the books (or "summary dots") that are most similar to your question. That's how the vector database and embeddings work.

So, if you want your super-smart robot to be really good at one specific topic, you use fine-tuning. But if you want it to quickly find information from a huge library of knowledge, you use vector databases and embeddings. Sometimes, you might even use both for different parts of the same task!

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#102

Earlier quoted context omitted.

This is true - afaik there’s been no specific rulings on whether training models on copyright material is a violation. But to my mind it harkens back to stuff like xerox and such where the tool itself isn’t the violating thing it’s the use of the tool. Likewise, derivative works are often largely reproductions with minor variations and are protected under fair use. A model that takes enormous amounts of data and dist…

In the history of media law I’ve seen judged lean into whatever interpretation balances the ecosystem more than what is “literally the law”. The law is meant to serve people not the other way around. I hope judges will understand the contribution and theft can’t just be “haha fuck humanity love, openAI”

Yes, laws are about politics and dispute resolution more than reasoning or correctness. Focusing on the pure logic is a trap for the computationally inclined.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#104
post #95
post #69

Earlier quoted context omitted.

AI generated work is not copyright-able. I guess the courts later could disagree though. https://www.copyright.gov/ai/

If the AI generates a new Eric Clapton album, with the same similar voice and guitar playing style?

your example doesn't have to be AI generated. Human cover-bands play Song X in the style of Y all the time.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#105

When is fine tuning worth it, rather than just prompt engineering?

I think these are two very separate concepts. What we are mostly seeing when it comes to fine-tuning is making a model promptable. Models like LLaMA or the original GPT3 weren't promptable. They were fine-tuned with demonstration data that looks like a prompt input, prompt output. See below: { "instruction": "What would be the output of the following JavaScript snippet?", "input": "let area = 6 * 5;\nlet radius = are…

Prompt engineering and fine tuning are in many cases alternative ways to achieve the same goal. You claim that the "original GPT3" wasn't promptable. I'm unsure which version you refer to, but I'm guessing you refer to text-davinci-003 and it was definitely promptable. For one app I used prompt engineering to make it behave like a spirit talking through a ouija board. For another, I used prompt engineering to make it act like a dystopian search engine from the future. So, yeah, it's promptable.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#106
post #100

Earlier quoted context omitted.

GPT-4 is trained on a large number of web pages, some of which will have had their own terms of service.

see LinkedIn vs HiQ (which HiQ won) covering fair use of logged-out web pages.

I have to log in to OpenAI to generate conversations but the conversations I can post on my own logged-out blog. It's the same thing OpenAI would probably say if they got sued because GPT spits copyrighted content it found on a logged-out webpage. They can't reasonably expect people to not use them for training.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#107

Can someone explain why I'd want to use fine-tuning instead of a vector database (or some other way of storing data/context)?

Fine-tuning could be useful to get a high text completion quality out of a small model within a specific domain. You would still use the resulting model alongside an info retrieval system to prompt with real context (unless you have a use case where hallucination is a feature).

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#109
I have been working in this space for quite a while and while I think the beginning of pytorch lightning meant well, it seems modern use-cases have outgrown it.

These days, when I see content from Lightning AI, I prepare for a contrived approach to doing something that fits within the ecosystem. I can't help but feel they are trying to induce "vendor lock-in" where there really isn't a business case for it...

Anyways, I tried to follow these steps and hit a dead-end. I have to say the content put out by huggingface is always way more straightforward and gets me to where I need to be when I want to spin up quickly.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#110

Has anyone tried to use this? The guide obv didn't make usable code and the github looks nearly unrelated. I'm somewhat surprised there isnt a parameter for 'input_data' and 'output_data' and it returns a trained model. I can't figure out why there is so much boilerplate when that stuff could be contained as parameters.

I got stuck.

Try these: https://huggingface.co/blog/stackllama, https://huggingface.co/blog/trl-peft, https://huggingface.co/blog/hf-bitsandbytes-integration

Post reply on HN