71–80 of 126 posts
What is the main difference between training and fine tuning? Can you start with a model trained only in producing the letter a, and then fine tune it to learn b, then c, then words, sentences, etc?
What is the main difference between training and fine tuning? Can you start with a model trained only in producing the letter a, and then fine tune it to learn b, then c, then words, sentences, etc?
However, sometimes you can't do that. For example, perhaps you want your model to always talk like a pirate, but you don't have billions of words spoken like a pirate to train on.
So the next best thing is to train a model on all english text (which you have lots of), and then finetune on your smaller dataset of pirate speech.
Finetuning is simply more training, but with a different dataset and often a different learning rate.
Typically, finetuning uses far far far less data and compute, and can be done by individuals with a home PC, whereas training a large language model from scratch is in the $1M - $1B range.
The guide obv didn't make usable code and the github looks nearly unrelated.
I'm somewhat surprised there isnt a parameter for 'input_data' and 'output_data' and it returns a trained model. I can't figure out why there is so much boilerplate when that stuff could be contained as parameters.
What is the main difference between training and fine tuning? Can you start with a model trained only in producing the letter a, and then fine tune it to learn b, then c, then words, sentences, etc?
Yeah, since fine tuning seems to be so much more cheaper than training why haven't OpenAI fine tuned ChatGPT on data past 2021?
I mention the "undoing RLHF" since it's not uncommon for fine-tuned models to increase in error in the original training objective after being fine-tuned with a different one. I think people saw this happen in BERT.
Also ChatGPT is almost certainly huge.
What is the main difference between training and fine tuning? Can you start with a model trained only in producing the letter a, and then fine tune it to learn b, then c, then words, sentences, etc?
Yeah, since fine tuning seems to be so much more cheaper than training why haven't OpenAI fine tuned ChatGPT on data past 2021?
We have already documented evidence of the effect of this. In the GPT-4 technical report [1], they reported contamination of humaneval data in the training data.
They did measure against a "non-contaminated" training set but no idea if that can still be trusted.
Why would this matter? We can have seemingly strong benchmarks for containments but measures poorly against new and quarantined information. Classic over fitting.
Another argument is that data being put out there could very much be wrong and the amounts of it amplified by other models. Take a look at this sample of demonstration data for codealpaca [2]. Not only is its output wrong but bad practices like,making up a random computation without it having access to a place to run a calculation, teaches the model these type of responses are ok.
{ "instruction": "What would be the output of the following JavaScript snippet?", "input": "let area = 6 * 5;\nlet radius = area / 3.14;", "output": "The output of the JavaScript snippet is the radius, which is 1.91." }
1: https://cdn.openai.com/papers/gpt-4.pdf 2: https://github.com/sahil280114/codealpaca/commit/0d265112c70...
What is the main difference between training and fine tuning? Can you start with a model trained only in producing the letter a, and then fine tune it to learn b, then c, then words, sentences, etc?
Useful for taking a generic model with a base level of knowledge, and tuning it so the output is more useful for an application specific use case.
Edit to add: There are a number of Google Colabs for fine-tuning SD and I wonder if there are (or if it is technically feasible) to accomplish the same with other txt2txt models.
What is the main difference between training and fine tuning? Can you start with a model trained only in producing the letter a, and then fine tune it to learn b, then c, then words, sentences, etc?
Not an expert, but my high level understanding is this: If a model is a set of inputs, some middle layers, and a set of outputs. Fine tuning concentrates on only the output layers. Useful for taking a generic model with a base level of knowledge, and tuning it so the output is more useful for an application specific use case.
Can someone explain why I'd want to use fine-tuning instead of a vector database (or some other way of storing data/context)?
Embeddings = Input
Fine-tuning is like a chef modifying a general pizza recipe to perfect a specific pizza, such as Neapolitan. This customization optimizes the result. In AI, fine-tuning adjusts a pre-existing model to perform better on a specific task.
Embeddings are like categorizing ingredients based on properties. They represent inputs so that similar inputs have similar representations. For instance, 'dog' and 'puppy' in an AI model have similar meanings. Like ingredients in a pizza, embeddings help the model understand and interpret the inputs. So, fine-tuning is about improving the model's performance, while embeddings help the model comprehend its inputs.
It turns out, you can search a vector space of embeddings to find similar embeddings. If I turned my above post into 2 embeddings, and you searched for "golden retreiver" though neither paragraph has that exact phrase, the model should know a golden retreiver is most similar to the second paragraph that compares puppy to dog.