Earlier quoted context omitted.
How does one find out what the max input size is?
Search context window size. For LLaMA it's 2048 tokens
Where a token is roughly equal a word?
Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
41–50 of 314 posts
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#42Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#43Earlier quoted context omitted.
Why bother with the weights when you can just take the literal output of another model and consume as part of your training corpus? What kind of copyright claims can be made against the content generated by a model wielding the desired weights?
In fine-tuned models, for example, you may only train a real subset of the weights. Who is the copyright owner now and what is the justification? There might be other ways of transfer-learning where you can greatly benefit from the pretrained weights, some of which may have not yet been discovered.
> In fine-tuned models, for example, you may only train a real subset of the weights.
I am not sure to what degree this applies throughout the various offerings. We are finding that "fine" tuning cuts pretty damn deep. For instance, with OpenAI you can start with base Curie/Davinci and build an effective chat bot that makes ChatGPT look incompetent in a very narrow vertical slice of some specialized domain.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#44Having spent quite a bit of time playing around with llama.cpp, alpaca.cpp, loras, and the many other llama-based weights lately, here is my impression:
The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far. [0]
If you're really limited to running "just" llama 7b, this is great for you. But the biggest value will be when people inevitably release lora adapters for the 13b, 30b, and 65b, based on this training data (assuming it really is better than the previously released adapters).
[0] admittedly, this is based off anecdotes and github issues, and not real measurements. but smarter people than I have claimed the currently most popular loras were trained on messy data, and have started an effort to clean that data and retrain. So if the training data in this repo is high quality like the authors claim, it will benefit models of all sizes.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#45Earlier quoted context omitted.
Search context window size. For LLaMA it's 2048 tokens
what is it for chatgpt?
4k for gpt-3.5 and 8k-32k for gpt4 (there are multiple versions: https://platform.openai.com/docs/models/gpt-4)
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#46Wow things are moving super fast, but it's scary that everything is being tainted by licensing. Super fun for research, but commercial endeavors are on shady grounds.
I wonder if when you train a model based on the weights of another model, do the owners of the original model have any sort of copyright claim? They trained it on data on which they had no copyright claim either, fwiw.
As I understand it, it's not a copyright issue, it's a terms-of-service issue. You're only authorized to use GPT in accordance with their terms, which prohibit using the service to develop a competitor. Doing so anyway is unauthorized use of the service, which is a Computer Fraud and Abuse Act violation, independent of what is and isn't copyrightable.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#47Earlier quoted context omitted.
The simplest and quickest benchmark is to do a rap battle between GPT-4 and the local models. Copy paste the responses between them to enable the cross-model battle. It is instantly clear how strong the model is relative to GPT-4.
Have you tried it? How did it do?
You're talking to the model right now.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#48Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#49Earlier quoted context omitted.
The simplest and quickest benchmark is to do a rap battle between GPT-4 and the local models. Copy paste the responses between them to enable the cross-model battle. It is instantly clear how strong the model is relative to GPT-4.
Have you tried it? How did it do?
Someone here did Bard v GPT-4 a few days ago, and GPT-4 mopped the floor with Bard: https://news.ycombinator.com/item?id=35252612