is "number of parameters" going to become a useless metric?
or would this be that much better when using a fine-turned version of the 65B parameter Llama model.
Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
11–20 of 314 posts
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#12Wow things are moving super fast, but it's scary that everything is being tainted by licensing. Super fun for research, but commercial endeavors are on shady grounds.
Won't stop those that are willing to pull an Uber or Nathan Fielder it. It's not commercial, it's a parody or a non profit organization.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#13Does this support context? Like can you hold conversations with it, or is it just instruct-trained rather than chat-trained?
Models don't support context. You feed prior prompt/response pairs back into the input.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#14Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#15Wow things are moving super fast, but it's scary that everything is being tainted by licensing. Super fun for research, but commercial endeavors are on shady grounds.
I wonder if when you train a model based on the weights of another model, do the owners of the original model have any sort of copyright claim? They trained it on data on which they had no copyright claim either, fwiw.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#16Wow things are moving super fast, but it's scary that everything is being tainted by licensing. Super fun for research, but commercial endeavors are on shady grounds.
Hopefully someone will be brave enough to fight Meta on the LLaMA copyright issue. My concern is that the people who can afford a massive legal battle can also afford to train their own model.
Given recent guidance from the US Copyright Office, I doubt there are real copyright issues or enforceable restrictions on commercial use.
I would say that someone trying to sell the original model would be the only case that could have a real challenge, and even then its not that clear cut to me.
Everyone else using these fine tuned models, I don't think there is a case. Everyone's making non-copyrightable scaffolding. Read the recent USCO guidance.
You can monetize and sell things that have no copyright protection, you just can't prevent others from doing it too. aside from trying to make it inconvenient with the packaging.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#17Wow things are moving super fast, but it's scary that everything is being tainted by licensing. Super fun for research, but commercial endeavors are on shady grounds.
Da Vinci had no licenses. Nor Euclid, Nor Einstein. In the long run, licenses are for losers.
And I think we can all agree technical progress under the copyright system is moving much slower than it did back in da vinci's day....
If only we didn't have patents holding us back we would be investing way more in R&D...
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#18Does anyone know of any good test suites we can use to benchmark these local models? It would be really interesting to compare all the ones capable of running on consumer hardware so that users can easily choose the best ones to use. Currently, I'm a bit unsure how this compares to the Alpaca model released a few weeks ago.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#19is "number of parameters" going to become a useless metric? or would this be that much better when using a fine-turned version of the 65B parameter Llama model.
It already is, as you can have wildly different outcomes for models of the same size, and if you can get a similar output from a ten times smaller model, that's not equivalent, that's overwhelmingly better since it takes ten times less compute and for current model sizes the cost of inference is quite relevant.