Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

github.com

Post reply on HN
Print view View original

31–40 of 314 posts

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#31

Post by sam0x17 »

brookst wrote: post #23
yieldcrv wrote: post #16

Earlier quoted context omitted.

Given recent guidance from the US Copyright Office, I doubt there are real copyright issues or enforceable restrictions on commercial use. I would say that someone trying to sell the original model would be the only case that could have a real challenge, and even then its not that clear cut to me. Everyone else using these fine tuned models, I don't think there is a case. Everyone's making non-copyrightable scaffoldi…

There’s a big difference between copyright for material produced by a model, and copyright on the software and weights for the model. As far as I know, USPTO has said that works crested primarily by AI may not receive copyright protection, but I don’t think there’s any indication that the software to run AI is not copyrightable. Maybe?

We clearly need to run the weights _through_ a GPT model because that seems to strip copyright from them :D

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#33

Post by Filligree »

aiappreciator wrote: post #28
hyperopt wrote: post #18

Does anyone know of any good test suites we can use to benchmark these local models? It would be really interesting to compare all the ones capable of running on consumer hardware so that users can easily choose the best ones to use. Currently, I'm a bit unsure how this compares to the Alpaca model released a few weeks ago.

The simplest and quickest benchmark is to do a rap battle between GPT-4 and the local models. Copy paste the responses between them to enable the cross-model battle. It is instantly clear how strong the model is relative to GPT-4.

Have you tried it? How did it do?

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#34

Post by qeternity »

barbazoo wrote: post #21
qeternity wrote: post #13

Earlier quoted context omitted.

Models don't support context. You feed prior prompt/response pairs back into the input.

Oh is that how it works under the hood?

Yes. The chat functionality is really just a clever trick. All LLMs are still just predicting the next token in a sequence (i.e. finishing the sentence).

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#35

Post by vdfs »

barbazoo wrote: post #21
qeternity wrote: post #13

Earlier quoted context omitted.

Models don't support context. You feed prior prompt/response pairs back into the input.

Oh is that how it works under the hood?

It's not a secret, in OpenAI api you have to keep sending previous question and responses on top of your new question, essentially you are asking new question but give it more context with the previous questions and answers

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#36

Post by sam0x17 »

PeterisP wrote: post #19
yieldcrv wrote: post #11

is "number of parameters" going to become a useless metric? or would this be that much better when using a fine-turned version of the 65B parameter Llama model.

It already is, as you can have wildly different outcomes for models of the same size, and if you can get a similar output from a ten times smaller model, that's not equivalent, that's overwhelmingly better since it takes ten times less compute and for current model sizes the cost of inference is quite relevant.

We need to see something like "dropout" gain popularity again where you take a trained network and keep removing hidden neurons and re-training a few iterations until you can't obtain the same accuracy anymore, at which point you've reached what should be the smallest that network can be while still encoding the same knowledge. With models of these sizes though, few have the compute or the time to do stuff like that. There's a way of doing the whole thing iteratively though while you train that shouldn't add too much extra complexity..

The fact that the big players aren't doing this doesn't surprise me --- the more parameters, the less they have to worry about small-time competititors. I wouldn't be surprised if you could make an order of magnitude reduction in parameters and get the same results, and they're all just terrified we'll figure that out

As an undergrad I did some research on mapping "computational complexity" concepts onto neural networks to answer questions like what classes of problems require what minimum hidden layer sizes or network topologies using strategies similar to this. I was actually astounded how simple some of the resulting networks were that could nonetheless solve complex tasks like OCR with very few weights.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#37

Post by paxys »

hyperopt wrote: post #18

Does anyone know of any good test suites we can use to benchmark these local models? It would be really interesting to compare all the ones capable of running on consumer hardware so that users can easily choose the best ones to use. Currently, I'm a bit unsure how this compares to the Alpaca model released a few weeks ago.

The measure of a "good" model is still very subjective. OpenAI has used stuff like standardized test scores to compare the latest iterations of GPT, but that is only one of the many possible objective measures and might not be relevant in a lot of cases. Maybe we'll come to a consensus around such a methodology soon, or maybe it'll be something every user has to judge on their own depending on their goals.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#39

Post by chatmasta »

sam0x17 wrote: post #30
bob1029 wrote: post #22

Earlier quoted context omitted.

Why bother with the weights when you can just take the literal output of another model and consume as part of your training corpus? What kind of copyright claims can be made against the content generated by a model wielding the desired weights?

According to the US Copyright office's recent ruling, nothing generated by these models can be copyrighted (!), so in theory you should be fine!

nit: that wasn't a "ruling" so much as it was "guidance" [0] [1]

[0] https://www.federalregister.gov/documents/2023/03/16/2023-05...

[1] Discussed on HN: https://news.ycombinator.com/item?id=35191206

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#40

Post by throwayyy479087 »

pfisch wrote: post #17
breck wrote: post #7

Earlier quoted context omitted.

Da Vinci had no licenses. Nor Euclid, Nor Einstein. In the long run, licenses are for losers.

And I think we can all agree technical progress under the copyright system is moving much slower than it did back in da vinci's day.... If only we didn't have patents holding us back we would be investing way more in R&D...

I can’t tell if this is satire, but i literally have autism
Post reply on HN