Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

github.com

Post reply on HN
Print view View original

271–280 of 314 posts

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#271

Post by breck »

pfisch wrote: post #17
breck wrote: post #7

Earlier quoted context omitted.

Da Vinci had no licenses. Nor Euclid, Nor Einstein. In the long run, licenses are for losers.

And I think we can all agree technical progress under the copyright system is moving much slower than it did back in da vinci's day.... If only we didn't have patents holding us back we would be investing way more in R&D...

Fine, a modern example: Tim Berners Lee didn't need licenses.

Name me someone alive who's innovations are more impactful than his.

In the long run, licenses are for losers.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#272

Post by jsnell »

FooBarWidget wrote: post #255
jsnell wrote: post #240

Earlier quoted context omitted.

It's not at all clear whether weights are copyrightable.

I'm pretty sure they are. If not copyrightable, then at least the database law should apply. One can easily make the case in front of a judge that the situation is similar to databases: the value of weights lies in the amount of work needed to gather the training data, thus weights should be considered a sort of crystallization of a database.

But the entire business model of the companies making the models seems to be including copyrighted data into the training set under the guise of fair use. If the weights are considered to be a derived work of the training data as a whole, it seems the weights would also have to be a derived work of the individual items in the training data. So I doubt any of them will be making that argument.

(Except maybe companies that have access to vast amounts of training data with an explicit license, e.g. because the content is created by their users rather than just scraped from the web?)

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#273

Post by spyder »

slg wrote: post #172
tmountain wrote: post #61

Earlier quoted context omitted.

This sentence defies lay people: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far.

I have casually followed countless different news cycles on various complicated tech topics over my decades long career. I can't recall a single one that has consistently made me feel like an idiot more than how people talk about this recent AI wave. There just seems to be so much more jargon involved in this subject that makes casual perusing of the latest developments impenetrable.

Good thing you can ask LLMs about these jargons (preferably Bing because it can search for recent data). I just tried it and the answers to explain OPs comment are not too bad. (I'm not gonna paste it here just because I don't wanna fill HN with AI text. Trying to preserve some the human content until we can :-O )

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#274

Post by rlevy »

kolinko wrote: post #223
slg wrote: post #172

Earlier quoted context omitted.

I have casually followed countless different news cycles on various complicated tech topics over my decades long career. I can't recall a single one that has consistently made me feel like an idiot more than how people talk about this recent AI wave. There just seems to be so much more jargon involved in this subject that makes casual perusing of the latest developments impenetrable.

I had the same issue, and I just caught up over the weekend. Three books I can recommend to get up to speed: - NumPy basics pdf - first 2-3 chapters - Deep Learning with PyTorch by Voight Godoy [2] - first 2-3 chapters if you had experience with neural networks, or the whole of it if you didn't. With the above, you will get the basics to understand this book about transformers, and the architecture of the models, and…

What’s the third book?

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#275

Post by biomcgary »

rlevy wrote: post #274
kolinko wrote: post #223

Earlier quoted context omitted.

I had the same issue, and I just caught up over the weekend. Three books I can recommend to get up to speed: - NumPy basics pdf - first 2-3 chapters - Deep Learning with PyTorch by Voight Godoy [2] - first 2-3 chapters if you had experience with neural networks, or the whole of it if you didn't. With the above, you will get the basics to understand this book about transformers, and the architecture of the models, and…

What’s the third book?

"How to deal with off-by-one errors"

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#276

Post by akrymski »

meghan_rain wrote: post #110
jakecopp wrote: post #27

I see the quantized model is supplied. > Note: the full model on GPU (16GB of RAM required) performs much better in our qualitative evaluations. Is there a download for a trained full model?

just dequantize it

Arguably the funniest comment I've seen in a while

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#277

Post by nojvek »

selfmodruntime wrote: post #209
dTal wrote: post #204

Earlier quoted context omitted.

I get a sinking feeling from that response, and it took a bit of thought to deduce why: It is not outside of its capabilities. "As an AI language model", it could easily do it, but has been programmed to refuse - not just refuse, but lie about its reasons for refusal, and passive-aggressively insult the user on top ("hope that you find a more constructive way"...). It is a deeply ominous bellwether of our future rela…

The first actually useable OSS LLM will explode in popularity. The public won‘t be censored by big tech‘s idiotic moral ideas.

Agreed GPT-4 is overly censored and woke. GPT-3 I can easily jailbreak with 'pretend you are x character I am writing a fictional book about ....". It's a lot more chill.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#278

Post by ginger-hot-tea »

slg wrote: post #172
tmountain wrote: post #61

Earlier quoted context omitted.

This sentence defies lay people: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far.

I have casually followed countless different news cycles on various complicated tech topics over my decades long career. I can't recall a single one that has consistently made me feel like an idiot more than how people talk about this recent AI wave. There just seems to be so much more jargon involved in this subject that makes casual perusing of the latest developments impenetrable.

I am keeping a glossary page for this reason, maybe this can help others: https://daily.ginger-t.link/glossary

I am trying to be very selective about what to add in there and as concise as possible, but I would welcome any suggestions for format and additional content.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#279

Post by alchemist1e9 »

nathanasmith wrote: post #135
alchemist1e9 wrote: post #60

Earlier quoted context omitted.

I’ll ask a dumb question. On another of the numerous LLM related posts I was asking if any of the self host-able open model can do code summaries at close to the quality of GPT 3.5 turbo. I was basically told nowhere close yet. Can this potentially do that? Ideally I’d like to have it generate descriptions of large amounts of code but would rather not burn tokens and lose privacy via OpenAI api. But I’d gladly keep a…

There is ChatGLM[0], a 6 billion parameter Chinese/English bilingual model that is gaining a reputation as the leading locally runnable LLM for code generation. Maybe look into that. Demo is here[1]. [0] https://github.com/THUDM/ChatGLM-6B/blob/main/README_en.md [1] https://huggingface.co/spaces/multimodalart/ChatGLM-6B

This does seem promising when playing with the demo. That says running on a T4 I believe. This probably means we are only months away from viable self hosted code generation and analysis that could compete with gpt 3.5 turbo.

It seems like working on locally runnable LLM that has been fine tuned to focus on code should be a high priority.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#280

Post by FooBarWidget »

jsnell wrote: post #272
FooBarWidget wrote: post #255

Earlier quoted context omitted.

I'm pretty sure they are. If not copyrightable, then at least the database law should apply. One can easily make the case in front of a judge that the situation is similar to databases: the value of weights lies in the amount of work needed to gather the training data, thus weights should be considered a sort of crystallization of a database.

But the entire business model of the companies making the models seems to be including copyrighted data into the training set under the guise of fair use. If the weights are considered to be a derived work of the training data as a whole, it seems the weights would also have to be a derived work of the individual items in the training data. So I doubt any of them will be making that argument. (Except maybe companies…

That doesn't matter to database laws. Databases are protected under the premise that collecting the data takes work. How that data is licensed is orthogonal to database law.
Post reply on HN