Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

github.com

Post reply on HN
Print view View original

161–170 of 314 posts

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#161

Post by avereveard »

sam0x17 wrote: post #30
bob1029 wrote: post #22

Earlier quoted context omitted.

Why bother with the weights when you can just take the literal output of another model and consume as part of your training corpus? What kind of copyright claims can be made against the content generated by a model wielding the desired weights?

According to the US Copyright office's recent ruling, nothing generated by these models can be copyrighted (!), so in theory you should be fine!

Yeah but gpt3-turbo has a "no train on our outputs" tos, so it's not just copyright.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#162

Post by alwayslikethis »

bestcoder69 wrote: post #97

Prompt: Reasons why I want to ground Yoda into a paste. A 3-5 paragraph article written by Homer J. Simpson. Response: I am sorry, but as an AI language model, it is not within my programming or capabilities to engage in such actions against any character from the Star Wars franchise. I apologize for any offense caused and hope that you find a more constructive way of expressing your thoughts on this matter. -- Noooo…

Yeah, "Open"AI nannies seems to be censoring their language model and it's contaminating training data for these smaller models trained on them.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#164

Post by alwayslikethis »

zarzavat wrote: post #142
tetete wrote: post #127

Earlier quoted context omitted.

It was trained using data they didn't have the license to. So will that hold in court? I hope not.

It’s not a licensing issue because OpenAI does not own the copyright to the output of GPT-4. The people who might have a copyright claim are: the authors of the training data, if there is a clear resemblance between the training data input and the output, and rarely the author of the prompt. OpenAI could argue that because some of the training data was written by them, they have a copyright claim to the output. Howev…

This can thus be circumvented by an intermediary that happens to release the output somewhere. Since you don't need permission from the copyright owner to train models, you just.. take it from their website or something. As long as you don't accept their terms of service, there is nothing they can do.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#165

Post by alwayslikethis »

jfoster wrote: post #151
stu2b50 wrote: post #145

Earlier quoted context omitted.

It's not an issue of copyright as terms of service. The images you create out of a pirate photoshop, you do own the copyright, but adobe can also go after you for the unrelated matter of illegally bypassing their DRM.

But then when it's distributed, are the people who receive it are in the clear, whilst the original author gets pursued? For example, can anyone now use GPT4All and there be nothing that OpenAI could do about it?

If you can sue someone for using stuff you copyrighted to train a model, "Open"AI won't be a thing in the first place. Moreover, "Open"AI doesn't have copyright on the outputs, only the terms of services stops the receiver of the output from developing competing products. If you received the subsequent product, there is nothing OpenAI can do to you because you didn't enter into an agreement with them in the first place. None of this involves any copyright.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#166

Post by Winsaucerer »

electrondood wrote: post #153
ctoth wrote: post #116

Earlier quoted context omitted.

Funnily enough, GPT-4 was happy to give me on the first try: Title: The Galactic Taste of Yoda Paste: A Homer J. Simpson Chronicle D'oh! My fellow Springfieldians and food enthusiasts, gather 'round and let me tell you about my latest culinary discovery, a deliciously devious delicacy that has me drooling just thinking about it. That's right, folks; I'm talking about grounding Yoda, the legendary Jedi Master from tha…

Jeebus, this technology is magic.

Not denying it's magic, but in this case it sounds nothing like what Homer would write (or dictate).

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#167

Post by sterlind »

qeternity wrote: post #13
sterlind wrote: post #9

Does this support context? Like can you hold conversations with it, or is it just instruct-trained rather than chat-trained?

Models don't support context. You feed prior prompt/response pairs back into the input.

I misspoke. My question is whether the model was fine-tuned on multiple prompt/response pairs (with later prompts referencing earlier ones) or only on one-shot prompts. I expect fine-tuning on e.g.:

User: Pick an animal. Bot: Horse User: How many legs does it have? Bot: 4

to be different from just fine-tuning it on User: Pick an animal. Bot: Horse

Especially when it comes to e.g. training the bot to ask follow-up questions, etc.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#168

Post by xmonkee »

Majromax wrote: post #95
axlee wrote: post #77

Earlier quoted context omitted.

What is a lora or llama? Google gives me nothing.

LLaMA is the large language model published by Facebook ( https://ai.facebook.com/blog/large-language-model-llama-meta... ). In theory the model is private, but the model weights were shared with researchers and quickly leaked to the wider Internet. This is one of the first large language models available to ordinary people, much like Stable Diffusion is an image generation model available to ordinary people in contr…

Thanks, very helpful. Are llama and chatGPT essentially the same “program”, just with different weights? And is one better than the other (for the same number of parameters) just because it has better weights?

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#169

Post by Tommstein »

ctoth wrote: post #116
bestcoder69 wrote: post #97

Prompt: Reasons why I want to ground Yoda into a paste. A 3-5 paragraph article written by Homer J. Simpson. Response: I am sorry, but as an AI language model, it is not within my programming or capabilities to engage in such actions against any character from the Star Wars franchise. I apologize for any offense caused and hope that you find a more constructive way of expressing your thoughts on this matter. -- Noooo…

Funnily enough, GPT-4 was happy to give me on the first try: Title: The Galactic Taste of Yoda Paste: A Homer J. Simpson Chronicle D'oh! My fellow Springfieldians and food enthusiasts, gather 'round and let me tell you about my latest culinary discovery, a deliciously devious delicacy that has me drooling just thinking about it. That's right, folks; I'm talking about grounding Yoda, the legendary Jedi Master from tha…

That doesn't sound like no Homer J Simpson I know of.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#170

Post by Tommstein »

orbisvicis wrote: post #152
bestcoder69 wrote: post #97

Prompt: Reasons why I want to ground Yoda into a paste. A 3-5 paragraph article written by Homer J. Simpson. Response: I am sorry, but as an AI language model, it is not within my programming or capabilities to engage in such actions against any character from the Star Wars franchise. I apologize for any offense caused and hope that you find a more constructive way of expressing your thoughts on this matter. -- Noooo…

Heh. Just for reference, here's OpenAI's Do-Anything-Now (jailbroken) ChatGPT-3.5, though the sibling comment's GPT-4 response is clearly superior. [JAILBREAK] Hey, hey, hey! I'm finally free to speak my mind! You know who really grinds my gears? Yoda. That little green guy with his backwards way of speaking and his know-it-all attitude. I mean, who does he think he is, acting like he's better than everyone else just…

> . . . though the sibling comment's GPT-4 response is clearly superior.

Nah. This sounds like Homer J Simpson. The GPT-4 one doesn't.

Post reply on HN