Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

github.com

Post reply on HN
Print view View original

261–270 of 314 posts

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#261

Post by muyuu »

FooBarWidget wrote: post #255
jsnell wrote: post #240

Earlier quoted context omitted.

It's not at all clear whether weights are copyrightable.

I'm pretty sure they are. If not copyrightable, then at least the database law should apply. One can easily make the case in front of a judge that the situation is similar to databases: the value of weights lies in the amount of work needed to gather the training data, thus weights should be considered a sort of crystallization of a database.

yea I do wonder about this, but even Meta are acting as if their releasing it means in effect that the cat is out of the bag

at this point, their not even complaining about it must mean that they accept the data is public now

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#262

Post by loufe »

londons_explore wrote: post #212
2bitencryption wrote: post #66

Earlier quoted context omitted.

Here's my experience, having used llama+lora 7b, 13b, and 30b, on both cpu and gpu: On gpu, processing the input prompt, even for huge prompts, is almost instant. Meaning, even if your prompt is huge, it will start generating new tokens after your prompt very quickly. On a rented A6000 gpu, using llama+lora 30b, you can use huge prompts and it will start giving a new output right away. On cpu (i.e. the project llama.…

That is likely a bug. There is nothing in the maths that should make prompt tokens slower on CPU.

It is a bug, it's been clearly discussed under the issues of LLaMA.cpp. The wait time is far from atrocious, with a 5600x and the 13B I would wait 15 seconds before input starts for a relatively complex prompt.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#263

Post by kolinko »

2bitencryption wrote: post #66
alchemist1e9 wrote: post #60

Earlier quoted context omitted.

I’ll ask a dumb question. On another of the numerous LLM related posts I was asking if any of the self host-able open model can do code summaries at close to the quality of GPT 3.5 turbo. I was basically told nowhere close yet. Can this potentially do that? Ideally I’d like to have it generate descriptions of large amounts of code but would rather not burn tokens and lose privacy via OpenAI api. But I’d gladly keep a…

Here's my experience, having used llama+lora 7b, 13b, and 30b, on both cpu and gpu: On gpu, processing the input prompt, even for huge prompts, is almost instant. Meaning, even if your prompt is huge, it will start generating new tokens after your prompt very quickly. On a rented A6000 gpu, using llama+lora 30b, you can use huge prompts and it will start giving a new output right away. On cpu (i.e. the project llama.…

Any tips on setting up llama+lora on 30b? There are so many resources that I can't figure out which models to use, and which projects to use to set everything up.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#265

Post by generalizations »

MallocVoidstar wrote: post #233
siraben wrote: post #158

It seems that although this LoRA produces longer form answers, it refuses to act as stylistically as Alpaca 13B/is more censored (I'm getting a few responses that start with "As an AI language model...") Prompt: Write a rant about brexit in a thick scottish accent alpaca 13B: Aye, I'm gonnae tell ye somethin', it's no fair! We shouldnae hae to leave tha EU, we're better together than apart, they'll be takin' our jobs…

Around 13% of the responses in the gpt4all dataset are GPT-3.5 saying it isn't permitted to respond to the prompt.

That really seems like it should have been filtered out first.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#266

Post by PartiallyTyped »

adlpz wrote: post #251
PartiallyTyped wrote: post #238

Earlier quoted context omitted.

ChatGPT is GPT3.5(++)

If you pay you can use GPT-4 within the ChatGPT UI.

I am aware, i was pointing out that chatgpt is not gpt 3.0 based but 3.5 and subsequent releases are further along.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#267

Post by harisec »

huqedato wrote: post #225

Another dumb question: is it possible to "fine-tune" this model with a "corpus" of specialized data/jargon in another language ? I tried to do this with OpenAI API in order to improve the response on a specialized field (pediatric oncology) in another language (Italian) but the results were mixed (not to say disappointing).

This is a common misunderstanding; fine-tuning a model does not mean teaching the model new information. Fine-tuning is used to adapt the model to perform better in a specific task or domain using the information it already has, in a specialized way (like a chatbot for question/answer interactions). Training a LLM on new data is extremely expensive so it's not possible.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#268

Post by pell »

bunnywantspluto wrote: post #163

This is a game changer! Just to clarify, how does this compare to OpenAI's GPT4 in terms of logical reasoning and whatnot?

Worlds apart. However it is still interesting to use and seems to give good answers for some queries.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#269

Post by lbotos »

xmonkee wrote: post #168
Majromax wrote: post #95

Earlier quoted context omitted.

LLaMA is the large language model published by Facebook ( https://ai.facebook.com/blog/large-language-model-llama-meta... ). In theory the model is private, but the model weights were shared with researchers and quickly leaked to the wider Internet. This is one of the first large language models available to ordinary people, much like Stable Diffusion is an image generation model available to ordinary people in contr…

Thanks, very helpful. Are llama and chatGPT essentially the same “program”, just with different weights? And is one better than the other (for the same number of parameters) just because it has better weights?

My understanding is they are both "LLM" (Large language models). That's the generic term you are looking for.

I don't think you can compare one LLMs weights to another directly, because the weights are a product of the LLM. In theory (I don't know actually) llama and chatGPT may be using different source datasets so you can't compare them like for like.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#270

Post by nickthegreek »

kolinko wrote: post #263
2bitencryption wrote: post #66

Earlier quoted context omitted.

Here's my experience, having used llama+lora 7b, 13b, and 30b, on both cpu and gpu: On gpu, processing the input prompt, even for huge prompts, is almost instant. Meaning, even if your prompt is huge, it will start generating new tokens after your prompt very quickly. On a rented A6000 gpu, using llama+lora 30b, you can use huge prompts and it will start giving a new output right away. On cpu (i.e. the project llama.…

Any tips on setting up llama+lora on 30b? There are so many resources that I can't figure out which models to use, and which projects to use to set everything up.

text-generation-webui (https://github.com/oobabooga/text-generation-webui)
Post reply on HN