Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

github.com

Post reply on HN
Print view View original

141–150 of 314 posts

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#141

Post by knodi123 »

What's up with this when running `python generate.py --config configs/generate/generate.yaml --prompt "Write a script to reverse a string in Python"` ?

    huggingface_hub.utils._errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-6423aed4-0f0192250fe8821a181f9b4f)

    Repository Not Found for url: https://huggingface.co/nomic-ai/vicuna-lora-1024/resolve/main/adapter_config.json.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#142

Post by zarzavat »

tetete wrote: post #127
d4rkp4ttern wrote: post #87

Earlier quoted context omitted.

Only concern is this violates terms of use of the proprietary LLM API, at least that is true for the current OpenAI API

It was trained using data they didn't have the license to. So will that hold in court? I hope not.

It’s not a licensing issue because OpenAI does not own the copyright to the output of GPT-4. The people who might have a copyright claim are: the authors of the training data, if there is a clear resemblance between the training data input and the output, and rarely the author of the prompt.

OpenAI could argue that because some of the training data was written by them, they have a copyright claim to the output. However, this is a very slippery slope for them as the entire existence of OpenAI is predicated their use of training data being fair use.

OpenAI can only control the output of GPT-4 via their terms of service. When you sign up to use ChatGPT or other services you agree to certain conditions.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#143

Post by robwwilliams »

Dwedit wrote: post #106

Trying it out: > Answer this Jeopardy question: This long-handled gardening tool can also mean an immoral pleasure seeker. The correct answer for the given jeopardy question would be "A hoe". The word 'ho' is slang term used to describe a promiscuous woman or prostitute, which makes it clear that this long-handled gardening tool can also mean an immoral pleasure seeker.

Or a rake.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#145

Post by stu2b50 »

Kerrick wrote: post #101
d4rkp4ttern wrote: post #87

Earlier quoted context omitted.

Only concern is this violates terms of use of the proprietary LLM API, at least that is true for the current OpenAI API

I am not a lawyer, but I don’t think that affects the copyright or license of works created with the broken-term software. For example, wouldn’t you still own the copyright to a brochure you made with pirated Photoshop, or a photo you took with a DJI drone while flying out of sight? (All of which may be moot if models can’t be copyrighted because they’re machine-generated.)

It's not an issue of copyright as terms of service. The images you create out of a pirate photoshop, you do own the copyright, but adobe can also go after you for the unrelated matter of illegally bypassing their DRM.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#146

Post by chronicler »

Majromax wrote: post #95
axlee wrote: post #77

Earlier quoted context omitted.

What is a lora or llama? Google gives me nothing.

LLaMA is the large language model published by Facebook ( https://ai.facebook.com/blog/large-language-model-llama-meta... ). In theory the model is private, but the model weights were shared with researchers and quickly leaked to the wider Internet. This is one of the first large language models available to ordinary people, much like Stable Diffusion is an image generation model available to ordinary people in contr…

Thanks for sharing. How do you know this? Can you recommend any papers to read to start learning about LLMs? I have very limited ML/AI knowledge.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#147

Post by stu2b50 »

jakecopp wrote: post #27

I see the quantized model is supplied. > Note: the full model on GPU (16GB of RAM required) performs much better in our qualitative evaluations. Is there a download for a trained full model?

They have the LoRA delta weights on huggingface, which is linked on the github. Since it's the just the deltas, they're substantially smaller (~8mb), and you'll need to supply the original 7b LLaMA yourself.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#148

Post by stu2b50 »

sam0x17 wrote: post #36
PeterisP wrote: post #19

Earlier quoted context omitted.

It already is, as you can have wildly different outcomes for models of the same size, and if you can get a similar output from a ten times smaller model, that's not equivalent, that's overwhelmingly better since it takes ten times less compute and for current model sizes the cost of inference is quite relevant.

We need to see something like "dropout" gain popularity again where you take a trained network and keep removing hidden neurons and re-training a few iterations until you can't obtain the same accuracy anymore, at which point you've reached what should be the smallest that network can be while still encoding the same knowledge. With models of these sizes though, few have the compute or the time to do stuff like that.…

That's not what dropout is, dropout is just a method of regularization (you remove a random, and small, subset of the neurons per training iteration, in order to reduce overfitting), and pretty much all LLM transformer blocks have dropout.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#150

Post by d4rkp4ttern »

BoorishBears wrote: post #122
d4rkp4ttern wrote: post #98

Earlier quoted context omitted.

Somehow I doubt there is any type of “prompt compression” being done.

It's not an uncommon technique so I'm not sure why you're so doubtful.

Because it introduces a possibly recursive application of the model itself and is a bit messy in that there would be some loss of information. It certainly would be a good technique for external applications, though.

Anyway, I got some clarification from ChatGPT itself:

Question: In a growing conversation, does ChatGPT create summarized version of the conversation history and send it to the model?

Answer:

In a growing conversation, ChatGPT does not create a summarized version of the entire conversation history and send it to the model. Instead, the model only uses the most recent turns of the conversation to generate a response to the current turn.

However, some implementations of ChatGPT do use a technique called "history trimming" to limit the size of the context window that the model considers. History trimming involves discarding some of the oldest turns of the conversation to keep the context window within a manageable size. This allows the model to focus on the most recent turns of the conversation, which may be more relevant to the current turn.

For example, OpenAI's GPT-3 API includes a feature called "context chunking" that automatically trims the conversation history to a maximum length of 2048 tokens. If the conversation history exceeds this length, the oldest turns are discarded to keep the context window within the maximum length.

Overall, while ChatGPT does not create a summarized version of the entire conversation history, some implementations may use history trimming to limit the size of the context window and improve the model's performance.

Post reply on HN