Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

github.com

Post reply on HN
Print view View original

61–70 of 314 posts

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#61

Post by tmountain »

2bitencryption wrote: post #44

Having spent quite a bit of time playing around with llama.cpp, alpaca.cpp, loras, and the many other llama-based weights lately, here is my impression: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far. [0] If yo…

This sentence defies lay people:

The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#62

Post by politician »

sam0x17 wrote: post #31
brookst wrote: post #23

Earlier quoted context omitted.

There’s a big difference between copyright for material produced by a model, and copyright on the software and weights for the model. As far as I know, USPTO has said that works crested primarily by AI may not receive copyright protection, but I don’t think there’s any indication that the software to run AI is not copyrightable. Maybe?

We clearly need to run the weights _through_ a GPT model because that seems to strip copyright from them :D

You can use the summarize, generate technique to strip copyright from source code too.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#63

Post by eyegor »

tacitusarc wrote: post #2

> Reverse a string in python. > In Python, you can reverse a list or tuple by using the reversed() function on it. Here's an example of how to use this method with strings: > my_string = "Hello World" # Define your original string here reversed_str = my_string[::-1] # Use a slice and negative index for reverse order print(reversed_str) Hmm…

It actually gave you 2 for the price of 1, that's quite the success

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#64

Post by rcpt »

sebzim4500 wrote: post #8
avereveard wrote: post #5

Wow things are moving super fast, but it's scary that everything is being tainted by licensing. Super fun for research, but commercial endeavors are on shady grounds.

Hopefully someone will be brave enough to fight Meta on the LLaMA copyright issue. My concern is that the people who can afford a massive legal battle can also afford to train their own model.

I've assumed llama was intentionally public to "commoditize the compliment"

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#66

Post by 2bitencryption »

alchemist1e9 wrote: post #60
2bitencryption wrote: post #44

Having spent quite a bit of time playing around with llama.cpp, alpaca.cpp, loras, and the many other llama-based weights lately, here is my impression: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far. [0] If yo…

I’ll ask a dumb question. On another of the numerous LLM related posts I was asking if any of the self host-able open model can do code summaries at close to the quality of GPT 3.5 turbo. I was basically told nowhere close yet. Can this potentially do that? Ideally I’d like to have it generate descriptions of large amounts of code but would rather not burn tokens and lose privacy via OpenAI api. But I’d gladly keep a…

Here's my experience, having used llama+lora 7b, 13b, and 30b, on both cpu and gpu:

On gpu, processing the input prompt, even for huge prompts, is almost instant. Meaning, even if your prompt is huge, it will start generating new tokens after your prompt very quickly. On a rented A6000 gpu, using llama+lora 30b, you can use huge prompts and it will start giving a new output right away.

On cpu (i.e. the project llama.cpu), it takes a very, very long time to process the input prompt, before it begins to generate new tokens. Meaning, if you provide a huge copy/paste of code, it will take a long time to ingest all that input, before it begins outputting new tokens.

Once it finally starts outputting new tokens, the rate is surprisingly fast, not much slower than gpu.

I wish I knew the reason for this, but I'm not an expert :) I've just seen this in practice.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#67

Post by rohfle »

mightytravels wrote: post #54

I feel bad - now someone has to pay 9 cents * 4GB in S3 costs when I downloaded the weights. Cloudflare R2 has free bandwidth and is much faster (my current speed for any S3 download is just 10MBit despite 300MBit connection) - way to few devs use R2.

I opened an issue for this thanks to your suggestion: https://github.com/nomic-ai/gpt4all/issues/11

Edit: talked to someone on discord, they're aware of it and working on moving the hosting

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#68

Post by kvetching »

avereveard wrote: post #5

Wow things are moving super fast, but it's scary that everything is being tainted by licensing. Super fun for research, but commercial endeavors are on shady grounds.

The question is, who's going to stop us from using it for commercial grounds. Most companies that would use these models would be startups and are years away from any sort of potential audit that may occur.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#70

Post by transitivebs »

The important takeaway for both GPT4All and Alpaca is that once an expensive proprietary model is released, people can easily train cheaper OSS models on input/output pairs.

⇒ LLMs are not defensible

⇒ LLMs will become commoditized

⇒ Prices will drop

⇒ Great for open source

Post reply on HN