Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

github.com

Post reply on HN
Print view View original

101–110 of 314 posts

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#101

Post by Kerrick »

d4rkp4ttern wrote: post #87
transitivebs wrote: post #70

The important takeaway for both GPT4All and Alpaca is that once an expensive proprietary model is released, people can easily train cheaper OSS models on input/output pairs. ⇒ LLMs are not defensible ⇒ LLMs will become commoditized ⇒ Prices will drop ⇒ Great for open source

Only concern is this violates terms of use of the proprietary LLM API, at least that is true for the current OpenAI API

I am not a lawyer, but I don’t think that affects the copyright or license of works created with the broken-term software. For example, wouldn’t you still own the copyright to a brochure you made with pirated Photoshop, or a photo you took with a DJI drone while flying out of sight?

(All of which may be moot if models can’t be copyrighted because they’re machine-generated.)

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#102

Post by aunty_helen »

alchemist1e9 wrote: post #79
aunty_helen wrote: post #75

Earlier quoted context omitted.

We're a long long long way off that. So check back in two months. Jokes aside, the limiting factor will be either a technique to pack all of the code into smaller tokens like semantic search (someone else will be able to comment on this as that's the limit of my understanding) or GPU memory for input tokens. Buying a "high end GPU" isn't buying a 4090 or even two, it's 250k on a DGX unit and putting it in a datacentr…

Are you sure? This is for inference, not training, and I’d do it on blocks of code with less than 4096 tokens. That’s what I have to do with gpt 3.5 turbo via api. I can do what I want with the API and it is around 1-2 cents per batch of code I run through it, typically 400 lines of average code is around 4k tokens.

If you can get it within 4k of tokens then your vram needs will be much less. As the other two have pointed out, there's workarounds to this.

If you're trying to get codebase level introspection you might need to wait a bit for some of these techs to mature.

Exciting space and yea, learning heaps myself day by day.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#103

Post by Casteil »

2bitencryption wrote: post #83
aunty_helen wrote: post #75

Earlier quoted context omitted.

We're a long long long way off that. So check back in two months. Jokes aside, the limiting factor will be either a technique to pack all of the code into smaller tokens like semantic search (someone else will be able to comment on this as that's the limit of my understanding) or GPU memory for input tokens. Buying a "high end GPU" isn't buying a 4090 or even two, it's 250k on a DGX unit and putting it in a datacentr…

It's $0.590/hr to rent an A6000 48GB gpu on jarvislabs.ai, which you can do right now, and run the 30b (or 65b with some hard work) model and get incredible results. No confidentiality agreement required :)

Do you know how their "spot pricing" works, exactly? "lets you use spare GPUs" doesn't tell us much. Is it granular to how long the duration the model is actively processing your prompt? ..or do you get dinged for an hour immediately and again on the hour?

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#104

Post by physPop »

axlee wrote: post #77
refulgentis wrote: post #71

Earlier quoted context omitted.

A Lora is a layer on top of a model, the big deal isn’t that this exists (it’s a Lora for the weakest llama), but the fact they shared their dataset. The stronger llamas trained with this data will produce even better Lora’s and better results.

What is a lora or llama? Google gives me nothing.

silly back-ronyms made by academics

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#105

Post by mightytravels »

Results feel like GPT-2.

How far away is Pluto?

Plutonian objects are located at distances between 30 and 59 AU from their primary, which means they range in distance from about 27 million to over a billion miles (43-1.6×10^8 km) away!

Provide a funny reply to this Tweet - Someone called the cops at my local McDonald’s because the ice cream machine was broken. No, I am not kidding

I'm sorry that someone had such an unpleasant experience with your local McDonald's! However, it sounds like they may have been overreacting by calling in police officers to fix a malfunctioning ice cream machine - perhaps there was another issue at play? Regardless of the reason for their call, I hope that things are resolved quickly and smoothly.

Provide a funny reply to this Tweet - Microsoft Teams now has 270m users. Slack has 20m users. Wild chart.

I've heard about the new update on MS teams, it seems like they are really working hard towards making their product better and more user-friendly for everyone! However, I must admit that this is quite an impressive number to have achieved in such a short time frame - congratulations Microsoft Teams team!!

..keeps rambling without stopping...

### Instruction:

Provide funny reply to the Tweet "I'm sorry but we don’t accept bitcoin as payment" ### Response:

Sorry, I can understand why you wouldn't want to take BitCoins. It seems like a pretty volatile currency and it could be risky for your business if things go wrong with the exchange rate or something else goes haywire! However, don’t worry too much...

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#106

Post by Dwedit »

Trying it out:

> Answer this Jeopardy question: This long-handled gardening tool can also mean an immoral pleasure seeker.

The correct answer for the given jeopardy question would be "A hoe". The word 'ho' is slang term used to describe a promiscuous woman or prostitute, which makes it clear that this long-handled gardening tool can also mean an immoral pleasure seeker.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#107

Post by wesleychen »

AlexanderTheGr8 wrote: post #69

Legal question : while openai gpt outputs cannot be used for training a commercial model, if someone reworded / rephrased them using another open src model, would it then be okay to use it to train a commercial model?

I question if OpenAI can legally enforce their TOS regarding creating commercial models using their outputs. According to the FTC memo, AI-generated content cannot be copyrighted. Perhaps the worst that can happen is a banned account.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#108

Post by A4ET8a8uTh0 »

6gvONxR4sf7o wrote: post #88
sebzim4500 wrote: post #8

Earlier quoted context omitted.

Hopefully someone will be brave enough to fight Meta on the LLaMA copyright issue. My concern is that the people who can afford a massive legal battle can also afford to train their own model.

You want people brave enough to fight meta for control of the work meta did and shared for free? It's attitudes like this that mean LLaMA will serve as a lesson for other companies to not share as much. It's too bad, because in normal open source, I like that the community usually tells people to abide by the license terms chosen by the authors. But in this, everyone's just demanding unrestricted use.

There is a real concern that this particular technology, as powerful as it is, will be kept behind the doors of the biggest corps and information gleamed from prompts will be further used to our collective detriment.

I am personally still coming to terms with it, but it is not just wanting stuff. It is making sure society does not get too separated in terms of power ( because it is already pretty stratified ).

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#109

Post by meghan_rain »

sdrinf wrote: post #29
hyperopt wrote: post #18

Does anyone know of any good test suites we can use to benchmark these local models? It would be really interesting to compare all the ones capable of running on consumer hardware so that users can easily choose the best ones to use. Currently, I'm a bit unsure how this compares to the Alpaca model released a few weeks ago.

Test suites are not reflection complete! https://sdrinf.com/reflection-completeness -essentially, the moment a set of testing data gets significant traction, it becomes a target to optimize for. Instead, I strongly recommend to put together a list of "control questions" of your own, that covers the general, and specific use cases you're interested in. Specifically, I'd recommend adding questions on topics you have hi…

Idea: Create a set of tests, that AI experts vet, but are kept secret. New models are run against them and only the scores are published.

Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

#110

Post by meghan_rain »

jakecopp wrote: post #27

I see the quantized model is supplied. > Note: the full model on GPU (16GB of RAM required) performs much better in our qualitative evaluations. Is there a download for a trained full model?

just dequantize it
Post reply on HN