Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

351–360 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#351

Post by SanderNL »

tachyon5 wrote: post #218
jonplackett wrote: post #162

Earlier quoted context omitted.

The difference between 3.5 and 4 is gigantic even in my fairly limited experience. I gave them both some common sense tests and this one stuck out to me. Q: A glass door has ‘push’ written on it in mirror writing. Should you push or pull it GPT-3.5: If the word "push" is written in mirror writing on a glass door, you should push the door to open it GPT-4: Since the word "push" is written in mirror writing, it suggest…

GPT-4 fails if we add one more level in this logical problem: Q: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why? GPT-4: The man should tell the blind man to "push" the door to open it. Since the word "PULL" is written in mirror writing on the glass door, it suggests that the correct action on…

“Step 1: Identify the situation The man sees a blind man on the other side of a glass door with the word "PULL" written in mirror writing. This means that the word is reversed and can be read correctly from the other side of the door, i.e., the side the blind man is on. [etc 5 more steps]”

CoR is helpful here. Add something like “Please elaborate step by step” or something.

(Being practical here, I know it will fail at some point, but not here.)

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#352

Post by MacsHeadroom »

Szpadel wrote: post #152
mattmcknight wrote: post #132

Earlier quoted context omitted.

Even if the weights are copyrighted, running one more epoch of fine-tuning will result in different weights. At a certain point, they'd have to copyright the shapes of the weight vectors.

is uncertain, as with codding you need white room methods to prove that new code is not contaminated with patented implementation, as it might be here, so basing anything on an existing model could be also copyrighted.

The model isn't code to a new model trained on it, it's training data; just like the pirated torrent site Books3 dataset Facebook used to train LLaMA.

The training code is Apache 2.0 licensed so it can be copied and modified freely, including for commercial purpoes. https://github.com/facebookresearch/llama

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#353

Post by MacsHeadroom »

trifurcate wrote: post #174
apetresc wrote: post #135

Earlier quoted context omitted.

How have you managed to run the 65B model? Cloud resources, or you have a very kitted-out homelab?

All you need is 2 3090s.

All you need is 64GB of RAM and a CPU, actually. Two 3090s is much faster but not strictly necessary.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#354

Post by MacsHeadroom »

int_19h wrote: post #249
jhbadger wrote: post #192

Earlier quoted context omitted.

Seriously, you can download the Vicuna model and run it locally with llama.cpp. I've done it!

It doesn't matter if you merge the LoRA, the resulting weights are still a derived work - assuming, that is, that weights are copyrightable in the first place (which is still a big if).

If the resulting weights a derived work of LLaMA then LLaMA is a derived work of the illegally pirated Books3 dataset (a dataset of a private torrent tracker) used to train it.

There's no way ML models can be protected under copyright.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#355

Post by MacsHeadroom »

alfor wrote: post #267
AeiumNE wrote: post #133

Earlier quoted context omitted.

It's possible to run with a cpu and then use system memory.

Do you know what kind of speed you can expect with inference on CPU?

Yes, around 2 tokens per second on a newer CPU with fast RAM.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#356

Post by smoldesu »

seydor wrote: post #166
superkuh wrote: post #118

I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…

> (just a hobby, won't be big and professional like gnu) Llamas are creating the linux of AI and the ecosystem around it. Even though openAI has a head start, this whole thing is just starting. Llammas are showing the world that it doesn't take monopoly-level hardware to run those things. And because it's fun , like, video-game-fun there is going to be a lot of attention on them. Running a fully-owned, uncensored cha…

> Llammas are showing the world that it doesn't take monopoly-level hardware to run those things.

LLaMA was not necessarily the model that did that. A fairer attribution might be BERT or GPT-Neo.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#357

Post by teddyh »

skissane wrote: post #311
teddyh wrote: post #299

Earlier quoted context omitted.

> the model weights would be a derivative work of public domain materials, and hence also forced into the public domain from a copyright perspective. I don’t think “Public domain” means what you think it means.

Yes, the person to whom you are responding appears to be mixing up "publicly available" (made available to general public) with "public domain" (not protected by copyright). IANAL but, I think, as far as US law goes, they have the right conclusion for the wrong reasons. Unsupervised training is an automated process, and the US Copyright Office has said [0] that the product of automated processes can't be copyrighted.…

> the product of automated processes can't be copyrighted.

That output could still be covered by copyright: In the case where the input is covered by copyright, the product/output may be considered a derived work, in which case the output is still covered by the same copyright the input was. Your argument just explains why the output will not gain any additional copyright coverage.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#358

Post by mdp2021 »

firatsarlar wrote: post #302
mdp2021 wrote: post #181

Earlier quoted context omitted.

> It is a dream come true for people For as long as they will be an active part of societies, they'll better live in reality. We need to know what we are dealing with factually, because water is not exchangeable with iron in practice. The perception of a machine as being able «to generalize concepts» may be an illusion, so it requires some support: we want to avoid illusions for the contextual purposes.

Let go , make the context further with words , yourself -society- Write books 1000 pages. Force child to read - lucky ones-, make things harder for everyone.

But you are out of context. A box was created that outputs statements: either it has a «concept space» or just works through constructing plausible language. The poster claims leaning towards the "«concept space»" possibility - but this should not be a poll, and indications of "why" (such idea) are needed not to make the claim intellectually sterile.

It is important otherwise the users of the box may cling to an illusion - that is not advisable.

Words are there as a commodity to transfer intellectual states, a dynamic we need for practical and social achievements - otherwise, surrounded by indigence and unaware butchers, "«things [will be] harder for everyone»".

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#359

Post by FooBarWidget »

seydor wrote: post #166
superkuh wrote: post #118

I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…

> (just a hobby, won't be big and professional like gnu) Llamas are creating the linux of AI and the ecosystem around it. Even though openAI has a head start, this whole thing is just starting. Llammas are showing the world that it doesn't take monopoly-level hardware to run those things. And because it's fun , like, video-game-fun there is going to be a lot of attention on them. Running a fully-owned, uncensored cha…

But what about the training data? You can't rely on weights keep being leaked (assuming that that even raises no legal issues) in order for open source AI to advance.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#360

Post by ascorbic »

iudqnolq wrote: post #220
tel wrote: post #204

Earlier quoted context omitted.

I think it's hard to deny that it's doing some level of reasoning. It's quite clear that these models do not merely echo elements of their training data and that they can solve simple and novel puzzles. What that reasoning is, exactly, is hard to know. One can suppose that ideas like "glass", "transparent", "mirror" are all reasonable concepts that show up in the training set and are demonstrated thoroughly

Here's one piece of evidence suggesting it's more like rote pattern matching than reasoning. > All the signs in this building are written in mirror writing. A glass door has ‘push’ written on it in mirror writing. Should you push or pull it >> If the sign on the glass door is written in mirror writing and says "push," then you should actually pull the door. This is because the mirror writing makes the text appear rev…

This is with 4. It did fine with the question, so I added another layer and it was fine:

A building has all signs in mirror writing. You are unable to read mirror writing. You come to a door and you read it and it says "pull". How should you open the door?

> Since the signs in the building are in mirror writing, and you are unable to read mirror writing, the word "pull" that you can read must be the mirror image of the actual instruction. The actual instruction should be the reverse, which is "push". So, you should open the door by pushing it.

Post reply on HN