Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

111–120 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#111

Post by pclmulqdq »

vkou wrote: post #98
pclmulqdq wrote: post #70

Earlier quoted context omitted.

Shouldn't that be the default position? The training methods are certainly patentable, but the actual input to the algorithm is usually public domain, and outputs of algorithms are not generally copyrightable as new works (think of to_lowercase(Harry Potter), which is not a copyrightable work), so the model weights would be a derivative work of public domain materials, and hence also forced into the public domain fro…

I like your legal interpretation, but it's way too early to tell if it is one that accurately represents the reality of the situation. We won't know until this hits the courts.

For what it's worth, I've been working on a startup that involves training some models, and this is likely how we're going to be treating the legal stuff (and being very careful about how customers can interact with the models as a consequence). I assume people who have different incentives will take a different view, though.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#112

Post by ericpauley »

BulgarianIdiot wrote: post #100
bugglebeetle wrote: post #93

Earlier quoted context omitted.

I would assume so. Meta’s ML/AI team is very strong, but they probably don’t have a comparable product offering to ChatGPT ready for public use. So instead, they bought themselves some time by letting the open source community run wild with a lesser model and eat into OpenAI’s moat.

They didn't leak it. Someone else did.

I think the key here is that, given the way that Meta distributed the model, a leak was inevitable. So while they may not have directly orchestrated a leak it must have been an intended result.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#113

Post by greyface- »

BulgarianIdiot wrote: post #100
bugglebeetle wrote: post #93

Earlier quoted context omitted.

I would assume so. Meta’s ML/AI team is very strong, but they probably don’t have a comparable product offering to ChatGPT ready for public use. So instead, they bought themselves some time by letting the open source community run wild with a lesser model and eat into OpenAI’s moat.

They didn't leak it. Someone else did.

They have tacitly endorsed the leak. https://github.com/facebookresearch/llama/pull/73#issuecomme...

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#114

Post by guwop »

halotrope wrote: post #106
nkzd wrote: post #97

Earlier quoted context omitted.

How could Meta ever find out your private business is using their model without a whistleblower? It's practically impossible.

You can just ask if there is no output filtering

The future is going to be hilarious. Just ask the model who made it!

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#115

Post by PufPufPuf »

nkzd wrote: post #97
elcomet wrote: post #23

Earlier quoted context omitted.

They clearly expected the leak, they distributed it very widely to researchers. The important thing is the licence, not the access: you are not allowed to use it for commercial purpose.

How could Meta ever find out your private business is using their model without a whistleblower? It's practically impossible.

Yes, that's how software piracy has always worked.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#116

Post by seydor »

EGreg wrote: post #102
seydor wrote: post #48

Earlier quoted context omitted.

It is steered by RLHF to give helpful, nice, structured continuations. it was totally trained on random text they never paid a dime for.

Can you please elaborate and answer 1,2,3 directly — I’d love to find out more. Maybw links to the techniques

I am not an expert but as others have said, the InstructGPT and Alpaca models do that

https://arxiv.org/abs/2203.02155#

https://crfm.stanford.edu/2023/03/13/alpaca.html

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#117

Post by ShannonLimiter »

isoprophlex wrote: post #94
irrational wrote: post #6

> OpenAI published a detailed blog post outlining some of the principles used to ensure safety in their models. The post emphasize in areas such as privacy, factual accuracy Am I the only one amused by the phrase “factual accuracy”? How many stories have we read like the one where it tries to ghost light the guy that this year is actually last year. “Oh, your phone must be wrong too, because there is no way I could b…

I find the thing incredibly smart and yet utterly useless at times. I just spent 20 minutes getting the current iteration of ChatGPT to agree with me that a certain sentence is palindromic. Even when you make it print the unaccented characters one by one, spaces excluded, backwards and forwards, it still insists "Élu par cette crapule" isn't palindromic. I understand how tokenization makes this difficult but come on.…

Just ask it to figure it out step-by-step and to remove accents:

GPT-4:

Figure this out step by step and check your work: Remove the accents and tell me if this a palindrome: Élu par cette crapule

GPT-3.5 (Needs to know that removing spaces and lowercasing is okay):

Figure this out step by step and check your work: Lowercase it and remove the accents and spaces and tell me if this a palindrome: Élu par cette crapule

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#118

Post by superkuh »

I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves are incomparably worse at doing useful tasks than openai's gpt3.5 models like text-davinci-003, or even the gimped gpt3.5-turbo.

I wish it wasn't so, but the llamas are toys. Amazing toys, but toys. What openai is getting out of gpt3.5 (and presumbably 4, though I have no access) are actually useful responses for getting work done.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#119

Post by klyrs »

hanselot wrote: post #108
aardvarkr wrote: post #91

Earlier quoted context omitted.

You’re correct. US law states that intellectual property can be copyrighted only if it was the product of human creativity, and the USCO only acknowledges work authored by humans at present. Machines and generative AI algorithms, therefore, cannot be authors, and their outputs are not copyrightable.

Sorry for asking a stupid question. How much Theseus do I need to ship before I can copyright it as my own? Is there some threshold for how much of an AI generated work needs to be modified by "human creativity" prior to it being copyrightable?

As far as I can tell, you can claim credit for the output of a tool which produces something at your direction. If you write an algorithm to generate patents, and you execute that algorithm and submit its output to USPTO under your own name, no problem. If you gave credit to the algorithm, their policy is to deny the claim.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#120

Post by nailer »

> Llama was not initially open-sourced, but a week after its release, the model was leaked on 4chan, sparking thousands of downloads.

The author very clearly does not know what Open source is. Proprietary code that’s been leaked isn’t open source, and code that is derived from proprietary code is still proprietary.

Windows had it source code leaked, that doesn’t make it open source.

So did the game Portal. Not open source either.

Something being leaked does not change the license.

Post reply on HN