Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

141–150 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#141

Post by muyuu »

cubefox wrote: post #122
superkuh wrote: post #118

I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…

Your conclusion seems not to be warranted since you haven't tried out the 65B model.

his conclusion is simultaneously not warranted and correct

a like-for-like comparison would be GPT-4 against the larger models like LLaMA 65B, but those cannot be run on consumer-grade hardware

so one ends up comparing the stuff one can run... against the top stuff from OpenAI running on high-end GPU farms, and this technology clearly benefits a lot still from much larger scale than most people can afford

the great revelation this year is how much does it get better as it get much, much bigger without a clear horizon on where will diminishing returns be hit

but at the same time, some useful stuff can be done on consumer hardware - just not the most impressive stuff

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#142

Post by lolinder »

superkuh wrote: post #129
cubefox wrote: post #122

Earlier quoted context omitted.

Your conclusion seems not to be warranted since you haven't tried out the 65B model.

I agree, but I think my experience is representative. So far most human people don't have the resources to be able to use 65B. And most small companies / university groups don't have the resources to fine-tune a 65B. I've talked to a couple dozen people in real time who've played with up to 30B but no one I know has the resources to run the 65B at all or fast enough to actually use and get an opinion of. None of the…

llama.cpp has great support for 65B, and I've been using it on a Linux box (I upgraded my RAM for that purpose). 64GB of RAM for a desktop is like $160 today, so it's not out of reach for most people here if they cared to.

Admittedly, it's quite slow and therefore not useful for chatting or real-time applications, and it's unreliable enough in its quality that I'd like to be able to iterate faster. Definitely more of a toy at this point, at least when run on CPU.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#143

Post by egillie »

klyrs wrote: post #46
seydor wrote: post #39

Someone needs to legally challenge openAI on using the output of their models to train other commercial models. If web scraping is legal, then this must be legal too , even if openAI tries to curtail it. After all it was all trained on data they don't have rights to.

Yeah, I'm particularly curious about that -- there's already legal precedent in the US that an AI cannot author copyrighted nor patented work. OpenAI can try to curtail it through a clickwrap agreement, but those are notoriously weak.

And even if they do, if someone uses the api and makes a publicly available website with no clickwrap, it would be legal to scrape that right?

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#144

Post by RhodesianHunter »

amrb wrote: post #137
adeon wrote: post #10

I'm a bit worried the LLaMA leak will make the labs much more cautious about who they distribute models to for future projects, closing down things even more. I've had tons of fun implementing LLaMA, learning and playing around with variations like Vicuna. I learned a lot and probably wouldn't have got so interested in this space if the leak didn't happen.

Devil's Advocate: The EU comes down hard on any AI company that doesn't work with researchers and institutions in future.

Outright banning due to fear seems far more likely.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#145

Post by Mathnerd314 »

greyface- wrote: post #113
BulgarianIdiot wrote: post #100

Earlier quoted context omitted.

They didn't leak it. Someone else did.

They have tacitly endorsed the leak. https://github.com/facebookresearch/llama/pull/73#issuecomme...

That's a contributor to the repo, not someone with commit access.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#146

Post by visarga »

bri3d wrote: post #68
rmdashrfstar wrote: post #58

Earlier quoted context omitted.

> If web scraping is legal Source? That LinkedIn case did not resolve how you think it did.

My understanding is that the current web scraping situation is this: * Web scraping is not a CFAA violation. (EF Travel v. Zefer, LinkedIn v. hiQ). * Scraping in spite of clickthrough / click-in ToS "violation" on public websites does not constitute an enforceable breach of contract, chattel trespass (ie - incidental damage to a website due to access), or really mean anything at all. This is not as clear once a user…

Is training a model on second-hand data laundering copyright? Second-hand data is data generated from a model that has been trained on copyrighted content.

Let's say I train a diffusion model on ten million images generated by diffusion models that have seen copyrighted data. I make sure to remove near duplicates from my training set. My model will only learn the styles but not the exact composition of the original dataset. So it won't be able to replicate original work, because it has never seen any original work.

Is this a neat way of separating ideas from their expression? Copyright should only cover expression. This kind of information laundering follows the definition to the letter and only takes the part that is ok to take - the ideas, hiding the original expression.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#147

Post by WXLCKNO »

Closi wrote: post #124
superkuh wrote: post #118

I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…

I’ve got access to 4 and it’s a huge leap up from 3.5 - much more subtlety in the response, less hallucinations, less hitting a brick wall, but all of it adding up to a giant leap.

It's funny how big the difference feels between 3.5 and 4 but also feels subtle. Like it's just doing what I expect it to versus seeing the limitations more clearly in 3.5.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#148

Post by Szpadel »

muyuu wrote: post #141
cubefox wrote: post #122

Earlier quoted context omitted.

Your conclusion seems not to be warranted since you haven't tried out the 65B model.

his conclusion is simultaneously not warranted and correct a like-for-like comparison would be GPT-4 against the larger models like LLaMA 65B, but those cannot be run on consumer-grade hardware so one ends up comparing the stuff one can run... against the top stuff from OpenAI running on high-end GPU farms, and this technology clearly benefits a lot still from much larger scale than most people can afford the great r…

if "leaks" are true then even a 65B model is a toy when GPT-4 is supposed to be 1000B I would really like to have an open source model that could compete with even gpt3, but basically all lammas I tried failed to create correct email validation regex, where gpt3 is able to even refer to what rfc it is

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#149

Post by hrpnk »

What's amazing to see is the effort to attempt to run the models on consumer-grade hardware, going as far as running 4-bit quantized models on phones or raspberry pi. All the debacle about mmap optimizations to llama.cpp [1] and the style these were committed to the product is a great testimony of open source. Both in the positive aspect (progress) and the negative ones (visibility affecting human judgement and collaboration). The sheer amount of experimentation is also providing a standard interface for different models that can easily be integrated and tried out.

[1] https://github.com/ggerganov/llama.cpp

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#150

Post by Scene_Cast2 »

lolinder wrote: post #142
superkuh wrote: post #129

Earlier quoted context omitted.

I agree, but I think my experience is representative. So far most human people don't have the resources to be able to use 65B. And most small companies / university groups don't have the resources to fine-tune a 65B. I've talked to a couple dozen people in real time who've played with up to 30B but no one I know has the resources to run the 65B at all or fast enough to actually use and get an opinion of. None of the…

llama.cpp has great support for 65B, and I've been using it on a Linux box (I upgraded my RAM for that purpose). 64GB of RAM for a desktop is like $160 today, so it's not out of reach for most people here if they cared to. Admittedly, it's quite slow and therefore not useful for chatting or real-time applications, and it's unreliable enough in its quality that I'd like to be able to iterate faster. Definitely more of…

Could you quantify "quite slow"?
Post reply on HN