Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

Post reply on HN
Print view View original

151–160 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#151

Post by hrpnk »

superkuh wrote: post #118

I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…

What this also shows is the degree of control that a single company has over the market. At a time where GPT-4 is integrated into products while others are on the waitlist, competition can move far ahead of a company that's just trying to gain access to the technology w/o clear insights on the prioritization approach. As said, there just are no alternatives at the moment, so to a degree competition is skewed.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#152

Post by Szpadel »

mattmcknight wrote: post #132
echelon wrote: post #26

Earlier quoted context omitted.

If the copyright office determines model weights are uncopyrightable (huge if), then one might imagine any institutional leak would benefit everyone else in the space. You might see hackers, employees, or contractors leaking models more frequently. And since models are distilled functionality (no microservices and databases to deploy), they're much easier to run than a constellation of cloud infrastructure.

Even if the weights are copyrighted, running one more epoch of fine-tuning will result in different weights. At a certain point, they'd have to copyright the shapes of the weight vectors.

is uncertain, as with codding you need white room methods to prove that new code is not contaminated with patented implementation, as it might be here, so basing anything on an existing model could be also copyrighted.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#153

Post by grugagag »

boppo1 wrote: post #74
drowsspa wrote: post #30

Earlier quoted context omitted.

It sounds like that king that wanted people to overcome their aversion for potatoes. So he put armed guards around the potato fields but instructed them to be very lax and allowed the people to rob it

Tell me more. Real or anecdote?

https://www.farmersalmanac.com/parmentier-made-potatoes-popu...

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#154

Post by lolinder »

Scene_Cast2 wrote: post #150
lolinder wrote: post #142

Earlier quoted context omitted.

llama.cpp has great support for 65B, and I've been using it on a Linux box (I upgraded my RAM for that purpose). 64GB of RAM for a desktop is like $160 today, so it's not out of reach for most people here if they cared to. Admittedly, it's quite slow and therefore not useful for chatting or real-time applications, and it's unreliable enough in its quality that I'd like to be able to iterate faster. Definitely more of…

Could you quantify "quite slow"?

A token per second-ish with a Ryzen 7 5800X. If I run it for too long it gets slower as heat throttling kicks in, I need a better cooling system if I'm going to run it non-stop.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#155

Post by qeternity »

superkuh wrote: post #129
cubefox wrote: post #122

Earlier quoted context omitted.

Your conclusion seems not to be warranted since you haven't tried out the 65B model.

I agree, but I think my experience is representative. So far most human people don't have the resources to be able to use 65B. And most small companies / university groups don't have the resources to fine-tune a 65B. I've talked to a couple dozen people in real time who've played with up to 30B but no one I know has the resources to run the 65B at all or fast enough to actually use and get an opinion of. None of the…

I agree with your premise: I have used 65b variants and of course they’re not as good as OpenAI. GPT3 has 175b parameters, and OpenAI has done more RLHF than anyone else. Why would we expect to get comparable performance with models a fraction of the size and a pittance of the fine tuning?

That said, it’s clear that replicating GPT4+ performance is within the resources of a number of large tech orgs.

And the smaller models can definitely still be useful for tasks.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#156

Post by dandongus »

lolinder wrote: post #154
Scene_Cast2 wrote: post #150

Earlier quoted context omitted.

Could you quantify "quite slow"?

A token per second-ish with a Ryzen 7 5800X. If I run it for too long it gets slower as heat throttling kicks in, I need a better cooling system if I'm going to run it non-stop.

For a bit of comparison, if you've tested, how fast are 13B or 7B on the same setup?

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#157

Post by greyface- »

Mathnerd314 wrote: post #145
greyface- wrote: post #113

Earlier quoted context omitted.

They have tacitly endorsed the leak. https://github.com/facebookresearch/llama/pull/73#issuecomme...

That's a contributor to the repo, not someone with commit access.

A contributor who is also a Facebook employee and co-author of the LLaMA paper, presumably speaking in official capacity.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#158

Post by Al-Khwarizmi »

superkuh wrote: post #118

I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…

In my particular tests (YMMV), even 13B vicuna beats Bard, though... tough times for Google.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#159

Post by jhbadger »

Tepix wrote: post #87
jhbadger wrote: post #33

Earlier quoted context omitted.

Given that free alternatives like Vicuna (from the University of California and CMU) are better than LLaMA, are freely and legally available for download, and are compatible with code like llama.cpp, even if every copy of LLaMA is taken down it will have no effect on the development of chatbots. It might even improve things as people who would otherwise go for the better known LLaMA will move towards these newer, bet…

They are all built on top of Llama…

Yes, but that doesn't matter mow. The University of California has released Vicuna as open source. It doesn't need the Llama model to be installed at this point. Nor do you need any of Meta's code to run it either as you can use llama.cpp (not created by Meta). That's the whole point of the article. It's open source now. There's nothing Meta can do.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#160

Post by morrbo »

Al-Khwarizmi wrote: post #158
superkuh wrote: post #118

I've spent an embarassing amount of time since the llamas leaked playing with them, the tools to run them, and writing wrappers for them. They are technically alternatives in the sense that they're incomparably better chat bots than anything in the past. But at least for the 30B and under versions (65B is too big for me to run), no matter what fine tuning is done (alpaca, gpt4all, vicuna, etc), the llamas themselves…

In my particular tests (YMMV), even 13B vicuna beats Bard, though... tough times for Google.

massive YMMV moment for me. my particular usecase was "extract the following attributes from a load of unstructured text, format the results as JSON". ChatGPT was the best (but only on 4 and Davinci), Vicuna just didn't perform at all (nor other variants of llama 7/13/33). Bard smashed it, relatively speaking, in terms of speed. I gave up pretty quickly though because of no information on pricing and/or API. It's funny how all-or-nothing these things seem to be
Post reply on HN