Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook LLAMA is being openly distributed via torrents

github.com

Post reply on HN
Print view View original

271–280 of 719 posts

Re: Facebook LLAMA is being openly distributed via torrents

#271

Post by spi »

fancyfredbot wrote: post #27

opt-175B weights are already openly available as I understand. Hugging-face also has openly available weights for a 176B parameter LLM called Bloom. Is LLAMA offering something over and above these?

opt-175B doesn't exist; the largest one is opt-66B. And, at least in the tests I've run (not with the biggest one, but only up to a dozen billion parameters), all the opt models severely underperform with respect to even much smaller models. To the point that the launch of OPT (before BLOOM) was literally advertised as "the biggest OpenSource language model released to date", because they couldn't push on much else.

BLOOM goes indeed up to 175B parameters, and is certainly better than OPT. However, at least in my specific tests, it's still significantly inferior to OpenAI models, and actually on par with a few smaller models. There's also a "newer" fine-tuned model, called BLOOMZ, but at least in my tests it's even worse. Of course, that depends a lot on what you ask the model to do...

If LLAMA can indeed match OpenAI products, and do so with much fewer parameters, then it would be really great, and I'd really like to test it. However, even if the weights are now in the wild, using them would be clearly against the user agreement, and there's no way I'm going to do that in my work time :-) so let's hope Meta will come to sense and release them with a more friendly set of terms...

Re: Facebook LLAMA is being openly distributed via torrents

#272

Post by sebzim4500 »

ok123456 wrote: post #252

Maybe this is an intentional leak to damage OpenAI. A supposedly better model by some accounts that strikes right at the heart of their business plan of selling access for $250k/year. One month of access to their service could buy a machine capable of running this leaked model. Facebook nerfs a potential upstart competitor to keep current big-tech cartel stable. Maybe this is a bit conspiratorial, but we live in the…

IMO it's way more likely that some random guy on 4chan leaked it than it being some vast conspiracy.

Re: Facebook LLAMA is being openly distributed via torrents

#274

Post by nodogoto »

Yoric wrote: post #236
londons_explore wrote: post #61

Earlier quoted context omitted.

They could contain arbitrary code... But typically do not. That means that with the right viewer application it will be trivial to know for sure. It isn't like a multi gigabyte game for example, where knowing if there is any malicious code could easily be a multi-month reverse engineering project to get to the answer of 'probably not, but we don't have time to check every byte with a fine tooth comb'

In theory, this could be done, sure. In practice, who's going to bother checking the language model? All the code that runs Stable Diffusion or other Hugging Face models that I've seen just downloads the model dynamically, then uses it without asking question. That's a pretty low-hanging supply chain attack waiting to happen, I believe.

It shouldn't be much work to verify that the file is just a set of floats.

Re: Facebook LLAMA is being openly distributed via torrents

#275

Post by electrosphere »

unethical_ban wrote: post #227

Let's say I wanted to use this for... whatever. How do I do it? I bookmarked some "AI for beginners" youtube videos. No, I'm not trolling. The jargon and the ideas around LLMs is completely foreign to me. I have no idea how they work.

I would like to know too.

Re: Facebook LLAMA is being openly distributed via torrents

#276

Post by shadowgovt »

bentcorner wrote: post #224
dmix wrote: post #183

Earlier quoted context omitted.

The Repilka subreddit became one of the weirdest places on the internet when their model got capped for adult content. https://www.reddit.com/r/replika/ Hundreds of men (and yes women) full on acting like they lost a spouse and posting constantly about it for weeks. AI is going to create some unusual social situations the general public isn't ready to grasp. And we're only in the early alpha stages.

I posit to a friend that: a) As these AI constructs become more advanced (especially around memory and personalization), we will eventually be able to treat them as people b) Some business will eventually sell an off-the-shelf product (hardware and/or software) that is an AI you can bring into your home, that you can treat as a friend, confidant and partner c) Someone will eventually lose their AI friend of many mont…

At the end of the day, the Turing Test for establishment of AI personhood is weak for two reasons.

1. We're seeing more and more systems that get very close to passing the Turing Test but fundamentally don't register to people as "People." When I was younger and learned of Searle's Chinese Room argument, I naively assumed it wasn't a thought experiment we would literally build in my lifetime.

2. Humanity has a history of treating other humans as less-than-persons, so it's naive to assume that a machine that could argue persuasively that it is an independent soul worthy of continued existence would be treated as such by a species that doesn't consistently treat its biological kin as such.

I strongly suspect AI personhood will hinge not on measures of intelligence, but on measures of empathy... Whether the machine can demonstrate its own willful independence and further come to us on our terms to advocate for / dictate the terms of its presence in human society, or whether the machine can build a critical mass of supporters / advocates / followers to protect it and guarantee its continued existence and a place in society.

Re: Facebook LLAMA is being openly distributed via torrents

#277

Post by Stagnant »

narrator wrote: post #151
Tiberium wrote: post #99

It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…

It's funny that part of the 4chan excitement over this is that they think they'll get back the AI girlfriend experience of when character.ai was hooked up to uncensored GPT-3. All that has been thoroughly shut down by character.ai and Replika and they just want their girlfriends back.

Correct me if I'm wrong but doesn't character.ai use their own model and isn't associated with OpenAI? At least I can't find any information that would claim so.

Anecdotally, as a roleplaying chat experience, char.ai seems to perform way better than anything else publicly available (doesn't get repetitive, very long memory). It also feels different to GPT3 on how it is affected by prompts.

I've just assumed that char.ai is doing its own thing as it was founded by two engineers who worked on google's LaMDA.

Re: Facebook LLAMA is being openly distributed via torrents

#278

Post by sebzim4500 »

Aissen wrote: post #157

It's nice that it's downloadable without filling a form (even though it should have been the default), a leak was bound to happen. The license is quite restrictive anyway: see RESTRICTIONS on https://forms.gle/jk851eBVbX1m5TAv5

If someone just decides to use the torrent and ignore those restrictions it might finally establish precident for if you can copyright model weights.

Re: Facebook LLAMA is being openly distributed via torrents

#279

Post by fauigerzigerk »

bentcorner wrote: post #224
dmix wrote: post #183

Earlier quoted context omitted.

The Repilka subreddit became one of the weirdest places on the internet when their model got capped for adult content. https://www.reddit.com/r/replika/ Hundreds of men (and yes women) full on acting like they lost a spouse and posting constantly about it for weeks. AI is going to create some unusual social situations the general public isn't ready to grasp. And we're only in the early alpha stages.

I posit to a friend that: a) As these AI constructs become more advanced (especially around memory and personalization), we will eventually be able to treat them as people b) Some business will eventually sell an off-the-shelf product (hardware and/or software) that is an AI you can bring into your home, that you can treat as a friend, confidant and partner c) Someone will eventually lose their AI friend of many mont…

And if we feel as if we were losing a real person, AIs will have to be treated to some degree as if they were real people (or at least pets) rather than objects.

This could be interesting, because so far the question of personhood and sentience of AIs has revolved around what they are and what they feel rather than what we feel when we interact with one of them.

Re: Facebook LLAMA is being openly distributed via torrents

#280

Post by underlines »

- how much vRAM needed to run each model parameter size?

- any inference optimization we can use similar to StableDiffusion, to bring down the vRAM requirements?

I only know about these:

- use 8bit precision

- https://github.com/bigscience-workshop/petals

- https://github.com/FMInference/FlexGen

- https://github.com/microsoft/DeepSpeed

Anything that could bring this to a 10GB 3080 or 24GB 3090 without 60s/it per token?

Post reply on HN