The smallest model (7B) is supposed to outperform GPT-3. Does anyone have any idea what hardware is needed to run this?
7B would require at least 14GB VRAM in 8 bit precision. 28GB in 16 bit precision.
Facebook LLAMA is being openly distributed via torrents
461–470 of 719 posts
Re: Facebook LLAMA is being openly distributed via torrents
#462Earlier quoted context omitted.
Why do 4chan users go out of their way to be so offensive in their posts?
Because the bump system combined with the finite number of threads incentivizes threads that get the highest number of replies per second. And the best way to increase replies per second is to start an internet fight.
And there is no voting, so the only possible gratification for posting is receiving a reply.
Re: Facebook LLAMA is being openly distributed via torrents
#463It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…
It's funny that part of the 4chan excitement over this is that they think they'll get back the AI girlfriend experience of when character.ai was hooked up to uncensored GPT-3. All that has been thoroughly shut down by character.ai and Replika and they just want their girlfriends back.
Oh, they will. And they'll exceed it.
Look at what fueled SD's ultimate K.O. of DALL-E 2: extremely high-quality custom-tailored porn images, one sentence away. The top models on civitai are all about it.
Re: Facebook LLAMA is being openly distributed via torrents
#464It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…
Why do 4chan users go out of their way to be so offensive in their posts?
I think because you can't be anywhere else on the Internet anymore. It's like the system's pressure relief valve. A blaring steam whistle that's only getting worse and worse the more the Internet squeezes elsewhere.
Re: Facebook LLAMA is being openly distributed via torrents
#465- how much vRAM needed to run each model parameter size? - any inference optimization we can use similar to StableDiffusion, to bring down the vRAM requirements? I only know about these: - use 8bit precision - https://github.com/bigscience-workshop/petals - https://github.com/FMInference/FlexGen - https://github.com/microsoft/DeepSpeed Anything that could bring this to a 10GB 3080 or 24GB 3090 without 60s/it per toke…
The checkpoint for the 7B parameter model is 13.5GB, so maybe? Larger models are multiple chunks at 13.6GB each or 16.3GB each. I am hoping I will be able to run on my 16GB Vram but I don't know how much overhead is needed. Maybe people on reddit will do their tricks and squeeze the models in to smaller cards.
EDIT: There seems to be a lot of overhead. Here someone struggles to fit the 7B parameter model (13.5GB checkpoint) on a 24GB card: https://github.com/facebookresearch/llama/issues/55
EDIT2: actually using the tip in that link I got it to run on a P5000 with 16GB of ram! It just barely fits so I had to log out of gnome and run it terminal only.
Re: Facebook LLAMA is being openly distributed via torrents
#466Earlier quoted context omitted.
It's funny that part of the 4chan excitement over this is that they think they'll get back the AI girlfriend experience of when character.ai was hooked up to uncensored GPT-3. All that has been thoroughly shut down by character.ai and Replika and they just want their girlfriends back.
The Repilka subreddit became one of the weirdest places on the internet when their model got capped for adult content. https://www.reddit.com/r/replika/ Hundreds of men (and yes women) full on acting like they lost a spouse and posting constantly about it for weeks. AI is going to create some unusual social situations the general public isn't ready to grasp. And we're only in the early alpha stages.
Someone could make a lot of money (most likely notoriety) making an AI supplemented Katawa Shoujo expansion for 4chan
Re: Facebook LLAMA is being openly distributed via torrents
#467Earlier quoted context omitted.
If I may tack on a question as someone with zero clue of ML: when, if ever, will someone like me be able to run this on a Mac Studio with a M1 Ultra and 128GB of ram?
As far as I can tell you can do it right now, at least for small 13B model, not sure about bigger models.
The 13B parameter model is two 13.5GB chunks, but the 7B parameter model is one 13.5GB chunk so that one might be possible.
EDIT: There seems to be a lot of overhead. Here someone struggles to fit the 7B parameter model (13.5GB checkpoint) on a 24GB card: https://github.com/facebookresearch/llama/issues/55
EDIT2: actually using the tip in that link I got it to run on a P5000 with 16GB of ram! It just barely fits so I had to log out of gnome and run it terminal only.
Re: Facebook LLAMA is being openly distributed via torrents
#468Earlier quoted context omitted.
Sure I'll download TeamMysticAvengers-meta-llm-x-cars-movie-model-x-angelina-jolie-naked-xxx-2023.zip.exe.torrent
Just a sec, need to find the crack on astalavista
this, and the soundtrack in the sibling reply is giving me fantastic series of flashbacks. thank you
Re: Facebook LLAMA is being openly distributed via torrents
#469Earlier quoted context omitted.
I posit to a friend that: a) As these AI constructs become more advanced (especially around memory and personalization), we will eventually be able to treat them as people b) Some business will eventually sell an off-the-shelf product (hardware and/or software) that is an AI you can bring into your home, that you can treat as a friend, confidant and partner c) Someone will eventually lose their AI friend of many mont…
At the end of the day, the Turing Test for establishment of AI personhood is weak for two reasons. 1. We're seeing more and more systems that get very close to passing the Turing Test but fundamentally don't register to people as "People." When I was younger and learned of Searle's Chinese Room argument, I naively assumed it wasn't a thought experiment we would literally build in my lifetime. 2. Humanity has a histor…
The way people informally talk about "passing a Turing test" is a weak test, but the original imitation game isn't if the players are skilled. It's not "acting like a human". It's more like playing the Werewolf party game.
Alice and Bob want to communicate, but the bot is attempting to impersonate Bob. Can Alice authenticate Bob?
This depends on what sort of shared secrets they have. Obviously, if they agreed ahead of time on a shared password and counter-password then the computer couldn't do it. If they, like, went to the same high school then the bot couldn't do it, unless the bot also knew what went on at that school.
So we need to assume Alice and Bob don't know each other and don't cheat. But, if they had nothing in common (like they don't even speak the same language) then they would find it very hard to win. There needs to be some sort of shared culture. How much?
Let's say there is a pool of players who come from the same country, but don't know each other and have played the game before. Then they can try to find a subject in common that they don't think the bot is good at. The first thing you do is talk about common interests with each player and find something you don't think bots can do. Like if they're both mathematicians then talk about math, or they're both cooks than talk about cooking.
If the players are skilled and you're playing to win then this is a difficult game for a bot.
Re: Facebook LLAMA is being openly distributed via torrents
#470The smallest model (7B) is supposed to outperform GPT-3. Does anyone have any idea what hardware is needed to run this?
Supposedly double the model size so 14gb. RTX 4090 might be able to handle it. You can use lambdalabs to rent a server gpu for one of the larger models.