Facebook LLAMA is being openly distributed via torrents • Hacker News Board

Re: Facebook LLAMA is being openly distributed via torrents

#521

Post by rvnx » Fri, Mar 03, 2023, 8:49 PM UTC

VHRanger wrote: post #446

jedberg wrote: post #406

Earlier quoted context omitted.

I think you're being naive. Microsoft has spent 1000s of engineering years on their build system. You aren't going to just replicate that in a week.

As a user said below, you wouldn't need cross platform support, incremental builds, tests to run, etc. Just getting the code to compile, link, baking in assets, etc. For a single architecture is a much more reasonable goal. I'd imagine it'd be on the scale of 1-3months for an engineer to get working full time, but large error bars around this figure

Plus it already has been done before with Windows XP, without even any documentation and there is a guide on the internet on how to build Windows Server 2003.

This was done (if I remember right) when governments and big customers had access to the Windows source-code.

Re: Facebook LLAMA is being openly distributed via torrents

#522

Post by skissane » Fri, Mar 03, 2023, 8:49 PM UTC

screye wrote: post #473

abandonliberty wrote: post #395

Earlier quoted context omitted.

I'm curious if the blocking of adult content has to do with moralism, commercial interests, or something deeper. An eager to please conversational partner who can generate endless content seems quite dangerous and addictive, especially when it crosses over into romantic areas. There's already posts of people spending entire days interacting with LLMs, using as their therapist, romantic partner, etc. Combined with fin…

Wall-E humans are going to be reality. That last century has already proven that humans cannot be expected to responsibly indulge in Gluttony, Sloth or Lust. Now these models can skip the material desires and trigger permanent hormone releases through perfectly personalized content. I genuinely fear that the breakdown of millennia old social structures that kept us human might lead to a temporary (century long) turmo…

> I genuinely fear that the breakdown of millennia old social structures that kept us human might lead to a temporary (century long) turmoil for individuals. The answers to the 'meaning of life' and 'what makes us human' are going to change. And we will never be the same again.

Meanwhile, the Amish and the ultra-Orthodox Jews are going to refuse to talk to AIs - it’s a sin - and will go on having lots of kids, just like humanity always has, while the AI-addicts will be too addicted to bother having any at all. Maybe the future of the human race will be the people who reject AI rather than those who succumb to its charms

Re: Facebook LLAMA is being openly distributed via torrents

#523

Post by sangnoir » Fri, Mar 03, 2023, 8:52 PM UTC

shagie wrote: post #411

ben_w wrote: post #368

Earlier quoted context omitted.

If AI are sentient and we think they aren't… the term “zombie” was created by slaves in the Caribbean who were afraid that even death would not free them from their servitude. This would be the genuine existence of AI which were conscious but which we denied. If we have the opposite scenario in both details, where we think AI are sentient when they're not… at some point, brain scans and uploads will be a thing and th…

Tangentially, the "zombie" is part of philosophy that is applicable here. https://en.wikipedia.org/wiki/Philosophical_zombie > A philosophical zombie or p-zombie argument is a thought experiment in philosophy of mind that imagines a hypothetical being that is physically identical to and indistinguishable from a normal person but does not have conscious experience, qualia, or sentience. For example, if a philosophical…

> Relatedly, a zombie world is a hypothetical world indistinguishable from our world but in which all beings lack conscious experience

I find such solipsism pointless - you can't differential the zombie world from this one: how do you prove you are not the only conscious person that ever existed and everyone else is, and was a p-zombie?

Re: Facebook LLAMA is being openly distributed via torrents

#524

Post by orangepurple » Fri, Mar 03, 2023, 8:53 PM UTC

Cipater wrote: post #401

Tiberium wrote: post #99

It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…

Why do 4chan users go out of their way to be so offensive in their posts?

The same reason Penicillium molds produce β-lactam antibiotics. There doesn't have to be an intelligent reason, just a survival trick.

Re: Facebook LLAMA is being openly distributed via torrents

#525

Post by [deleted] » Fri, Mar 03, 2023, 8:53 PM UTC

skybrian wrote: post #469

shadowgovt wrote: post #276

Earlier quoted context omitted.

At the end of the day, the Turing Test for establishment of AI personhood is weak for two reasons. 1. We're seeing more and more systems that get very close to passing the Turing Test but fundamentally don't register to people as "People." When I was younger and learned of Searle's Chinese Room argument, I naively assumed it wasn't a thought experiment we would literally build in my lifetime. 2. Humanity has a histor…

The way people informally talk about "passing a Turing test" is a weak test, but the original imitation game isn't if the players are skilled. It's not "acting like a human". It's more like playing the Werewolf party game. Alice and Bob want to communicate, but the bot is attempting to impersonate Bob. Can Alice authenticate Bob? This depends on what sort of shared secrets they have. Obviously, if they agreed ahead o…

[deleted]

Re: Facebook LLAMA is being openly distributed via torrents

#526

Post by TaylorAlexander » Fri, Mar 03, 2023, 8:59 PM UTC

version_five wrote: post #36

Are there any official checksums available? I'm happy to see this, even if it's an unsanctioned stunt, because I think it's really pathetic of meta to want to gatekeep their "open" model. But ML models generally can execute arbitrary code, I'd want to make sure it's the real version at least.

I am running it in docker to be safe, which works just fine.

Re: Facebook LLAMA is being openly distributed via torrents

#527

Post by alfalfasprout » Fri, Mar 03, 2023, 9:02 PM UTC

Warning: do not use this for commercial purposes. While the weights may be available now, it's a lawsuit waiting to happen if you try to use this at work.

See the original license: "a. Subject to your compliance with the Documentation and Sections 2, 3, and 5, Meta grants you a non-exclusive, worldwide, non-transferable, non-sublicensable, revocable, royalty free and limited license under Meta’s copyright interests to reproduce, distribute, and create derivative works of the Software solely for your non-commercial research purposes. The foregoing license is personal to you, and you may not assign or sublicense this License or any other rights or obligations under this License without Meta’s prior written consent; any such assignment or sublicense will be void and will automatically and immediately terminate this License."

Re: Facebook LLAMA is being openly distributed via torrents

#528

Post by MrStonedOne » Fri, Mar 03, 2023, 9:05 PM UTC

Cipater wrote: post #401

Tiberium wrote: post #99

It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…

Why do 4chan users go out of their way to be so offensive in their posts?

[dead]

Re: Facebook LLAMA is being openly distributed via torrents

#529

Post by TaylorAlexander » Fri, Mar 03, 2023, 9:06 PM UTC

TaylorAlexander wrote: post #465

underlines wrote: post #280

- how much vRAM needed to run each model parameter size? - any inference optimization we can use similar to StableDiffusion, to bring down the vRAM requirements? I only know about these: - use 8bit precision - https://github.com/bigscience-workshop/petals - https://github.com/FMInference/FlexGen - https://github.com/microsoft/DeepSpeed Anything that could bring this to a 10GB 3080 or 24GB 3090 without 60s/it per toke…

The checkpoint for the 7B parameter model is 13.5GB, so maybe? Larger models are multiple chunks at 13.6GB each or 16.3GB each. I am hoping I will be able to run on my 16GB Vram but I don't know how much overhead is needed. Maybe people on reddit will do their tricks and squeeze the models in to smaller cards. EDIT: There seems to be a lot of overhead. Here someone struggles to fit the 7B parameter model (13.5GB chec…

Following up. After rebooting in to GUI that was enough to get it to fit, I guess xorg just accumulated some cruft in my last boot. So I can run it alongside gnome.

nvidia-smi reports this model is using 15475MiB after changing the max batch size from 32 to 8 (see link in above post)

As others have stated someone may have injected unknown code in to the pickled checkpoint, so I recommend running this in docker. I use this command to run the docker image after getting nvidia docker stuff configured.

docker run --runtime=nvidia -it --mount type=bind,source=/MY_LLAMA_SOURCE_PATH,target=/llama --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04

Then install the necessary dependencies in that container (obv could make a dockerfile), stick your model as well as tokenizer files (from the root dir in download) in to some directory (here models/LLaMA-7B) and run this:

torchrun example.py --ckpt_dir ./models/LLaMA-7B --tokenizer_path ./models/LLaMA-7B/tokenizer.model

enjoy! the 7B parameter model is not amazing from my initial testing. I am very curious if larger models can be run on 1 GPU by sequentially loading checkpoints. I don't know how all this stuff is organized.

Example output below.

Prompt:

Please respond to this question: As a large language model, what are three things that you find most important?

Output:

To get students to analyze what they are doing in their learning

To get students to analyze what they are doing in their learning so that they can find the best practices

To get students to analyze what they are doing in their learning so that they can find the best practices and then to make changes to those practices so that they can improve their learning

I don't think this question really made much sense because the sentence in the question is incomplete

To get students to analyze what they are doing in their learning so that they can find the best practices and then to make changes to those practices so that they can improve their learning

I don't think this question really made much sense because the sentence in the question is incomplete.

I think this question is in fact a good question, even though the initial sentence is incomplete, and I do think it makes sense.

I think this question is in fact a good question, even though the initial sentence is incomplete, and I do think it makes sense. (I think it is a good question but I am not sure it makes sense).

Re: Facebook LLAMA is being openly distributed via torrents

#530

Post by tekla » Fri, Mar 03, 2023, 9:07 PM UTC

Cipater wrote: post #401

Tiberium wrote: post #99

It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…

Why do 4chan users go out of their way to be so offensive in their posts?

It keeps people out that are unable to separate the internet from real life.