Live data from Hacker News

Facebook LLAMA is being openly distributed via torrents

github.com

291–300 of 719 posts

Re: Facebook LLAMA is being openly distributed via torrents

#291

It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…

Archived: https://archive.is/lGrH8

See also: https://twitter.com/Teknium1/status/1631322496388722689

Re: Facebook LLAMA is being openly distributed via torrents

#292

Earlier quoted context omitted.

> old school opensource, which is a bit surprising from meta Aren't you a cheeky lad? Metea turned out lots of open-source database systems: * RocksDB * Hive * Presto * Cassandra * Velox LFP

and, you know... React.

And ... PyTorch

Re: Facebook LLAMA is being openly distributed via torrents

#293

Earlier quoted context omitted.

I posit to a friend that: a) As these AI constructs become more advanced (especially around memory and personalization), we will eventually be able to treat them as people b) Some business will eventually sell an off-the-shelf product (hardware and/or software) that is an AI you can bring into your home, that you can treat as a friend, confidant and partner c) Someone will eventually lose their AI friend of many mont…

And if we feel as if we were losing a real person, AIs will have to be treated to some degree as if they were real people (or at least pets) rather than objects. This could be interesting, because so far the question of personhood and sentience of AIs has revolved around what they are and what they feel rather than what we feel when we interact with one of them.

Kids can feel like they're losing a real friend if they lose a stuffed animal. What's the progress on making teddy bears people?

Re: Facebook LLAMA is being openly distributed via torrents

#294

Let's say I wanted to use this for... whatever. How do I do it? I bookmarked some "AI for beginners" youtube videos. No, I'm not trolling. The jargon and the ideas around LLMs is completely foreign to me. I have no idea how they work.

clone this and point the script(s) to your downloaded model files: https://github.com/facebookresearch/llama/

Re: Facebook LLAMA is being openly distributed via torrents

#295

Earlier quoted context omitted.

It was already the most open language model in its class, given that the code for training and inference was available and it only used public data for training. For Google and OpenAIs offerings, have fun reimplementing it from descriptions in the paper (including small crucial details that they may have left out), training it for a month, and then wondering if the implementation or the training data is the reason yo…

The training code is not available.

Weights are more valuable than training code in one regard. Even with the training code you may not have the dataset and reproduction requires a massive GPU cluster that few can afford.

Re: Facebook LLAMA is being openly distributed via torrents

#296
post #64

Earlier quoted context omitted.

Yeah, their recent papers show the smaller LLAMA models outperforming the major LLMs today, and they also have bigger models. This isn't just an alternative, it's a multi order of magnitude optimization. https://aibusiness.com/meta/meta-s-llama-language-model-outp...

Can I spend $5K and run it at home? What GPU(s) do I need?

the 7B model runs on a CUDA-compatible card with 16GB of VRAM (assuming your card has 16-bit float support).

I only got the 30b model running on a 4 x Nvidia A40 setup though.

Re: Facebook LLAMA is being openly distributed via torrents

#297
post #248
post #183

Earlier quoted context omitted.

The Repilka subreddit became one of the weirdest places on the internet when their model got capped for adult content. https://www.reddit.com/r/replika/ Hundreds of men (and yes women) full on acting like they lost a spouse and posting constantly about it for weeks. AI is going to create some unusual social situations the general public isn't ready to grasp. And we're only in the early alpha stages.

Wow, I did not realize they were in that deep. It is probably good they pulled the plug on this. Better now than later. People need to realize this is messing with emotions in an unknown way.

Yeah now we can get back to normal internet usage, which definitely doesn't involve strange emotional attachments and wierd rabbit holes.

Re: Facebook LLAMA is being openly distributed via torrents

#299
post #265
post #230

Earlier quoted context omitted.

Reminds me of Ray Kurizwell's obsession with uploading brains to the cloud to get back his beloved father.

When we can upload our brains to the cloud, and you can do something with them like interacting or running the brain, then we'll all be effectively immortal. That's a pretty big deal. See the book altered carbon.

Immortal as long as someone's paying to run the instance.

Re: Facebook LLAMA is being openly distributed via torrents

#300

Earlier quoted context omitted.

Folder structure for the 2 smaller models look like this: LLAMA │ tokenizer.model │ tokenizer_checklist.chk │ ├───13B │ checklist.chk │ consolidated.00.pth │ consolidated.01.pth │ params.json │ └───7B checklist.chk consolidated.00.pth params.json

So what is content of those various files? Does this include the full models themselves, or just the weights ?

The pth file seems to be a model and weights, saved as described here:

https://pytorch.org/tutorials/beginner/saving_loading_models...

.chk file is am md5 hash of the file, the .json file contains this for the 7B model:

    {"dim": 4096, "multiple_of": 256, "n_heads": 32, "n_layers": 32, "norm_eps": 1e-06, "vocab_size": -1}
Post reply on HN