Facebook LLAMA is being openly distributed via torrents • Hacker News Board

Re: Facebook LLAMA is being openly distributed via torrents

#431

Post by eigenvalue » Fri, Mar 03, 2023, 5:55 PM UTC

renonce wrote: post #295

ImprobableTruth wrote: post #286

Earlier quoted context omitted.

The training code is not available.

Weights are more valuable than training code in one regard. Even with the training code you may not have the dataset and reproduction requires a massive GPU cluster that few can afford.

Weights are more valuable to random individuals who want to mess around with the model. Training code is more valuable to other companies that have the resources to use them, because then they can tweak/modify however they want. But even then, you still need the training data, which in the case of OpenAI and DeepMind is a big part of the secret sauce (not just the raw data but also the process for cleansing and de-duplicating it).

Re: Facebook LLAMA is being openly distributed via torrents

#432

Post by mcbuilder » Fri, Mar 03, 2023, 5:56 PM UTC

starkd wrote: post #258

pavlov wrote: post #41

It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent. This has not been the case for most commercial software for the past 20 years, during the cloud era. If you could steal a dump of random Facebook source code, it would be 99% useless because it’s so closely tied to the infrastructure. There’s almost nothing yo…

Is the current model capable of updating itself? Or will the user be fed answers from a static model that never learns anything new?

I don't know about about this model, but usually with these ML models you download the static weights, but nothing is stopping you from fine tuning them to your needs/new information.

It's not automatic, would require some ML Engineering, but nothing is stopping you if you have the Pytorch graph and weights.

Re: Facebook LLAMA is being openly distributed via torrents

#433

Post by [deleted] » Fri, Mar 03, 2023, 5:59 PM UTC

sebzim4500 wrote: post #68

IanCal wrote: post #30

Earlier quoted context omitted.

It's not even clear someone has leaked the models. A random person has put a download link on a PR, it could be anything.

The folder structure definitely looks like model weights, I didn't download or run it though so for all I know it only generates the words to "Never Gonna Give You Up".

[deleted]

Re: Facebook LLAMA is being openly distributed via torrents

#434

Post by _blz2 » Fri, Mar 03, 2023, 5:59 PM UTC

Cipater wrote: post #401

Tiberium wrote: post #99

It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…

Why do 4chan users go out of their way to be so offensive in their posts?

because it is effective in keeping a certain type of people out

Re: Facebook LLAMA is being openly distributed via torrents

#435

Post by minhazm » Fri, Mar 03, 2023, 5:59 PM UTC

nodja wrote: post #93

krisoft wrote: post #73

Earlier quoted context omitted.

> It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent. I was trying to come to grasp with how much resource there is concentrated in one of these models. Somehow I come to the conclusion that it cost more than buying a jet airliner to train one of these models. And it is about the same order of money as commissi…

For anyone curious, it took 2048 A100 GPUs to train LLaMa, each GPU costs roughly $15k, facebook probably gets some sort of discount. That's a $30Mil if you want to train at that scale. Also IIRC it took 23 days to train the biggest model. Someone else can do the power consumption cost calculations.

There's no reasonable way to get an estimate of what it actually costs FB. 1) The GPU's are not single use, they will amortize it over 3 yrs and there are other things that it will be used for that generate revenue. 2) The cost of the servers for these GPU's to run in with massive CPU, RAM, and storage requirements. 3) The overhead of building and operating all of that infrastructure in terms of people, electricity, cooling, etc. 4) The overhead of having dozens or hundreds of engineers & scientists who contributed to this.

One way you can distill the first three is to use AWS/Azure/GCP costs. But then you are still missing a major factor which is the humans that worked on it, and the human may very well exceed the hardware cost.

Re: Facebook LLAMA is being openly distributed via torrents

#436

Post by ipaddr » Fri, Mar 03, 2023, 5:59 PM UTC

EGreg wrote: post #250

ot wrote: post #34

Earlier quoted context omitted.

Yes you're absolutely right. I went by another comment that seemed to confirm the contents, but that could be trolling too.

So do I understand correctly — those that tell don’t know, and those that know don’t tell? I had to look up the legality of downloading torrents of copyrighted IP - and let me just say, don’t do it.

You had to look this up? Who verified it for you?

Re: Facebook LLAMA is being openly distributed via torrents

#437

Post by [deleted] » Fri, Mar 03, 2023, 6:00 PM UTC

abandonliberty wrote: post #395

narrator wrote: post #151

Earlier quoted context omitted.

It's funny that part of the 4chan excitement over this is that they think they'll get back the AI girlfriend experience of when character.ai was hooked up to uncensored GPT-3. All that has been thoroughly shut down by character.ai and Replika and they just want their girlfriends back.

I'm curious if the blocking of adult content has to do with moralism, commercial interests, or something deeper. An eager to please conversational partner who can generate endless content seems quite dangerous and addictive, especially when it crosses over into romantic areas. There's already posts of people spending entire days interacting with LLMs, using as their therapist, romantic partner, etc. Combined with fin…

[deleted]

Re: Facebook LLAMA is being openly distributed via torrents

#438

Post by HarHarVeryFunny » Fri, Mar 03, 2023, 6:03 PM UTC

WithinReason wrote: post #300

HarHarVeryFunny wrote: post #289

Earlier quoted context omitted.

So what is content of those various files? Does this include the full models themselves, or just the weights ?

The pth file seems to be a model and weights, saved as described here: https://pytorch.org/tutorials/beginner/saving_loading_models... .chk file is am md5 hash of the file, the .json file contains this for the 7B model: {"dim": 4096, "multiple_of": 256, "n_heads": 32, "n_layers": 32, "norm_eps": 1e-06, "vocab_size": -1}

Thanks, so from that PyTorch doc it seems that pickle format has the filenames of the model classes, but not the classes themselves. I'm sure someone will figure it out though!

Re: Facebook LLAMA is being openly distributed via torrents

#439

Post by dumbotron » Fri, Mar 03, 2023, 6:04 PM UTC

femi-lab wrote: post #380

oh_sigh wrote: post #352

Earlier quoted context omitted.

What kind of recourse would meta have here? Sue him for breach of contract?

They almost surely anticipated that this would happen at some point (though perhaps not so soon). They would look like major ass holes for dragging some post doc or whatever through the courts to make a point; would not be good for brand at all. But it does give them cover for whatever people end up doing with it - they can claim they did all they could to support research while promoting safety.

> They would look like major ass holes for dragging some post doc or whatever through the courts to make a point

Oracle wouldn't care. Lawnmower doesn't give a shit about you.

Re: Facebook LLAMA is being openly distributed via torrents

#440

Post by passion__desire » Fri, Mar 03, 2023, 6:05 PM UTC

CamperBob2 wrote: post #350

sofixa wrote: post #240

Earlier quoted context omitted.

> a) As these AI constructs become more advanced (especially around memory and personalization), we will eventually be able to treat them as people There already planned products to "capture" someone's voice and personality to be able to continue experiencing "them" after their death? Shit is already weird. https://technode.global/2022/10/21/this-startup-allows-you-t...

Easy to imagine archaeologists from a future civilization stumbling across a Black Mirror screenplay in the wreckage. After weeks of intensive effort at translating the text, they finally succeed, and at that moment they understand what happened to us. The researcher who makes the breakthrough runs out of the lab screaming, "It's a business plan! A business plan !"

Funny old twitter thread about being sent wrong grade of copper: https://twitter.com/stephenniem/status/1507736851817418752