~220 GB :O That's quite big!
What do you mean "big"? fits on the average laptop :)
Facebook LLAMA is being openly distributed via torrents
91–100 of 719 posts
Re: Facebook LLAMA is being openly distributed via torrents
#92FWIW this information was already freely available via DHT scrapers like btdig [1] I think everyone at Facebook knows that torrents aren't secret and the Google form is basically a legal tool to shield them from liability while making litigation against anyone misusing the model easier. [1]: https://btdig.com/b8287ebfa04f879b048d4d4404108cf3e8014352/l...
btdig blocked in the UK and many other countries. Use a USA VPN for access.
I'm in the UK and can view that link without a VPN.
Re: Facebook LLAMA is being openly distributed via torrents
#93It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent. This has not been the case for most commercial software for the past 20 years, during the cloud era. If you could steal a dump of random Facebook source code, it would be 99% useless because it’s so closely tied to the infrastructure. There’s almost nothing yo…
> It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent. I was trying to come to grasp with how much resource there is concentrated in one of these models. Somehow I come to the conclusion that it cost more than buying a jet airliner to train one of these models. And it is about the same order of money as commissi…
For anyone curious, it took 2048 A100 GPUs to train LLaMa, each GPU costs roughly $15k, facebook probably gets some sort of discount.
That's a $30Mil if you want to train at that scale. Also IIRC it took 23 days to train the biggest model. Someone else can do the power consumption cost calculations.
Re: Facebook LLAMA is being openly distributed via torrents
#94Here is the magnet link for posterity: magnet:?xt=urn:btih:ZXXDAUWYLRUXXBHUYEMS6Q5CE5WA3LVA&dn=LLaMA
Great, now how do I run it? Do I need a GPU with over 65GB RAM?
Try this, it's for running llms that won't fit in the gpu: https://github.com/FMInference/FlexGen
Re: Facebook LLAMA is being openly distributed via torrents
#95Earlier quoted context omitted.
Hell, I'd love to be able to buy a $30k server to run these models. I think to run BLOOM required something more along the lines of a $200k server.
No need to spend $30k, use Azure or AWS.
Yep. It’s expensive to spin up an A100 80GB instance but not THAT expensive. Oracles cloud offering (first thing to show up in google search I know you probably won’t use them and it seems extra expensive) is $4.00 per hour. If you are motivated to screw around with this stuff there’s definitely options.
Re: Facebook LLAMA is being openly distributed via torrents
#96It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent. This has not been the case for most commercial software for the past 20 years, during the cloud era. If you could steal a dump of random Facebook source code, it would be 99% useless because it’s so closely tied to the infrastructure. There’s almost nothing yo…
I fail to see how this is different from other software in that regard. If you have parameters but not the network architecture, then it's not very useful.
Re: Facebook LLAMA is being openly distributed via torrents
#97Earlier quoted context omitted.
btdig blocked in the UK and many other countries. Use a USA VPN for access.
I'm in the UK and can view that link without a VPN.
Consider yourself very lucky that your ISP doesn't suck. I'm in the UK and the link won't load, due to TLS SNI filtering (Virgin Media).
Re: Facebook LLAMA is being openly distributed via torrents
#98opt-175B weights are already openly available as I understand. Hugging-face also has openly available weights for a 176B parameter LLM called Bloom. Is LLAMA offering something over and above these?
Yes, LLaMA is state of the art in several domains. The model was trained on a much larger data set than most models which is why it is higher scoring vs other models with similar numbs of parameters. This represents millions of dollars in compute time alone for the training.
This should lead to quite a lot of innovation and it’s inevitable that someone will get these working slowly on your average MacBook.
Re: Facebook LLAMA is being openly distributed via torrents
#99It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to.
[1]: https://boards.4channel.org/g/thread/91848262#p91850335
[2]: https://boards.4channel.org/g/thread/91848262#p91849717
[3]: https://boards.4channel.org/g/thread/91848262#p91849855
[3]: https://boards.4channel.org/g/thread/91848262#p91850503
Re: Facebook LLAMA is being openly distributed via torrents
#100I give it a week before we see tools for subtly watermarking your secret LLM's weights, so you can trace leaks like this later.
The original 4chan thread seems to indicate that the leaker verified that his hashes matched with another person who had access to the weights, to make sure that the weights aren't watermarked [0]