Facebook LLAMA is being openly distributed via torrents

Post by lmeyerov » Fri, Mar 03, 2023, 2:53 PM UTC

Taek wrote: post #164

nodja wrote: post #93

Earlier quoted context omitted.

For anyone curious, it took 2048 A100 GPUs to train LLaMa, each GPU costs roughly $15k, facebook probably gets some sort of discount. That's a $30Mil if you want to train at that scale. Also IIRC it took 23 days to train the biggest model. Someone else can do the power consumption cost calculations.

Electricity costs are basically irrelevant because the cards are so expensive. A100 cards consume 250w each, with datacenter overheads we will call it 1000 kilowatts for all 2048 cards. 23 days is 552 hours, or 552,000 kilowatt hours total. Most dataceneters are between 7 and 10 cents per kilowatt hour for electricity. Some are below 4. At 10 cents, that's $53,000 in electricity costs, which is nothing next to $30 mi…

Post by sourcecodeplz » Fri, Mar 03, 2023, 2:54 PM UTC

VadimPR wrote: post #114

px43 wrote: post #57

Earlier quoted context omitted.

Hell, I'd love to be able to buy a $30k server to run these models. I think to run BLOOM required something more along the lines of a $200k server.

True, and that's why there is a project that is using volunteered, distributed GPUs to run BLOOM/BLOOMZ: https://github.com/bigscience-workshop/petals , http://chat.petals.ml .

Post by winterqt » Fri, Mar 03, 2023, 2:55 PM UTC

ddtaylor wrote: post #40

FWIW this information was already freely available via DHT scrapers like btdig [1] I think everyone at Facebook knows that torrents aren't secret and the Google form is basically a legal tool to shield them from liability while making litigation against anyone misusing the model easier. [1]: https://btdig.com/b8287ebfa04f879b048d4d4404108cf3e8014352/l...

Post by yacine_ » Fri, Mar 03, 2023, 2:55 PM UTC

optimalsolver wrote: post #119

Tiberium wrote: post #100

Earlier quoted context omitted.

The original 4chan thread seems to indicate that the leaker verified that his hashes matched with another person who had access to the weights, to make sure that the weights aren't watermarked [0] 0: https://boards.4channel.org/g/thread/91848262#p91849855

The leaker accidentally doxxed themselves by adding the original download script to the torrent: https://boards.4channel.org/g/thread/91848262#p91850503

Post by trentdotexe » Fri, Mar 03, 2023, 2:56 PM UTC

px43 wrote: post #53

zb3 wrote: post #50

Earlier quoted context omitted.

But ML models generally can execute arbitrary code Is it the case if we're only talking about weights? I thought the rest is actually "open".

My understanding is that weights are normally stored as pickled python blobs, which means arbitrary code execution as they are unpickled.