Facebook LLAMA is being openly distributed via torrents • Hacker News Board

Re: Facebook LLAMA is being openly distributed via torrents

#151

Post by narrator » Fri, Mar 03, 2023, 1:30 PM UTC

Tiberium wrote: post #99

It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…

It's funny that part of the 4chan excitement over this is that they think they'll get back the AI girlfriend experience of when character.ai was hooked up to uncensored GPT-3. All that has been thoroughly shut down by character.ai and Replika and they just want their girlfriends back.

Re: Facebook LLAMA is being openly distributed via torrents

#152

Post by ithkuil » Fri, Mar 03, 2023, 1:32 PM UTC

nodja wrote: post #143

fancyfredbot wrote: post #115

Earlier quoted context omitted.

$30m training cost is too high. Amazon's p4d.24xlarge is $32.77 an hour for 8 A100 GPUs. 2048 A100 GPUs for 23 days costs $4.6m at that rate. You might even get a discount.

15k is the price to buy a GPU, not to run it, you'd have to account for electricity costs which isn't so straight forward.

The point was that if you only need to train once, then it's cheaper to rent the GPUs than to buy them. If you need to train it multiple times, then the cost of buying the GPUs is amortized among runs.

In any case the cost per run is going to be lower than 30m

Re: Facebook LLAMA is being openly distributed via torrents

#153

Post by weberer » Fri, Mar 03, 2023, 1:32 PM UTC

causi wrote: post #102

Tiberium wrote: post #99

It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…

Just a warning to readers, I would not recommend clicking 4chan links while at work.

/g/ is one of the SFW boards

Re: Facebook LLAMA is being openly distributed via torrents

#154

Post by gambiting » Fri, Mar 03, 2023, 1:33 PM UTC

koheripbal wrote: post #116

seydor wrote: post #45

Earlier quoted context omitted.

> massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent. So, like movies or software

Or a copy of Windows or Office source code.

I don't think that's right - even if you had the full source code for either of those, it's extremely unlikely you'd be able to build them on your own machine.

Re: Facebook LLAMA is being openly distributed via torrents

#155

Post by brookst » Fri, Mar 03, 2023, 1:33 PM UTC

nodja wrote: post #93

krisoft wrote: post #73

Earlier quoted context omitted.

> It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent. I was trying to come to grasp with how much resource there is concentrated in one of these models. Somehow I come to the conclusion that it cost more than buying a jet airliner to train one of these models. And it is about the same order of money as commissi…

For anyone curious, it took 2048 A100 GPUs to train LLaMa, each GPU costs roughly $15k, facebook probably gets some sort of discount. That's a $30Mil if you want to train at that scale. Also IIRC it took 23 days to train the biggest model. Someone else can do the power consumption cost calculations.

Buying the GPU lets you amortize cost over years, probably 20-30 models of this size, at least. Probably better to use cost over time as a unit.

If an A100 costs $15k and is useful for 3 years, that’s $5k/year, $425/mo. 2048 A100’s cost $870k for a month.

Re: Facebook LLAMA is being openly distributed via torrents

#156

Post by jpgvm » Fri, Mar 03, 2023, 1:34 PM UTC

Beaver117 wrote: post #20

Funny. iirc some of the big tech (I think it was Google?) use torrents internally to deploy very large images to servers. Piracy is not the only use case!

Ironically that is Facebook that used torrent for binary distribution. (no idea if it's still the case, that was a very long time ago).

Re: Facebook LLAMA is being openly distributed via torrents

#157

Post by Aissen » Fri, Mar 03, 2023, 1:35 PM UTC

It's nice that it's downloadable without filling a form (even though it should have been the default), a leak was bound to happen. The license is quite restrictive anyway: see RESTRICTIONS on https://forms.gle/jk851eBVbX1m5TAv5

Re: Facebook LLAMA is being openly distributed via torrents

#158

Post by Brian_K_White » Fri, Mar 03, 2023, 1:35 PM UTC

bayindirh wrote: post #105

justinclift wrote: post #77

Earlier quoted context omitted.

Or Linux distro's.

Or BSD distros.

These are actually trivial and silly examples. The bulk of really valuable commercial code is not self contained or portable like those.

Where is the torrent with a runnable copy of paypal, or amazon?

Re: Facebook LLAMA is being openly distributed via torrents

#159

Post by brookst » Fri, Mar 03, 2023, 1:36 PM UTC

version_five wrote: post #106

pavlov wrote: post #41

It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent. This has not been the case for most commercial software for the past 20 years, during the cloud era. If you could steal a dump of random Facebook source code, it would be 99% useless because it’s so closely tied to the infrastructure. There’s almost nothing yo…

Finding a sha256 hash with N leading zeros is basically arbitrarily computationally expensive but could be written on a piece of paper. I don't see training an ML model as an egregious example of concentrating compute power

Your SHA256 hash won’t be able to summarize text, write poems, or make up plots for books.

The crazy thing about these models is that the compute power going into them is at least somewhat reversible.

Re: Facebook LLAMA is being openly distributed via torrents

#160

Post by oefrha » Fri, Mar 03, 2023, 1:36 PM UTC

moffkalast wrote: post #109

px43 wrote: post #53

Earlier quoted context omitted.

My understanding is that weights are normally stored as pickled python blobs, which means arbitrary code execution as they are unpickled.

"They turned the model into a pickle? Funniest shit I've ever seen." But seriously, why not something more human readable and text-based if it's just weights?

Because human-readable text-based formats are really inefficient to both download and load, especially when in the hundreds of GB range. And no human cares to read billions of weights.