Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook LLAMA is being openly distributed via torrents

github.com

Post reply on HN
Print view View original

51–60 of 719 posts

Re: Facebook LLAMA is being openly distributed via torrents

#52

Post by WithinReason »

rnosov wrote: post #2

Just to make it clear, does this torrent include model weights?

Folder structure for the 2 smaller models look like this:

    LLAMA
    │   tokenizer.model
    │   tokenizer_checklist.chk
    │
    ├───13B
    │       checklist.chk
    │       consolidated.00.pth
    │       consolidated.01.pth
    │       params.json
    │
    └───7B
            checklist.chk
            consolidated.00.pth
            params.json

Re: Facebook LLAMA is being openly distributed via torrents

#53

Post by px43 »

zb3 wrote: post #50
version_five wrote: post #36

Are there any official checksums available? I'm happy to see this, even if it's an unsanctioned stunt, because I think it's really pathetic of meta to want to gatekeep their "open" model. But ML models generally can execute arbitrary code, I'd want to make sure it's the real version at least.

But ML models generally can execute arbitrary code Is it the case if we're only talking about weights? I thought the rest is actually "open".

My understanding is that weights are normally stored as pickled python blobs, which means arbitrary code execution as they are unpickled.

Re: Facebook LLAMA is being openly distributed via torrents

#54

Post by RobotToaster »

injidup wrote: post #33
sebzim4500 wrote: post #24

Is there anything stopping anyone from using this for commercial purposes? I know that when you fill in the google form you need to agree to noncommercial use, but someone downloading this will never have agreed to that licence agreement.

I don't know. Is there anything stopping you using the latest Miley Cyrus album for commercial purposes if you downloaded it via torrent and never agreed to any licencing terms?

IANAL, but I imagine it's a legal grey area if the weights can be copyrighted? Works produced by purely mechanical means don't normally meet the threshold of originality.

Re: Facebook LLAMA is being openly distributed via torrents

#55

Post by londons_explore »

pavlov wrote: post #41

It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent. This has not been the case for most commercial software for the past 20 years, during the cloud era. If you could steal a dump of random Facebook source code, it would be 99% useless because it’s so closely tied to the infrastructure. There’s almost nothing yo…

If you were to steal a chunk of source code or a binary from meta/Google, you could probably get it running inside a few weeks effort.

Sure, the binary probably depends on a lot of internal proprietary infrastructure, but also most of that infrastructure is easy to write a mock implementation of, as long as you are happy for it to be in-ram, not multi-homed and don't need it to scale to billions of users.

Most of the binaries have a standalone mode for running on a developers PC with few/no dependencies anyway.

Re: Facebook LLAMA is being openly distributed via torrents

#56

Post by Yoric »

px43 wrote: post #53
zb3 wrote: post #50

Earlier quoted context omitted.

But ML models generally can execute arbitrary code Is it the case if we're only talking about weights? I thought the rest is actually "open".

My understanding is that weights are normally stored as pickled python blobs, which means arbitrary code execution as they are unpickled.

If it's PyTorch, it can definitely contain and execute arbitrary code.

One of the reasons I'm not a huge fan of PyTorch.

Re: Facebook LLAMA is being openly distributed via torrents

#57

Post by px43 »

kif wrote: post #37

I wonder what the memory requirements would be to run such a large model. I'd love to be able to run this model, alas my MacBook can barely run toy models.

Hell, I'd love to be able to buy a $30k server to run these models. I think to run BLOOM required something more along the lines of a $200k server.

Re: Facebook LLAMA is being openly distributed via torrents

#58

Post by londons_explore »

ddtaylor wrote: post #40

FWIW this information was already freely available via DHT scrapers like btdig [1] I think everyone at Facebook knows that torrents aren't secret and the Google form is basically a legal tool to shield them from liability while making litigation against anyone misusing the model easier. [1]: https://btdig.com/b8287ebfa04f879b048d4d4404108cf3e8014352/l...

btdig blocked in the UK and many other countries. Use a USA VPN for access.

Re: Facebook LLAMA is being openly distributed via torrents

#60

Post by eigenvalue »

I'm not surprised-- I recently suggested that someone might try to pull an Aaron Swartz with the LLAMA weights (i.e., release them in an uncontrolled way similar to how Aaron attempted to release the JSTOR database). It's quite misleading for FB to claim that they are being so open, but then hoard the weights and only release it to a few academics. If the paper is to be believed, this is a major development, allowing you to get close to GPT3 performance on a single GPU (at least for inference on the smallest model). Clearly some renegade academic feels the same way.
Post reply on HN