Here is the magnet link for posterity: magnet:?xt=urn:btih:ZXXDAUWYLRUXXBHUYEMS6Q5CE5WA3LVA&dn=LLaMA
Great, now how do I run it? Do I need a GPU with over 65GB RAM?
Nope, more like 111gb
Facebook LLAMA is being openly distributed via torrents
121–130 of 719 posts
Re: Facebook LLAMA is being openly distributed via torrents
#122Earlier quoted context omitted.
Just a warning to readers, I would not recommend clicking 4chan links while at work.
Fortune favors the brave
Personally I apply the "how would I feel about this page being printed out and laying on my boss's desk" test to every site I visit at work.
Re: Facebook LLAMA is being openly distributed via torrents
#123Hypothetically, what would the consequences be if I ran this on my university's computing cluster?
Re: Facebook LLAMA is being openly distributed via torrents
#124It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…
Just a warning to readers, I would not recommend clicking 4chan links while at work.
magnet:?xt=urn:btih:ZXXDAUWYLRUXXBHUYEMS6Q5CE5WA3LVA&dn=LLaMA
Re: Facebook LLAMA is being openly distributed via torrents
#125Earlier quoted context omitted.
Fortune favors the brave
Personally I apply the "how would I feel about this page being printed out and laying on my boss's desk" test to every site I visit at work.
I would feel like that's a waste of paper. They could have pulled it up on a laptop or tablet.
Re: Facebook LLAMA is being openly distributed via torrents
#126Earlier quoted context omitted.
With code modifications, it should be possible to run this with a very modest machine as long as you're happy for performance to suck. Transformer models typically need to read all the weights per 'word' output, so if your model is 20GB and you have not enough ram or vram, but have an SSD that reads 1GB/sec, expect 3 words per minute output speed. However, code changes are necessary to achieve that, although they won…
The most time/cost optimal solution is probably to buy 32 or 64 gigs of ram. That'll still be slow but most people are already half way there.
That will be very VERY slow. Pcie bandwidth is way too slow.
Re: Facebook LLAMA is being openly distributed via torrents
#127I wonder what the memory requirements would be to run such a large model. I'd love to be able to run this model, alas my MacBook can barely run toy models.
Hell, I'd love to be able to buy a $30k server to run these models. I think to run BLOOM required something more along the lines of a $200k server.
you can - slowly - run Bloom 3b and 7b1 on the free (trial) tiers of Google Cloud Compute if you use the low_cpu_mem_usage parameter of from_pretrained
Re: Facebook LLAMA is being openly distributed via torrents
#128Earlier quoted context omitted.
Fortune favors the brave
Personally I apply the "how would I feel about this page being printed out and laying on my boss's desk" test to every site I visit at work.
... how is that a good test of anything?
I mean what if I click on a /b/ link "at work"? Does that make my work output immediately tainted and the company has to immediately file for bankruptcy?
Re: Facebook LLAMA is being openly distributed via torrents
#129Earlier quoted context omitted.
It’s not the power they’re referring to, it’s the density of information and effort that went into its creation.
But these ML models are like neutron stars of computation density I guess I interpreted that differently
"The osmium of computational density" just doesn't have the same ring to it.
Re: Facebook LLAMA is being openly distributed via torrents
#130opt-175B weights are already openly available as I understand. Hugging-face also has openly available weights for a 176B parameter LLM called Bloom. Is LLAMA offering something over and above these?
Yeah, their recent papers show the smaller LLAMA models outperforming the major LLMs today, and they also have bigger models. This isn't just an alternative, it's a multi order of magnitude optimization. https://aibusiness.com/meta/meta-s-llama-language-model-outp...
Can I spend $5K and run it at home? What GPU(s) do I need?