Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook LLAMA is being openly distributed via torrents

github.com

Post reply on HN
Print view View original

641–650 of 719 posts

Re: Facebook LLAMA is being openly distributed via torrents

#641

Post by grrowl »

flangola7 wrote: post #535
alfalfasprout wrote: post #527

Warning: do not use this for commercial purposes . While the weights may be available now, it's a lawsuit waiting to happen if you try to use this at work. See the original license: "a. Subject to your compliance with the Documentation and Sections 2, 3, and 5, Meta grants you a non-exclusive, worldwide, non-transferable, non-sublicensable, revocable, royalty free and limited license under Meta’s copyright interests…

And where did I sign my name to that agreement?

This might be a bit of an assumption, but it seems likely Meta is willing to lose more on lawyers than you'd be willing to ever spend.

Re: Facebook LLAMA is being openly distributed via torrents

#642

Post by koheripbal »

arthurcolle wrote: post #627
q1w2 wrote: post #304

Earlier quoted context omitted.

I wonder how many people are scrambling to set this up on their startup infra. 6x24GB NVRAM on 6 GPUs linked with NVSwitch is a little pricey, but totally doable.

I got it running using Colab Pro+ (immediately got a V100 40GB VRAM GPU) - the 7B model works with batch size of 8 and a max seq len of 1024

Sure, but the real value here is the 65B. Can you have multiple GPUs on colab?

Re: Facebook LLAMA is being openly distributed via torrents

#643

Post by fareesh »

WithinReason wrote: post #589
fareesh wrote: post #581

Earlier quoted context omitted.

What GPU is it and how long did it take?

4090, about 10 seconds to load the weights and another 15 seconds to generate all the completions from the example script

Yikes, thanks

I have a 2060 and I am too afraid and poor to buy a 4090 after import duties and taxes in a tropical country

Re: Facebook LLAMA is being openly distributed via torrents

#644

Post by bick_nyers »

flangola7 wrote: post #543
bick_nyers wrote: post #412

Earlier quoted context omitted.

Why 3 words per minute as opposed to second? Is that a typo? If you have enough RAM (but not VRAM), does it basically become limited by the PCIE lanes? So for the 112GB model with a Gen 5 GPU (64 GB/s PCIE bandwidth) that would be roughly 2 seconds per word right?

If read is 1GB/s then it takes 20s to infer across a 20GB model. That's 3 tokens a minute.

Yeah I'm not sure what kind of math I was subscribed to yesterday, thanks

Re: Facebook LLAMA is being openly distributed via torrents

#645

Post by thisisnotadr1ll »

shadowgovt wrote: post #276
bentcorner wrote: post #224

Earlier quoted context omitted.

I posit to a friend that: a) As these AI constructs become more advanced (especially around memory and personalization), we will eventually be able to treat them as people b) Some business will eventually sell an off-the-shelf product (hardware and/or software) that is an AI you can bring into your home, that you can treat as a friend, confidant and partner c) Someone will eventually lose their AI friend of many mont…

At the end of the day, the Turing Test for establishment of AI personhood is weak for two reasons. 1. We're seeing more and more systems that get very close to passing the Turing Test but fundamentally don't register to people as "People." When I was younger and learned of Searle's Chinese Room argument, I naively assumed it wasn't a thought experiment we would literally build in my lifetime. 2. Humanity has a histor…

here's a spicy take: maybe the Turing test was always going to end up being the evaluation of the evaluator. much like nobody is really bringing up the providence of stylometry, kinaesthetics, & NLP embeddings as precursors to the next generation of IQ test (which is likely to be as obsolete as the Turing test).

There's plenty of pathology for PC vs NPC mindsets. Nobody is going to think their conversational partner is the main character of their story. There's just a popcorn-worthy cultural shift about the blackbox having the empathy or intelligence to satisfy the main character/ epic hero trope, and the resulting conflict of words & other things to resist the blackbox from having enough resources to iterate the trope past human definition.

Re: Facebook LLAMA is being openly distributed via torrents

#646

Post by ok123456 »

tmalsburg2 wrote: post #520
ok123456 wrote: post #252

Maybe this is an intentional leak to damage OpenAI. A supposedly better model by some accounts that strikes right at the heart of their business plan of selling access for $250k/year. One month of access to their service could buy a machine capable of running this leaked model. Facebook nerfs a potential upstart competitor to keep current big-tech cartel stable. Maybe this is a bit conspiratorial, but we live in the…

Why leak it instead of just publishing it along with a press release about openness and democratizing AI and so on?

because then you don't need to explain to shareholders why you're giving away something that could potentially be worth a few hundred million dollars.

Re: Facebook LLAMA is being openly distributed via torrents

#647

Post by arthurcolle »

koheripbal wrote: post #642
arthurcolle wrote: post #627

Earlier quoted context omitted.

I got it running using Colab Pro+ (immediately got a V100 40GB VRAM GPU) - the 7B model works with batch size of 8 and a max seq len of 1024

Sure, but the real value here is the 65B. Can you have multiple GPUs on colab?

I can't even get the 13B on colab to do inference with a very small sequence length.

Re: Facebook LLAMA is being openly distributed via torrents

#648

Post by Red_Leaves_Flyy »

TaylorAlexander wrote: post #526
version_five wrote: post #36

Are there any official checksums available? I'm happy to see this, even if it's an unsanctioned stunt, because I think it's really pathetic of meta to want to gatekeep their "open" model. But ML models generally can execute arbitrary code, I'd want to make sure it's the real version at least.

I am running it in docker to be safe, which works just fine.

Docker escapes exist and if this was released by spooks then including sandbox escapes is par. Unlikely for sure but your confidence is naïve.

Re: Facebook LLAMA is being openly distributed via torrents

#649

Post by sebzim4500 »

dougmwne wrote: post #585
charcircuit wrote: post #532

Earlier quoted context omitted.

that would be a derivative work

That’s a legal unknown. And it’s also a technical unknown how you would even determine it was descended from the same model in a way that would hold up in court.

I would imagine that the weights of a finetuned model are highly correlated with the original weights. Having said that, simply permuting the neurons would make it way harder to match them up, I can't think of a straightforward way to reverse it.

Re: Facebook LLAMA is being openly distributed via torrents

#650

Post by jiggawatts »

bentcorner wrote: post #224
dmix wrote: post #183

Earlier quoted context omitted.

The Repilka subreddit became one of the weirdest places on the internet when their model got capped for adult content. https://www.reddit.com/r/replika/ Hundreds of men (and yes women) full on acting like they lost a spouse and posting constantly about it for weeks. AI is going to create some unusual social situations the general public isn't ready to grasp. And we're only in the early alpha stages.

I posit to a friend that: a) As these AI constructs become more advanced (especially around memory and personalization), we will eventually be able to treat them as people b) Some business will eventually sell an off-the-shelf product (hardware and/or software) that is an AI you can bring into your home, that you can treat as a friend, confidant and partner c) Someone will eventually lose their AI friend of many mont…

That’s a core plot element of the new Bladerunner movie. Seems less like science fiction with every passing day.
Post reply on HN