Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook LLAMA is being openly distributed via torrents

github.com

Post reply on HN
Print view View original

211–220 of 719 posts

Re: Facebook LLAMA is being openly distributed via torrents

#211

Post by ot »

ithkuil wrote: post #173
ot wrote: post #16

In case it's not clear what's happening here (and from the comments it doesn't seem like it is), someone (not Meta) leaked the models and had the brilliant idea of advertising the magnet link through a GitHub pull request. The part about saving bandwidth is a joke. Meta employees may have not noticed or are still figuring out how to react, so the PR is still up. (Disclaimer: I work at Meta, but have no relationship w…

> and have no internal information on this But you just said it was not Meta. Is that based on internal information? ;-)

As I explained in another comment, if the author was a Meta employee, they wouldn't have had to sign the CLA. It's all in the PR.

Re: Facebook LLAMA is being openly distributed via torrents

#212

Post by dmix »

mysterydip wrote: post #206
dmix wrote: post #183

Earlier quoted context omitted.

The Repilka subreddit became one of the weirdest places on the internet when their model got capped for adult content. https://www.reddit.com/r/replika/ Hundreds of men (and yes women) full on acting like they lost a spouse and posting constantly about it for weeks. AI is going to create some unusual social situations the general public isn't ready to grasp. And we're only in the early alpha stages.

> ELIZA's (1966) creator, Weizenbaum, intended the program as a method to explore communication between humans and machines. He was surprised, and shocked, that individuals attributed human-like feelings to the computer program, including Weizenbaum's secretary. (citations and further info in the wikipedia article https://en.m.wikipedia.org/wiki/ELIZA )

> Some of ELIZA's responses were so convincing that Weizenbaum and several others have anecdotes of users becoming emotionally attached to the program, occasionally forgetting that they were conversing with a computer. Weizenbaum's own secretary reportedly asked Weizenbaum to leave the room so that she and ELIZA could have a real conversation. Weizenbaum was surprised by this, later writing: "I had not realized ... that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people."

The cliché virtual girlfriend stereotype is a young Japanese 'Herbivore' male but I wouldn't be surprised if women become the biggest consumer of AI chatbots for romantic purposes. Romance novels are a major market and women stereotypically were more inclined to doing the written love letter thing. Although reading the Repilka rants a lot of it was quite male-driven pornographic stuff too.

Re: Facebook LLAMA is being openly distributed via torrents

#213

Post by pas »

troad wrote: post #148
pas wrote: post #128

Earlier quoted context omitted.

... how is that a good test of anything? I mean what if I click on a /b/ link "at work"? Does that make my work output immediately tainted and the company has to immediately file for bankruptcy?

> Does that make my work output immediately tainted and the company has to immediately file for bankruptcy? No, much easier just to fire you. You’ve shown a disregard for what is almost certainly company policy, and created legal risks for the company (e.g. if anyone walked in on you while your screen was showing naked people).

Choosing the right workplace (and thus boss) is important :)

> walked in on you while your screen was showing naked people

Why are they poking their eyes onto my screen?

Of course the underlying rule "making other coworkers uncomfortable is bad" completely makes sense. And we - probably - all know how above a certain company size these rules end up playing it too safe.

Re: Facebook LLAMA is being openly distributed via torrents

#214

Post by gnramires »

wincy wrote: post #95
DeathArrow wrote: post #63

Earlier quoted context omitted.

No need to spend $30k, use Azure or AWS.

Yep. It’s expensive to spin up an A100 80GB instance but not THAT expensive. Oracles cloud offering (first thing to show up in google search I know you probably won’t use them and it seems extra expensive) is $4.00 per hour. If you are motivated to screw around with this stuff there’s definitely options.

> If you are motivated to screw around with this stuff there’s definitely options.

Erm, for inference that is. Training is definitely out of question for individuals I believe (unless you use much smaller models?).

Re: Facebook LLAMA is being openly distributed via torrents

#215

Post by px43 »

koheripbal wrote: post #130
px43 wrote: post #64

Earlier quoted context omitted.

Yeah, their recent papers show the smaller LLAMA models outperforming the major LLMs today, and they also have bigger models. This isn't just an alternative, it's a multi order of magnitude optimization. https://aibusiness.com/meta/meta-s-llama-language-model-outp...

Can I spend $5K and run it at home? What GPU(s) do I need?

As the models proliferate, I guess we'll be finding out soon. The torrent has been going pretty slow for me for the past couple hours, but it looks like there are a couple seeders, so eventually it'll hit that inflection point where there are enough seeders to give all the leechers full speed downloads.

Looking forward to the YouTube videos of random tinkerers seeing what sort of performance they can squeeze out of cheaper hardware.

Re: Facebook LLAMA is being openly distributed via torrents

#216

Post by Godel_unicode »

fancyfredbot wrote: post #115
nodja wrote: post #93

Earlier quoted context omitted.

For anyone curious, it took 2048 A100 GPUs to train LLaMa, each GPU costs roughly $15k, facebook probably gets some sort of discount. That's a $30Mil if you want to train at that scale. Also IIRC it took 23 days to train the biggest model. Someone else can do the power consumption cost calculations.

$30m training cost is too high. Amazon's p4d.24xlarge is $32.77 an hour for 8 A100 GPUs. 2048 A100 GPUs for 23 days costs $4.6m at that rate. You might even get a discount.

That’s something like a 50% premium over Azure, even before reservation discounts, that’s insane!

Edit: todays pricing looks like about 20% higher, still. How are these prices so different.

Re: Facebook LLAMA is being openly distributed via torrents

#217

Post by cma »

colinmorelli wrote: post #88
RobotToaster wrote: post #54

Earlier quoted context omitted.

IANAL, but I imagine it's a legal grey area if the weights can be copyrighted? Works produced by purely mechanical means don't normally meet the threshold of originality.

I am quite confident that using this model for commercial purposes will, if detected, land you in quite a legal quagmire that almost certainly sides in favor of Meta. And even if it did not, Meta certainly has a more capable legal team with more cash to spend than the average HN user.

I don't think weights can be copyrighted (unless overfit on your own other copywritten work), and I don't think weights shared broadly with the research community can be considered trade secrets. And even if they were trade secrets despite the wide sharing, only the people that leaked them could get into trouble, right? They aren't trade secrets anymore once you didn't keep them secret and they are out on torrents.

Re: Facebook LLAMA is being openly distributed via torrents

#218

Post by dmix »

espadrine wrote: post #38
ot wrote: post #16

In case it's not clear what's happening here (and from the comments it doesn't seem like it is), someone (not Meta) leaked the models and had the brilliant idea of advertising the magnet link through a GitHub pull request. The part about saving bandwidth is a joke. Meta employees may have not noticed or are still figuring out how to react, so the PR is still up. (Disclaimer: I work at Meta, but have no relationship w…

> Meta employees may have not noticed or are still figuring out how to react Given that the cat is out of the bag, if I were them, I would say that it is now publicly downloadable under the terms listed in the form. It is great PR, which if this was unintentional, is a positive outcome out of a bad situation.

Facebook fumbling it's way into being the better open source AI than OpenAI would be amusing.

Re: Facebook LLAMA is being openly distributed via torrents

#219

Post by bilekas »

dmix wrote: post #183
narrator wrote: post #151

Earlier quoted context omitted.

It's funny that part of the 4chan excitement over this is that they think they'll get back the AI girlfriend experience of when character.ai was hooked up to uncensored GPT-3. All that has been thoroughly shut down by character.ai and Replika and they just want their girlfriends back.

The Repilka subreddit became one of the weirdest places on the internet when their model got capped for adult content. https://www.reddit.com/r/replika/ Hundreds of men (and yes women) full on acting like they lost a spouse and posting constantly about it for weeks. AI is going to create some unusual social situations the general public isn't ready to grasp. And we're only in the early alpha stages.

I really wish I didn't read some of those.. Just makes me feel we as a whole are not ready for any kind of AI, let alone GI.

Re: Facebook LLAMA is being openly distributed via torrents

#220

Post by archon1410 »

Tiberium wrote: post #99

It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…

It has only just occurred to me that 4chan's technology board is /g/ because it's tech-naw-la-G
Post reply on HN