Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook LLAMA is being openly distributed via torrents

github.com

Post reply on HN
Print view View original

501–510 of 719 posts

Re: Facebook LLAMA is being openly distributed via torrents

#501

Post by slowmotiony »

narrator wrote: post #151
Tiberium wrote: post #99

It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…

It's funny that part of the 4chan excitement over this is that they think they'll get back the AI girlfriend experience of when character.ai was hooked up to uncensored GPT-3. All that has been thoroughly shut down by character.ai and Replika and they just want their girlfriends back.

I'd want an uncensored GPT-3 too and I don't want an AI girlfriend - I just find that chatgpt has too much moral censorship to be fun to use. Want to ask about a health condition? Nope, forbidden. Have a question related to IT security? That's a big no-no. Anything remotely sexual even in educational context? No can do. Yesterday I finished watching a TV show about French intelligence and asked it to recommend some good books about espionage - it told me I shouldn't be reading such things because it's dangerous.

I ended up deleting my account, i won't allow some chatbot made by a couple 20 year old silicon valley billionnaires teach me about ethics and morality.

Re: Facebook LLAMA is being openly distributed via torrents

#502

Post by cma »

htrp wrote: post #179
linearalgebra45 wrote: post #123

Hypothetically, what would the consequences be if I ran this on my university's computing cluster?

Either you get a nice invitation to collaborate on research with one of your uni's professors..... or you get sent to academic/disciplinary review and probably suspended for the semester.

Why? Model weights aren't copyrighted and they didn't protect it as a trade secret.

Re: Facebook LLAMA is being openly distributed via torrents

#503

Post by gpm »

londons_explore wrote: post #65
px43 wrote: post #57

Earlier quoted context omitted.

Hell, I'd love to be able to buy a $30k server to run these models. I think to run BLOOM required something more along the lines of a $200k server.

With code modifications, it should be possible to run this with a very modest machine as long as you're happy for performance to suck. Transformer models typically need to read all the weights per 'word' output, so if your model is 20GB and you have not enough ram or vram, but have an SSD that reads 1GB/sec, expect 3 words per minute output speed. However, code changes are necessary to achieve that, although they won…

I just tried this on the 7B model. Steady state single threaded CPU performance of 23 seconds/token on a Ryzen 5800x (I'm not sure why it's only using a single thread... usually these libraries automatically use more) and 14GB of ram. It used more than double that amount of ram while loading the model, and the first token took 183 seconds (potentially it's doing more work to parse the prompt that I'm not measuring properly).

Re: Facebook LLAMA is being openly distributed via torrents

#504

Post by sebzim4500 »

ok123456 wrote: post #450
tinyspacewizard wrote: post #356

Earlier quoted context omitted.

Not a conspiracy at all. See also IE, Android, Kubernetes...

Any links about IE, Android and Kubernetes? I'm not up on these being ops.

He's not saying they are leaks, he's saying that they are examples of a large company releasing a product for free to crush the competition. I don't necessarily agree with him, especially about Kubernetes.

Re: Facebook LLAMA is being openly distributed via torrents

#505

Post by toxicFork »

boole1854 wrote: post #399
mwill wrote: post #389

Earlier quoted context omitted.

Off topic, but I clicked around /g/, which I haven't done in probably more than a decade, and a thread caught my eye about learning to code. The replies were overwhelmingly of the position that it is useless, and you will be replaced by AI before you can get a job if you start learning now. I think that's nonsense, and 4chan is bent towards pessimism but it's still surprising to me.

4chan has been in full doomer mode for years. It didn't used to be, from what I remember, though I was never an active denizen. I'd love to understand the sociology behind the change in vibe that happened there.

Too much anime and weed

Re: Facebook LLAMA is being openly distributed via torrents

#506

Post by narrator »

screye wrote: post #473
abandonliberty wrote: post #395

Earlier quoted context omitted.

I'm curious if the blocking of adult content has to do with moralism, commercial interests, or something deeper. An eager to please conversational partner who can generate endless content seems quite dangerous and addictive, especially when it crosses over into romantic areas. There's already posts of people spending entire days interacting with LLMs, using as their therapist, romantic partner, etc. Combined with fin…

Wall-E humans are going to be reality. That last century has already proven that humans cannot be expected to responsibly indulge in Gluttony, Sloth or Lust. Now these models can skip the material desires and trigger permanent hormone releases through perfectly personalized content. I genuinely fear that the breakdown of millennia old social structures that kept us human might lead to a temporary (century long) turmo…

Perpetual happiness is already a solved problem in humans. It's called the mu-opioid receptor. That's what opioid junkies sprawled on the sidewalk half-naked in San Francisco have discovered. Fentanyl is very cheap and you could put someone in factory farm like confines and feed them bare sustenance and fentanyl for the rest of their lives and they'd probably be "happy" if kept perpetually high.

However, those opioid receptors should not be pushed synthetically because they have been positioned by evolution in all sorts of strategic spots to encourage pro-social behavior, mating, eating, etc. that are part of our millions year old evolutionary program that must have intrinsic value in itself. If it has no intrinsic value and any happiness is as good as any other happiness, then someone spending the rest of their lives in an opioid haze and someone interacting with the world in a way that evolution tells them to in order to be happy would be considered equivalent, and that would be the end of the human race essentially.

Re: Facebook LLAMA is being openly distributed via torrents

#507

Post by xg15 »

im3w1l wrote: post #331
narrator wrote: post #151

Earlier quoted context omitted.

It's funny that part of the 4chan excitement over this is that they think they'll get back the AI girlfriend experience of when character.ai was hooked up to uncensored GPT-3. All that has been thoroughly shut down by character.ai and Replika and they just want their girlfriends back.

I think it's funny that out of all the scifis I know, Chobits of all things is looking to be the most accurate.

Yeah, the irony is almost palpable.

Then again, if you're consider which kind of people would have the most motivation to actually develop AGI, maybe not so surprising again.

Re: Facebook LLAMA is being openly distributed via torrents

#508

Post by lazide »

nico_h wrote: post #495
screye wrote: post #473

Earlier quoted context omitted.

Wall-E humans are going to be reality. That last century has already proven that humans cannot be expected to responsibly indulge in Gluttony, Sloth or Lust. Now these models can skip the material desires and trigger permanent hormone releases through perfectly personalized content. I genuinely fear that the breakdown of millennia old social structures that kept us human might lead to a temporary (century long) turmo…

You’re confusing reality in the US with reality in general. The obesity epidemic (Gluttony ) is extreme in the US but not in other just as rich countries. I don’t know what you are referring to with the irresponsible Sloth indulging.

Oh, it’s getting there. Obesity is spreading.

Re: Facebook LLAMA is being openly distributed via torrents

#509

Post by turmeric_root »

q1w2 wrote: post #314
turmeric_root wrote: post #296

Earlier quoted context omitted.

the 7B model runs on a CUDA-compatible card with 16GB of VRAM (assuming your card has 16-bit float support). I only got the 30b model running on a 4 x Nvidia A40 setup though.

The 30B is 64.8GB and the A40s have 48GB NVRAM ea - so does this mean you got it working on one GPU with an NVLink to a 2nd, or is it really running on all 4 A40s? Is there a sub/forum/discord where folks talk about the nitty-gritty?

> so does this mean you got it working on one GPU with an NVLink to a 2nd, or is it really running on all 4 A40s?

it's sharded across all 4 GPUs (as per the readme here: https://github.com/facebookresearch/llama). I'd wait a few weeks to a month for people to settle on a solution for running the model, people are just going to be throwing pytorch code at the wall and seeing what sticks right now.

Re: Facebook LLAMA is being openly distributed via torrents

#510

Post by yunwal »

verall wrote: post #449
yunwal wrote: post #392

Earlier quoted context omitted.

Long term I also imagine it's much cheaper to run these large model trainings on renewables. It's a very centralized process that doesn't necessarily need 100% availability. The manufacturing process, however, is totally decentralized, and NVIDIA mostly manufactures in China where coal is cheap.

A100s are manufactured in Taiwan.

True, but previous chips have been manufactured in China, and they’re also developing and manufacturing their successor to A100s (H100s) in China.

https://www.cnbc.com/2022/09/01/nvidia-says-us-government-al...

Post reply on HN