Facebook LLAMA is being openly distributed via torrents • Hacker News Board

Re: Facebook LLAMA is being openly distributed via torrents

#621

Post by Red_Leaves_Flyy » Sat, Mar 04, 2023, 3:19 AM UTC

bqmjjx0kac wrote: post #357

randomdestructn wrote: post #335

Earlier quoted context omitted.

Or maybe they sell that data to another company that operates kind of like a collections agency, which takes on the 'risk' of storing the data, then repeatedly calls and offers to give them their AI friend back at an extortionate rate. The data privacy side of this is an interesting conversation as well. Think of the information an employee or hacker could leak about a person after they spent some time with such an i…

Imagine if they could transform the AI companion model into an extortionist model.

I can see the headlines:

3,567 Dead - Destitute Robosexual Blows Up Collections Agency In Suicide Bombing

“This is the 53rd such incident this year. Current year death toll from these attacks is now 118,689 in current city, Legislators are pointedly ignoring protestors demanding AI rights and an end to extortionate fees charged to reinstate AI lover subscriptions.”

Re: Facebook LLAMA is being openly distributed via torrents

#622

Post by koheripbal » Sat, Mar 04, 2023, 3:22 AM UTC

espadrine wrote: post #38

ot wrote: post #16

In case it's not clear what's happening here (and from the comments it doesn't seem like it is), someone (not Meta) leaked the models and had the brilliant idea of advertising the magnet link through a GitHub pull request. The part about saving bandwidth is a joke. Meta employees may have not noticed or are still figuring out how to react, so the PR is still up. (Disclaimer: I work at Meta, but have no relationship w…

> Meta employees may have not noticed or are still figuring out how to react Given that the cat is out of the bag, if I were them, I would say that it is now publicly downloadable under the terms listed in the form. It is great PR, which if this was unintentional, is a positive outcome out of a bad situation.

How likely is it that there is a larger model that they haven't discussed?

Re: Facebook LLAMA is being openly distributed via torrents

#623

Post by jpieper » Sat, Mar 04, 2023, 3:32 AM UTC

markasoftware wrote: post #602

TaylorAlexander wrote: post #529

Earlier quoted context omitted.

Following up. After rebooting in to GUI that was enough to get it to fit, I guess xorg just accumulated some cruft in my last boot. So I can run it alongside gnome. nvidia-smi reports this model is using 15475MiB after changing the max batch size from 32 to 8 (see link in above post) As others have stated someone may have injected unknown code in to the pickled checkpoint, so I recommend running this in docker. I use…

I was able to run 7B on a CPU, inferring several words per second: https://github.com/markasoftware/llama-cpu

Beginner pytorch user here... it looks like it is using only one CPU on my machine. Is it feasible to use more than one? If so, what options/env vars/code change are necessary?

Re: Facebook LLAMA is being openly distributed via torrents

#624

Post by jjeaff » Sat, Mar 04, 2023, 3:35 AM UTC

sangnoir wrote: post #596

OkayPhysicist wrote: post #568

Earlier quoted context omitted.

Probably. The wording of the Declaration of Independence makes it clear that rights, at least in the American tradition, are not granted to you by law, they are inalienable human rights that are protected by law. That's why immigrants, tourists, and other visitors to America are still protected by the Constitution. Now, over time we've eroded some of that, but we still have some of the most radical free speech laws i…

I don't mean Dutch immigrants - I mean Dutch people living in the Netherlands (or Russians in Russia). One can incorporate an American entity as a non-resident without ever stepping foot on American soil - do you think it's a good idea for that entity to have the same rights as American citizens, and more rights than its members (who are neither citizens, nor on American soil)?

I know that foreign nationals and foreign governments are prohibited from donating money to super PACs. They are also prohibited from even indirect, non-coordinated expenditures for or against a political candidate. (which is basically what a super PAC does).

However, foreign nationals can contribute to "Social Welfare Organizations" like the NRA which, in order to be classified as a SWO, must spend less than half it's budget on political stuff. That SWO can then donate to super PACs but don't have to disclose where the money came from.

Foreign owned companies with US based subsidiaries can donate to Super PACs as well. But the super PACs are not allowed to solicit donations from foreign nationals (see Jeb Bush's fines for soliciting money from a British tobacco company for his super pac).

I would imagine that if foreign nationals setup a corporation in the US in order to funnel money to political causes, that would be illegal. But if they are using established, legitimate businesses to launder their donations, that seems to be allowed as long as we can't prove that foreign entities are earmarking specific funds to end up in PACs and campaigns in the US.

Re: Facebook LLAMA is being openly distributed via torrents

#625

Post by Red_Leaves_Flyy » Sat, Mar 04, 2023, 3:36 AM UTC

mecsred wrote: post #297

starkd wrote: post #248

Earlier quoted context omitted.

Wow, I did not realize they were in that deep. It is probably good they pulled the plug on this. Better now than later. People need to realize this is messing with emotions in an unknown way.

Yeah now we can get back to normal internet usage, which definitely doesn't involve strange emotional attachments and wierd rabbit holes.

Have you read the 4chan threads? Anons will figure out how to make convincing waifus that they run locally so they can explore their weird kinks that no company will consider. There are a lot of extremely intelligent, sexually frustrated, and emotionally immature coders. Cf Fiona from Silicon Valley, Her, Westworld, simulator scenes in Star Trek, etc.

Pandora’s box is open. We should be funding research and social services to help these people better integrate and find a healthy balance between their fetishes and escapism. We probably won’t since even healthcare is too much to ask for from half our legislature.

Re: Facebook LLAMA is being openly distributed via torrents

#626

Post by Red_Leaves_Flyy » Sat, Mar 04, 2023, 3:45 AM UTC

slowmotiony wrote: post #501

narrator wrote: post #151

Earlier quoted context omitted.

It's funny that part of the 4chan excitement over this is that they think they'll get back the AI girlfriend experience of when character.ai was hooked up to uncensored GPT-3. All that has been thoroughly shut down by character.ai and Replika and they just want their girlfriends back.

I'd want an uncensored GPT-3 too and I don't want an AI girlfriend - I just find that chatgpt has too much moral censorship to be fun to use. Want to ask about a health condition? Nope, forbidden. Have a question related to IT security? That's a big no-no. Anything remotely sexual even in educational context? No can do. Yesterday I finished watching a TV show about French intelligence and asked it to recommend some g…

Neglecting to give consideration to the reasons for these limitations is a sign that you might have some low hanging fruit to pick off the ethical and morality trees of knowledge.

Re: Facebook LLAMA is being openly distributed via torrents

#627

Post by arthurcolle » Sat, Mar 04, 2023, 4:43 AM UTC

q1w2 wrote: post #304

aent wrote: post #32

For anyone wondering, it includes 4 models: 7/13/30/65 billion parameters, the smallest one is 14Gb, the largest one is 131GB, all four are 235Gb.

I wonder how many people are scrambling to set this up on their startup infra. 6x24GB NVRAM on 6 GPUs linked with NVSwitch is a little pricey, but totally doable.

I got it running using Colab Pro+ (immediately got a V100 40GB VRAM GPU) - the 7B model works with batch size of 8 and a max seq len of 1024

Re: Facebook LLAMA is being openly distributed via torrents

#628

Post by markasoftware » Sat, Mar 04, 2023, 5:50 AM UTC

jpieper wrote: post #623

markasoftware wrote: post #602

Earlier quoted context omitted.

I was able to run 7B on a CPU, inferring several words per second: https://github.com/markasoftware/llama-cpu

Beginner pytorch user here... it looks like it is using only one CPU on my machine. Is it feasible to use more than one? If so, what options/env vars/code change are necessary?

Perhaps try setting `OMP_NUM_THREADS`, for example `OMP_NUM_THREADS=4 torchrun ...`.

But on my machine, it automatically used all 12 available physical cores. Setting OMP_NUM_THREADS=2 for example lets me decrease the number of cores being used, but increasing it to try and use all 24 logical threads has no effect. YMMV.

Re: Facebook LLAMA is being openly distributed via torrents

#629

Post by sinity » Sat, Mar 04, 2023, 6:21 AM UTC

Name_Chawps wrote: post #551

lukeplato wrote: post #548

Earlier quoted context omitted.

It would be interesting if there was a WikiLeaks-type of organization that facilitates safely leaking large models from big corporations. Not sure how that would play out for accelerationism and existential risk, but I certainly don't trust the current powers that be.

Open sourcing is widely recognized to be a bad thing when it comes to AI existential risk. (For the same reason you don't want simple instructions for how to build bio weapons posted to the internet.) Modern AI is pretty harmless though, so it doesn't matter yet.

> Modern AI is pretty harmless though, so it doesn't matter yet.

Yes, that's why the only thing people flipping out about "safety" of making them public achieve is making public distrustful about AI safety.

Re: Facebook LLAMA is being openly distributed via torrents

#630

Post by Tepix » Sat, Mar 04, 2023, 6:25 AM UTC

exo-pla-net wrote: post #390

q1w2 wrote: post #304

Earlier quoted context omitted.

I wonder how many people are scrambling to set this up on their startup infra. 6x24GB NVRAM on 6 GPUs linked with NVSwitch is a little pricey, but totally doable.

How pricey would you estimate?

If you want to do it the cheap way by buying used stuff, the most expensive parts are:

- $2000 for a Threadripper 3xx5WX with a socket sWRX8 mainboard

- $5000 for 6x RTX 3090

- $350 for two 1500W PSUs

- $700 for 256GB RAM

You will also need PCIe extenders and perhaps some watercooling. And find a suitable case. The 2-card NVLink bridges are between $100 and $300 each (you nay want 3). All in all i think less than $10k.