Funny. iirc some of the big tech (I think it was Google?) use torrents internally to deploy very large images to servers. Piracy is not the only use case!
It was used for years to distribute World of Warcraft updates. No idea if it still is.
Facebook LLAMA is being openly distributed via torrents
241–250 of 719 posts
Re: Facebook LLAMA is being openly distributed via torrents
#242Earlier quoted context omitted.
I posit to a friend that: a) As these AI constructs become more advanced (especially around memory and personalization), we will eventually be able to treat them as people b) Some business will eventually sell an off-the-shelf product (hardware and/or software) that is an AI you can bring into your home, that you can treat as a friend, confidant and partner c) Someone will eventually lose their AI friend of many mont…
> b) Some business will eventually sell an off-the-shelf product And by sell you mean a monthly subscription, ha ha.
Yeah, that's probably the most dystopian thing. This is almost a guaranteed outcome - someone pays a high subscription cost and cultivates a model with their personal details for years, and then loses all of it when they can't keep up the subscription cost. Cue a month or two later - they buy back in and their model has been wiped and their AI friend now knows nothing about them.
It's easy to poke fun at people who use these things but I believe these kinds of events are going to be truly traumatic.
Re: Facebook LLAMA is being openly distributed via torrents
#243Re: Facebook LLAMA is being openly distributed via torrents
#244Earlier quoted context omitted.
For anyone curious, it took 2048 A100 GPUs to train LLaMa, each GPU costs roughly $15k, facebook probably gets some sort of discount. That's a $30Mil if you want to train at that scale. Also IIRC it took 23 days to train the biggest model. Someone else can do the power consumption cost calculations.
Electricity costs are basically irrelevant because the cards are so expensive. A100 cards consume 250w each, with datacenter overheads we will call it 1000 kilowatts for all 2048 cards. 23 days is 552 hours, or 552,000 kilowatt hours total. Most dataceneters are between 7 and 10 cents per kilowatt hour for electricity. Some are below 4. At 10 cents, that's $53,000 in electricity costs, which is nothing next to $30 mi…
> Electricity costs are basically irrelevant because the cards are so expensive.
You mean in terms of money. I think this is exactly the problem that we have in CS, nobody really cares about CO2.
Re: Facebook LLAMA is being openly distributed via torrents
#245It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent. This has not been the case for most commercial software for the past 20 years, during the cloud era. If you could steal a dump of random Facebook source code, it would be 99% useless because it’s so closely tied to the infrastructure. There’s almost nothing yo…
Whole new vistas open up to possible retaliation for piracy. Imagine how a bootlegged AI could have been set up to not just steal your info but manipulate you into ruining your life as revenge for bootlegging it...
Re: Facebook LLAMA is being openly distributed via torrents
#246Re: Facebook LLAMA is being openly distributed via torrents
#247Earlier quoted context omitted.
I posit to a friend that: a) As these AI constructs become more advanced (especially around memory and personalization), we will eventually be able to treat them as people b) Some business will eventually sell an off-the-shelf product (hardware and/or software) that is an AI you can bring into your home, that you can treat as a friend, confidant and partner c) Someone will eventually lose their AI friend of many mont…
> b) Some business will eventually sell an off-the-shelf product And by sell you mean a monthly subscription, ha ha.
With micro transactions
Re: Facebook LLAMA is being openly distributed via torrents
#248Earlier quoted context omitted.
It's funny that part of the 4chan excitement over this is that they think they'll get back the AI girlfriend experience of when character.ai was hooked up to uncensored GPT-3. All that has been thoroughly shut down by character.ai and Replika and they just want their girlfriends back.
The Repilka subreddit became one of the weirdest places on the internet when their model got capped for adult content. https://www.reddit.com/r/replika/ Hundreds of men (and yes women) full on acting like they lost a spouse and posting constantly about it for weeks. AI is going to create some unusual social situations the general public isn't ready to grasp. And we're only in the early alpha stages.
Wow, I did not realize they were in that deep. It is probably good they pulled the plug on this. Better now than later. People need to realize this is messing with emotions in an unknown way.
Re: Facebook LLAMA is being openly distributed via torrents
#249Earlier quoted context omitted.
$30m training cost is too high. Amazon's p4d.24xlarge is $32.77 an hour for 8 A100 GPUs. 2048 A100 GPUs for 23 days costs $4.6m at that rate. You might even get a discount.
That’s something like a 50% premium over Azure, even before reservation discounts, that’s insane! Edit: todays pricing looks like about 20% higher, still. How are these prices so different.
Azure is generally a pretty terrible cloud (poor UX, very slow for anything, multiple highly critical cross-tenant security issues, etc.) far behind the market leader, AWS, so they have to compensate with pricing (same reason why Oracle Cloud is so reasonably price, they're already so far behind their usual pricing wouldn't make any sense).
Re: Facebook LLAMA is being openly distributed via torrents
#250Earlier quoted context omitted.
It's not even clear someone has leaked the models. A random person has put a download link on a PR, it could be anything.
Yes you're absolutely right. I went by another comment that seemed to confirm the contents, but that could be trolling too.
So do I understand correctly — those that tell don’t know, and those that know don’t tell?
I had to look up the legality of downloading torrents of copyrighted IP - and let me just say, don’t do it.