Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook LLAMA is being openly distributed via torrents

github.com

Post reply on HN
Print view View original

261–270 of 719 posts

Re: Facebook LLAMA is being openly distributed via torrents

#261

Post by furyofantares »

ddulaney wrote: post #166
gambiting wrote: post #154

Earlier quoted context omitted.

I don't think that's right - even if you had the full source code for either of those, it's extremely unlikely you'd be able to build them on your own machine.

Building them would be a challenge, but definitely not an insurmountable one. I’ve worked on a couple of C++ projects at a similar scale to Windows (millions of LOC) and the build systems were a major pain. But a determined engineer with the readme file and no other help could get it building in a week or so. (This probably says more about how hard it is to build C++ than anything else)

Some years ago someone that worked at Microsoft told me he didn't think any individual engineer who already works on Windows could ever get Windows building by themselves with just the code.

Re: Facebook LLAMA is being openly distributed via torrents

#262

Post by trentdotexe »

londons_explore wrote: post #61
Yoric wrote: post #56

Earlier quoted context omitted.

If it's PyTorch, it can definitely contain and execute arbitrary code. One of the reasons I'm not a huge fan of PyTorch.

They could contain arbitrary code... But typically do not. That means that with the right viewer application it will be trivial to know for sure. It isn't like a multi gigabyte game for example, where knowing if there is any malicious code could easily be a multi-month reverse engineering project to get to the answer of 'probably not, but we don't have time to check every byte with a fine tooth comb'

I mentioned this above, but Trail of Bits has a tool just for this purpose called Fickling. https://github.com/trailofbits/fickling

Re: Facebook LLAMA is being openly distributed via torrents

#263

Post by brookst »

nerpderp82 wrote: post #239
brookst wrote: post #159

Earlier quoted context omitted.

Your SHA256 hash won’t be able to summarize text, write poems, or make up plots for books. The crazy thing about these models is that the compute power going into them is at least somewhat reversible.

Are you saying they are like compact memoizers? What Stable Diffusion can fit into that model is amazing.

Certainly they retain not just information but compute capacity in a way that other expensive transformations don’t. I’m hard pressed to think of another example where compute spend now can be banked and used to reduce compute requirements later. Rainbow tables maybe? But they’re much less general purpose.

Re: Facebook LLAMA is being openly distributed via torrents

#264

Post by shultays »

pavlov wrote: post #41

It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent. This has not been the case for most commercial software for the past 20 years, during the cloud era. If you could steal a dump of random Facebook source code, it would be 99% useless because it’s so closely tied to the infrastructure. There’s almost nothing yo…

I mean the answer to the life is 42 but it took 7.5 million years for an advanced alien tech computer with the size of a building to calculate that

/s

Calculating things takes time and unrelated to output size. There are NP problems that simply outputs true or false yet requires more computational power than the universe can support

Re: Facebook LLAMA is being openly distributed via torrents

#265

Post by rnk »

dmix wrote: post #230
bentcorner wrote: post #224

Earlier quoted context omitted.

I posit to a friend that: a) As these AI constructs become more advanced (especially around memory and personalization), we will eventually be able to treat them as people b) Some business will eventually sell an off-the-shelf product (hardware and/or software) that is an AI you can bring into your home, that you can treat as a friend, confidant and partner c) Someone will eventually lose their AI friend of many mont…

Reminds me of Ray Kurizwell's obsession with uploading brains to the cloud to get back his beloved father.

When we can upload our brains to the cloud, and you can do something with them like interacting or running the brain, then we'll all be effectively immortal. That's a pretty big deal. See the book altered carbon.

Re: Facebook LLAMA is being openly distributed via torrents

#266

Post by nicoburns »

ddulaney wrote: post #166
gambiting wrote: post #154

Earlier quoted context omitted.

I don't think that's right - even if you had the full source code for either of those, it's extremely unlikely you'd be able to build them on your own machine.

Building them would be a challenge, but definitely not an insurmountable one. I’ve worked on a couple of C++ projects at a similar scale to Windows (millions of LOC) and the build systems were a major pain. But a determined engineer with the readme file and no other help could get it building in a week or so. (This probably says more about how hard it is to build C++ than anything else)

> But a determined engineer with the readme file and no other help could get it building in a week or so.

That's probably true, but I wouldn't be surprised if something like windows doesn't have a README file. And it does have build instructions they may well be in some wiki separated from the source code.

Re: Facebook LLAMA is being openly distributed via torrents

#267

Post by FusionRockets »

dmix wrote: post #183
narrator wrote: post #151

Earlier quoted context omitted.

It's funny that part of the 4chan excitement over this is that they think they'll get back the AI girlfriend experience of when character.ai was hooked up to uncensored GPT-3. All that has been thoroughly shut down by character.ai and Replika and they just want their girlfriends back.

The Repilka subreddit became one of the weirdest places on the internet when their model got capped for adult content. https://www.reddit.com/r/replika/ Hundreds of men (and yes women) full on acting like they lost a spouse and posting constantly about it for weeks. AI is going to create some unusual social situations the general public isn't ready to grasp. And we're only in the early alpha stages.

Can you show me example links?

Re: Facebook LLAMA is being openly distributed via torrents

#268

Post by willy_k »

DownGoat wrote: post #184
ddulaney wrote: post #166

Earlier quoted context omitted.

Building them would be a challenge, but definitely not an insurmountable one. I’ve worked on a couple of C++ projects at a similar scale to Windows (millions of LOC) and the build systems were a major pain. But a determined engineer with the readme file and no other help could get it building in a week or so. (This probably says more about how hard it is to build C++ than anything else)

I don't really think it is a language ting, just more of a project size thing.

Well, it kind of is a language thing. Many newer languages (Rust and Go come to mind) are much more consistent in the way you interact with them as your project scales.

Re: Facebook LLAMA is being openly distributed via torrents

#269

Post by rihegher »

mlboss wrote: post #256
aent wrote: post #32

For anyone wondering, it includes 4 models: 7/13/30/65 billion parameters, the smallest one is 14Gb, the largest one is 131GB, all four are 235Gb.

Is it possible to run the smallest one on a consumer gpu with 24gb ram ?

I would be surprised if you can't. The smallest weight file is 14gb apparently

Re: Facebook LLAMA is being openly distributed via torrents

#270

Post by sebzim4500 »

krisoft wrote: post #150
sebzim4500 wrote: post #89

Earlier quoted context omitted.

No you are probably overestimating the cost by 1-2 orders of magnitude. GPT-3 probably cost under $5 million, and this model is smaller and there have been algorithmic improvements to training transformers since then.

> No you are probably overestimating the cost by 1-2 orders of magnitude. You are right! Wow. Thank you for correcting me. > GPT-3 probably cost under $5 million, Is that one training run or includes all the fiddling to find the right hyperparameters? Or there aren't many of those in these training or they are not that sensitive?

I think they probably did a lot of hyperparameter searching to train the smaller models and then extrapolated for the largest model, but I'm just guessing. OpenAI had a finite amount of money when they were training GPT-3, they likely do it differently now that inference costs are significant compared to training costs.
Post reply on HN