Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook LLAMA is being openly distributed via torrents

github.com

Post reply on HN
Print view View original

361–370 of 719 posts

Re: Facebook LLAMA is being openly distributed via torrents

#361

Post by basch »

kir-gadjello wrote: post #313
basch wrote: post #172

Earlier quoted context omitted.

“Brute forcing a really inefficient approximation/estimator” is a good way to summarize it. It’s like having an overfit equation to a sample of data points, instead of the simpler actual line they fall near. They end up being black boxes, we have almost no idea how they work inside, and we have no idea how overtrained they are when something simpler could do the same thing.

Can "something simpler", for example, code correct function bodies from comments describing functions in natural language? I think people are too quick to dismiss the power of these models.

I am by no means dismissing the power. They are created very chaotically, however. Spaghetti thrown at a wall. They are brute force approximations.

They are wasteful. If LLaMa 13B is as powerful as previous 65B models, that's a significant amount of unnecessary paramaters lost/pruned in just this iterative upgrade alone. How small can they go? The fewest parameters that get the job done 99% as well is the way to go.

There is also the difference between the rules and use of language being directly compressed into the model, vs all the information known to humans compressed into the model. A smaller model that ingests relevant information on the fly (more like Bing, that supplements itself with search), may be less wasteful and perform better.

The current models being released are chosen because "they work" not because they are least fit and most performant optimized.

Re: Facebook LLAMA is being openly distributed via torrents

#362

Post by iandanforth »

bentcorner wrote: post #224
dmix wrote: post #183

Earlier quoted context omitted.

The Repilka subreddit became one of the weirdest places on the internet when their model got capped for adult content. https://www.reddit.com/r/replika/ Hundreds of men (and yes women) full on acting like they lost a spouse and posting constantly about it for weeks. AI is going to create some unusual social situations the general public isn't ready to grasp. And we're only in the early alpha stages.

I posit to a friend that: a) As these AI constructs become more advanced (especially around memory and personalization), we will eventually be able to treat them as people b) Some business will eventually sell an off-the-shelf product (hardware and/or software) that is an AI you can bring into your home, that you can treat as a friend, confidant and partner c) Someone will eventually lose their AI friend of many mont…

I feel like all these conversations need to be a diff against the movie Her. If it was already covered well there, why repeat it?

Re: Facebook LLAMA is being openly distributed via torrents

#363

Post by lolpython »

archon1410 wrote: post #220
Tiberium wrote: post #99

It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…

It has only just occurred to me that 4chan's technology board is /g/ because it's tech-naw-la- G

> The board letter /g/ stands for gijutsu (技術), the Japanese word for technology

https://wiki.installgentoo.com/wiki//g/#:~:text=%2Fg%2F%20is....

Re: Facebook LLAMA is being openly distributed via torrents

#364

Post by bertday »

ImprobableTruth wrote: post #324
madmod wrote: post #235

Would there be some way to “launder” the model to make it plausibly viable for commercial use? Train a new model with the weights of this model with some kind of noise added to make it hard to tell what it is based on?

Distillation would be the ideal way (especially because it also has efficiency gains), but as far as I know distillation for LLMs is kinda unproven. Honestly though, even if you just finetune it, which you will want anyway for any serious commercial application, it's essentially impossible to determine the origin.

Randomly perturbing the weights and then finetuning would probably make it impossible. If someone had access to the finetune dataset and you didn’t add noise, they could see if the finetuning curves intersect.

I guess in practice, it’ll look suspicious if you have an identical model architecture and have similar performance.

Re: Facebook LLAMA is being openly distributed via torrents

#365

Post by zamnos »

orbz wrote: post #132
fancyfredbot wrote: post #115

Earlier quoted context omitted.

$30m training cost is too high. Amazon's p4d.24xlarge is $32.77 an hour for 8 A100 GPUs. 2048 A100 GPUs for 23 days costs $4.6m at that rate. You might even get a discount.

At the same time I guarantee you they didn’t get it right the first time. I’m sure there were multiple (both serially and in parallel) runs as they worked out kinks and tuned hyper parameters.

Not to mention, the kind of expertise to run this for a major corporation doesn't come for free either? Facebook employs quite a few high profile ML researchers who undoubtedly make mid-high six figure salaries.

Re: Facebook LLAMA is being openly distributed via torrents

#366

Post by kragen »

eigenvalue wrote: post #60

I'm not surprised-- I recently suggested that someone might try to pull an Aaron Swartz with the LLAMA weights (i.e., release them in an uncontrolled way similar to how Aaron attempted to release the JSTOR database). It's quite misleading for FB to claim that they are being so open, but then hoard the weights and only release it to a few academics. If the paper is to be believed, this is a major development, allowing…

if aaron had attempted to release the jstor database he would have succeeded

Re: Facebook LLAMA is being openly distributed via torrents

#367

Post by humanizersequel »

londons_explore wrote: post #207
londons_explore wrote: post #65

Earlier quoted context omitted.

With code modifications, it should be possible to run this with a very modest machine as long as you're happy for performance to suck. Transformer models typically need to read all the weights per 'word' output, so if your model is 20GB and you have not enough ram or vram, but have an SSD that reads 1GB/sec, expect 3 words per minute output speed. However, code changes are necessary to achieve that, although they won…

There is a neat potential speedup here for the case where the bandwidth to your model weights is the limiting factor. If you have a guess what the model will output, then you can verify that your guess is correct very cheaply, since you can do it in parallel. That means there is the possibility to have a highly quantized small model in RAM, and then use the big model only from time to time. You might be able to get a…

This is an interesting concept, could you share a paper or some writeup about this?

Re: Facebook LLAMA is being openly distributed via torrents

#368

Post by ben_w »

beepbooptheory wrote: post #354
shadowgovt wrote: post #276

Earlier quoted context omitted.

At the end of the day, the Turing Test for establishment of AI personhood is weak for two reasons. 1. We're seeing more and more systems that get very close to passing the Turing Test but fundamentally don't register to people as "People." When I was younger and learned of Searle's Chinese Room argument, I naively assumed it wasn't a thought experiment we would literally build in my lifetime. 2. Humanity has a histor…

I dont know if I understand this general take I see a lot. Why care about this "AI personhood" at all? What is the tacit endgame everyone is always referencing with this? Isn't there just so many more both interesting and problematic aspects already here? What is the use of diverting the focus to some other point. "I see you are talking about cows, but I have thoughts about the ocean."

If AI are sentient and we think they aren't… the term “zombie” was created by slaves in the Caribbean who were afraid that even death would not free them from their servitude. This would be the genuine existence of AI which were conscious but which we denied.

If we have the opposite scenario in both details, where we think AI are sentient when they're not… at some point, brain scans and uploads will be a thing and then people are going to try mind uploading even just as a way to solve bodily injuries that could be fixed, and in that future nobody will even notice that while "the lights are on, nobody is home".

https://kitsunesoftware.wordpress.com/2022/06/18/lamda-turin...

Re: Facebook LLAMA is being openly distributed via torrents

#369

Post by wiml »

Filligree wrote: post #328
EamonnMR wrote: post #327

Earlier quoted context omitted.

My compliance brain says no, but the fact that models get trained with data they obtain without explicit permission makes says that finders keepers would be the relevant case law.

It's not clear that the model weights can be copyrighted. But of course, I wouldn't want to be the test case.

I would expect that they can be, in the same way that a curated collection of uncopyrightable facts can be.

Re: Facebook LLAMA is being openly distributed via torrents

#370

Post by dougmwne »

sebzim4500 wrote: post #278
Aissen wrote: post #157

It's nice that it's downloadable without filling a form (even though it should have been the default), a leak was bound to happen. The license is quite restrictive anyway: see RESTRICTIONS on https://forms.gle/jk851eBVbX1m5TAv5

If someone just decides to use the torrent and ignore those restrictions it might finally establish precident for if you can copyright model weights.

But even if you could copyright them, once you do some fine-tuning, they are not the same model weights!
Post reply on HN