Facebook LLAMA is being openly distributed via torrents

Post by skybrian » Sat, Mar 04, 2023, 6:41 PM UTC

richardhod wrote: post #633

skybrian wrote: post #632

Earlier quoted context omitted.

I think of the Turing test as just another game, like chess or Go. It’s not a captcha or a citizenship test. Making an AI that can beat good players would be a significant milestone. What sort of achievement is letting the AI win at a game, or winning against incompetent players? So of course you play to win. If you want to adjust the difficulty, change the rules giving one side or the other an advantage.

I was confused by your first reply at first. I think that's because you are answering a different question from a number of other people. You're asking about the conditions under which and AI might fool people into thinking it was a human, whereas I think others are considering the conditions under which a human might consistently emotionally attach to an AI, even if the human doesn't really think it's real.

Post by flangola7 » Sat, Mar 04, 2023, 7:26 PM UTC

bick_nyers wrote: post #644

flangola7 wrote: post #543

Earlier quoted context omitted.

If read is 1GB/s then it takes 20s to infer across a 20GB model. That's 3 tokens a minute.

Yeah I'm not sure what kind of math I was subscribed to yesterday, thanks

Post by make3 » Sat, Mar 04, 2023, 8:24 PM UTC

eigenvalue wrote: post #60

I'm not surprised-- I recently suggested that someone might try to pull an Aaron Swartz with the LLAMA weights (i.e., release them in an uncontrolled way similar to how Aaron attempted to release the JSTOR database). It's quite misleading for FB to claim that they are being so open, but then hoard the weights and only release it to a few academics. If the paper is to be believed, this is a major development, allowing…

Post by make3 » Sat, Mar 04, 2023, 8:28 PM UTC

kif wrote: post #37

I wonder what the memory requirements would be to run such a large model. I'd love to be able to run this model, alas my MacBook can barely run toy models.

Post by WithinReason » Sat, Mar 04, 2023, 9:18 PM UTC

fareesh wrote: post #658

WithinReason wrote: post #656

Earlier quoted context omitted.

I could drop the batch size to 5, then the VRAM use seemed to be around 15GB. Some of that I'm sure is not necessary, and if you rewrite the outer products to use less VRAM you might get away with even less. Eventually someone will make a library so you can run it without extra work.

Yeah true, do you think that it a realistic expectation though? I ask this given the events that have led to the leaking of the models. I am genuinely not sure what the optics / real world ramifications are of being publicly associated with projects that leverage models obtained via torrents through either hacking or negligence.

    //find index of element in sorted array in O(log(N)) time using binary search
    int find_idx(int a[N], int element) {
        int low = 0, high = N-1;
        while (low