Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fork of Facebook’s LLaMa model to run on CPU

github.com

Post reply on HN
Print view View original

61–70 of 178 posts

Re: Fork of Facebook’s LLaMa model to run on CPU

#61

Post by psychphysic »

saurik wrote: post #41
aqme28 wrote: post #21

Earlier quoted context omitted.

Crazy to me that as soon as one GPU wave is dying (crypto), another one is picking up slack.

One day we'll find out that all of the VR, crypto, and maybe now AI bubbles were nothing but conspiracies being driven by big-GPU to keep their share price up.

DLSS and DLAA were at least, FSR proved that.

Re: Fork of Facebook’s LLaMa model to run on CPU

#62

Post by pelagicAustral »

HeartStrings wrote: post #38
dsign wrote: post #15

The thing I like the most about the current AI wave is the pressure is putting on computing hardware. Yes, mobile phones with long battery lives are cool and all of that, but most cool things I like are locked behind huge computational requirements.

John Hopkins are working on organoids that will replace silicon GPUs for AI.

If they can get pass these new ethical committees...

Re: Fork of Facebook’s LLaMa model to run on CPU

#63

Post by taneq »

saurik wrote: post #41
aqme28 wrote: post #21

Earlier quoted context omitted.

Crazy to me that as soon as one GPU wave is dying (crypto), another one is picking up slack.

One day we'll find out that all of the VR, crypto, and maybe now AI bubbles were nothing but conspiracies being driven by big-GPU to keep their share price up.

VR has been a godsend for forcing hardware, OS and driver developers to actually pay attention to jitter and max latency. If crypto means we get nice fast pretty games and fancy AI then I’m for it. :)

Re: Fork of Facebook’s LLaMa model to run on CPU

#64

Post by cfcf14 »

visarga wrote: post #7

It's useless before the model gets instruction and preference tunings. Won't even follow a simple ask, it will just assume it is a list of questions and generate more, or continue with slightly related comments. FB trained a LLaMA-I (instruction tuned) variant for sports, just to show they can, but I don't think it got released.

You have to prompt it correctly, non-instruction-aligned models don't behave like agent simulators by default.

Re: Fork of Facebook’s LLaMa model to run on CPU

#67

Post by colordrops »

HeartStrings wrote: post #38
dsign wrote: post #15

The thing I like the most about the current AI wave is the pressure is putting on computing hardware. Yes, mobile phones with long battery lives are cool and all of that, but most cool things I like are locked behind huge computational requirements.

John Hopkins are working on organoids that will replace silicon GPUs for AI.

Reminds me of this Choose Your Own Adventure book from 1984. It was about how PCs had organic AI components and each was unique, and you happened to get your hands on a super intelligent one.

https://www.goodreads.com/en/book/show/755062

Re: Fork of Facebook’s LLaMa model to run on CPU

#68

Post by singularity2001 »

ingenieroariel wrote: post #56
ComplexSystems wrote: post #34

Earlier quoted context omitted.

How are you getting this to run fast? I'm on a top of the line M1 MBP and getting 1 token every 8 minutes.

Try switching all the .cuda() to .mps() I got a 100x speedup on a different language model on a Macbook M1 Air. https://pytorch.org/docs/stable/notes/mps.html

dedicated fork: https://github.com/remixer-dec/llama-mps
Post reply on HN