Cerebras Inference: AI at Instant Speed
cerebras.ai
Cerebras Inference: AI at Instant Speed
1–10 of 75 posts
Re: Cerebras Inference: AI at Instant Speed
#2This is where we always assumed the industry was going. Expensive GPUs are great for training, but inference is getting so optimized that it will run on smaller and/or cheaper processors (per token).
Re: Cerebras Inference: AI at Instant Speed
#3The numbers are pretty incredible. Will the competition be able to match them?
Re: Cerebras Inference: AI at Instant Speed
#4This is where we always assumed the industry was going. Expensive GPUs are great for training, but inference is getting so optimized that it will run on smaller and/or cheaper processors (per token).
"wafer scale" inference using 44GB of straight up SRAM per chip does not exactly sound "smaller and cheaper" to me. Just optimized.
Re: Cerebras Inference: AI at Instant Speed
#5This is where we always assumed the industry was going. Expensive GPUs are great for training, but inference is getting so optimized that it will run on smaller and/or cheaper processors (per token).
Cerebras' Wafer Scale Engine is the opposite of small and cheap.
Re: Cerebras Inference: AI at Instant Speed
#6This is where we always assumed the industry was going. Expensive GPUs are great for training, but inference is getting so optimized that it will run on smaller and/or cheaper processors (per token).
"wafer scale" inference using 44GB of straight up SRAM per chip does not exactly sound "smaller and cheaper" to me. Just optimized.
Cheaper on a per token basis.
Re: Cerebras Inference: AI at Instant Speed
#7Their CEO talks about this in the gradient dissent podcast [1]
Re: Cerebras Inference: AI at Instant Speed
#8This is where we always assumed the industry was going. Expensive GPUs are great for training, but inference is getting so optimized that it will run on smaller and/or cheaper processors (per token).
Cerebras' Wafer Scale Engine is the opposite of small and cheap. https://cerebras.ai/product-chip/
I updated my comment to be clearer. I meant smaller and/or cheaper per token. In this case it's cheaper per token.
Re: Cerebras Inference: AI at Instant Speed
#9Re: Cerebras Inference: AI at Instant Speed
#10I want to know what the current power requirements are, as well as the cost for the machine. The last time I looked at one of these it was an absolute beast (although very impressive).