21–30 of 75 posts
Can really fast inference (e.g. 1M tok/sec) make LLMs more intelligent? I am imagining you could run multiple agents and can choose and discard outputs using other LLMs simultaneously. Will the output look more like a real thought process? Or will it remain just same?
Earlier quoted context omitted.
Cerebras' Wafer Scale Engine is the opposite of small and cheap. https://cerebras.ai/product-chip/
That's totally nuts. How do they deal with the silicon warping around disabled cores or dark silicon? How long of hard running does it take before the chip gets fatally damaged and needs to be replaced in their system? Word on the street is that h100s fail surprisingly often, this can't be better
This video from Cerebras perfectly explain how they solve the interconnect problem, and why their approach greatly reduces the risk of Blackwell-type hardware design challenges.
Earlier quoted context omitted.
Cerebras' Wafer Scale Engine is the opposite of small and cheap. https://cerebras.ai/product-chip/
That's totally nuts. How do they deal with the silicon warping around disabled cores or dark silicon? How long of hard running does it take before the chip gets fatally damaged and needs to be replaced in their system? Word on the street is that h100s fail surprisingly often, this can't be better
https://www.anandtech.com/show/16626/cerebras-unveils-wafer-...
Earlier quoted context omitted.
"wafer scale" inference using 44GB of straight up SRAM per chip does not exactly sound "smaller and cheaper" to me. Just optimized.
Cheaper on a per token basis.
I want to know what the current power requirements are, as well as the cost for the machine. The last time I looked at one of these it was an absolute beast (although very impressive).
"If you care and have to ask it's not for you".
In all seriousness I've worked with and am familiar with Cerebras, Groq, etc. Let's just say GPUs still reign supreme in terms of practicality outside of usage of their hardware via cloud for nearly all use-cases.
Groq, for example, has essentially stopped selling their "real" HW directly because the borderline absurd amount of floor space, etc was found to be challenging once they hit the market. There's enough demand and more (recurring) money to be made anyway hosting services on your chips.
Similar to the Bitcoin mining ASIC game in the heyday - sure we could sell these or we could just use them to mine, develop next gen, sell previous gen, repeat.
Earlier quoted context omitted.
Cerebras' Wafer Scale Engine is the opposite of small and cheap. https://cerebras.ai/product-chip/
That's totally nuts. How do they deal with the silicon warping around disabled cores or dark silicon? How long of hard running does it take before the chip gets fatally damaged and needs to be replaced in their system? Word on the street is that h100s fail surprisingly often, this can't be better
The cooling is significantly better than what you'd see on a server platform with water cooling channels going to each row of the wafer.
It would be understandable that they are focused currently on inference speed, but features like structured output and prompt caching make it possible to build more capable LLM applications. Does Cerebras support reliable structured output like the recent OpenAI 4o?
This question is a bit out of context. Cerebras is a startup producing innovative AI chips. Their chips are super cool, and I personally believe Cerebras is ahead of the industry and is on the right technical path. As a matter of fact, Cerebras started with HPC chips. Then pivoted to AI like everyone else. They are still deep in the trench for survival. Given that, they have very little software prowess compared to A…
Sure but can they fit a bigger model? I don’t think they can string together these to fit bigger models like llama3.1 405b
CS-3 system is built for single node domain scaling to 24 Trillion parameter models. I.e., they claim you can run the same code without hand-written distributed training code to reach 24 Trillion parameter models.
Earlier quoted context omitted.
That's totally nuts. How do they deal with the silicon warping around disabled cores or dark silicon? How long of hard running does it take before the chip gets fatally damaged and needs to be replaced in their system? Word on the street is that h100s fail surprisingly often, this can't be better
They have a way of bypassing bad cores, and over-provision both logic and memory by 1.5% to account for that. They get 100% yields this way. https://www.anandtech.com/show/16626/cerebras-unveils-wafer-...