They have a cloud platform. I just ran a test query on their version of Llama 3.1 70B and got 566 tokens/sec.
AI chipmaker Cerebras files for IPO
61–70 of 138 posts
Re: AI chipmaker Cerebras files for IPO
#62They have a cloud platform. I just ran a test query on their version of Llama 3.1 70B and got 566 tokens/sec.
Is that a lot? Do they have MLPerf submissions?
Re: AI chipmaker Cerebras files for IPO
#63Does cerebras make gaming GPUs, or is it enterprise-only?
Very solidly enterprise-only. They make single chips that take an entire wafer, use something like 10 kilowatts, and have liquid cooling channels that go through the chip. Systems are >$1M.
It's the return of the supercomputer! I really didn't think the supercomputer would come back as a thing, for so long it seemed stuck as a weird research project that only made sense for a tiny set of workloads... but it does make sense now
Re: AI chipmaker Cerebras files for IPO
#64NVIDIA is pretty established but there's also Intel, AMD, Google to contend with. Sure Cerebras is unique in that they make one large chip out of the entire wafer but nothing prevents these other companies from doing the same thing. Currently they are choosing not to because of wafer economics but if they chose to, Cerebras would pretty much lose their advantage. https://www.servethehome.com/cerebras-wse-3-ai-chip-la…
> wafer economics What are they? Is this related to defects? Can't they disable parts of defective chip just like other CPUs do? Sounds cheaper than cutting up and packaging chips individually!
Process development, feature size, and ultimate yield are probably what theyre after. Yes, for the past 30+ years everyone has used a combination of disabling (“fusing”) unused/unreliable logic on the die. In addition everyone also “bins” the chips from the same wafer to different SKUs based on stable clock speed, available/fused components, test results, etc. This can be very effective in increasing yield and salable parts.
My recollection is that theres speculation cerebras is building in significant duplicate features to account for defects. They cant “bin” their wafers in the same way as packaged chips. That will reduce total yield/utilization of the surface area.
The actual packaging steps are relatively low tech/cost compared to the semiconductor manufacturing. Theyre commonly outsourced somwhere like malaysia or thailand.
Re: AI chipmaker Cerebras files for IPO
#65NVIDIA is pretty established but there's also Intel, AMD, Google to contend with. Sure Cerebras is unique in that they make one large chip out of the entire wafer but nothing prevents these other companies from doing the same thing. Currently they are choosing not to because of wafer economics but if they chose to, Cerebras would pretty much lose their advantage. https://www.servethehome.com/cerebras-wse-3-ai-chip-la…
> 56x the size of H100 but only 8x the performance improvement isn't something I would brag about.
It doesn't sound like it's too bad for a 9 year old company. Nvidia had a 20-year head start. I would expect that they will continue to shrink it and increase performance. At some point, that might become compelling?
Re: AI chipmaker Cerebras files for IPO
#66Earlier quoted context omitted.
I think the wafer itself isn’t the whole deal. If you watch their videos and read the link you posted the wafer size allows them to stack them in a block with integrated power and cooling at a higher density than blades and attach enormous amounts of memory. Not including the system, cooling, cluster, etc seems like a relatively unfair comparison too given the node includes all of those things - which are very expens…
At the end of the day, Cerebras has not submitted any MLPerf results (of which I am aware). That means they are hiding something. Something not very competitive. So, performance is iffy. Density for density sake doesn’t matter since clusters are power limited.
Re: AI chipmaker Cerebras files for IPO
#67They have a cloud platform. I just ran a test query on their version of Llama 3.1 70B and got 566 tokens/sec.
Is that a lot? Do they have MLPerf submissions?
Yes, that's very fast. The same query on Groq, which is known for its fast AI inference, got 249 tokens/s, and 25 tokens/s on Together.ai. However, it's unclear what (if any) quantization was used and it's just a spot check, not a true benchmark.
https://www.zdnet.com/article/cerebras-did-not-spend-one-min...
Re: AI chipmaker Cerebras files for IPO
#68Cerebras is well-known in the AI chip market. They make chips that are an entire wafer. https://spectrum.ieee.org/cerebras-chip-cs3
Cerebrus made a great (now deleted) video on the whole computer hosting the wafer: https://web.archive.org/web/20230812020202/https://www.youtu... It’s fascinating.
This is a great video, thank you for sharing. My favorite part:
"...next we have this rubber sheet, which is very clever, and very patented!"
Re: AI chipmaker Cerebras files for IPO
#69Earlier quoted context omitted.
You seemed surprised that this company is having an IPO to actually raise funds for operations and expansion, vs as just an "exit" where VCs and other insiders can dump their shares onto the broader public. I might be a bit suspicious if a company in some low-capital-intensive industry was IPOing while unprofitable, but this is chip making. Even if they're not making their own fabs this is still an industry with high…
Does this mean that they couldn't find VCs to raise more cash?
VCs offer cash on different terms than the public does. This just
means Cerebras believes it can get capital more cheaply (or on otherwise better terms) than it can from VCs.
That might mean VCs are turning them down, yeah, but that’s just one of many possible factors into “where do we raise money”
Re: AI chipmaker Cerebras files for IPO
#70Earlier quoted context omitted.
Is that a lot? Do they have MLPerf submissions?
Yes, that's very fast. The same query on Groq, which is known for its fast AI inference, got 249 tokens/s, and 25 tokens/s on Together.ai. However, it's unclear what (if any) quantization was used and it's just a spot check, not a true benchmark. https://www.zdnet.com/article/cerebras-did-not-spend-one-min...
Met them at an MIT event last week, they don't quantize any models.