I assume people are aware, but Cerebras has a web demo and API which is open to try and it is 2000 tokens per second for Llama 3.3 70b and 1000 tokens per second for Llama 3.1 405b.
100x defect tolerance: How we solved the yield problem
111–120 of 186 posts
Re: 100x defect tolerance: How we solved the yield problem
#112Earlier quoted context omitted.
> And just as every other hype cycle, this one will crash down hard. Isn't that an inherent problem with pretty much everything nowadays: crypto, blockchain, AI, even the likes of serverless and Kubernetes, or cloud and microservices in general. There's always some hype cycle where the people who are early benefit and a lot of people chasing the hype later lose when the reality of the actual limitations and the real…
> I don't think the current "AI" is special in any way As someone who loves to pour ice water on AI hype, I have to say: you can't be serious. The current AI tech has opened up paths to develop applications that were impossible just a few years ago. Even if the tech freezes in place, I think it will yield substantial economic value in the coming years. It's very different from crypto, the main use case for which appe…
>It's very different from crypto, the main use case for which appears to be money laundering.
Which has substantial economic value (for certain groups of people).
Re: 100x defect tolerance: How we solved the yield problem
#113So they massively reduce the area lost to defects per wafer, from 361 to 2.2 square mm. But from the figures in this blog, this is massively outweighed by the fact that they only get 46222 sq mm useable area out of the wafer, as opposed to 56247 that the H100 gets - because they are using a single square die instead of filling the circular wafer with smaller square dies, they lose 10,025 sq mm! Not sure how that's a…
Why does their chip have to be rectangular, anyways? Couldn't they cut out a (blocky) circle too?
You need a rectilinear polygon that tessellates, and has the fewest sides possible to minimize the number of cuts necessary. And it would probably help the cutting if the shape is entirely convex, so that cuts can overshoot a bit without damaging anything.
That suggests a rectangle is the only possible shape.
Re: 100x defect tolerance: How we solved the yield problem
#114TSMC also have a manufacturing process used by Tesla's Dojo where you can cut up the chips, throw away the defective ones, and then reassemble working ones into a sort of wafer scale device (5x5 chips for Dojo). Seems like a more logical design to me.
Is this similar to a chiplet design? Chiplets have been a thing for a while, so I assume Cerebras avoided them on purpose.
Re: 100x defect tolerance: How we solved the yield problem
#115Re: 100x defect tolerance: How we solved the yield problem
#116I think this is an important step, but it skips over that 'fault tolerant routing architecture' means you're spending die space on routes vs transistors. This is exactly analogous to using bits in your storage for error correcting vs storing data. That said, I think they do a great job of exploiting this technique to create a "larger"[1] chip. And like storage it benefits from every core is the same and you don't nee…
[deleted]
Re: 100x defect tolerance: How we solved the yield problem
#117Earlier quoted context omitted.
It's a hype cycle with many of the hypers and deciders having zero idea about what AI actually is and how it works. ChatGPT, while amazing, is at its core a token predictor, it cannot ever get to an AGI level that you'd assume to be competitive to a human, even most animals. And just as every other hype cycle, this one will crash down hard. The crypto crashes were bad enough but at least gamers got some very cheap GP…
it cannot ever get to an AGI level that you'd assume to be competitive to a human, even most animals. Suppose you turn out to be wrong. What would convince you?
It could diagram a sentence it had never seen.
Re: 100x defect tolerance: How we solved the yield problem
#118Earlier quoted context omitted.
"While I continue to believe that many people are going to collectively lose trillions of dollars ultimately pursuing "AI" at this stage" Can you please explain more why you think so ? Thank you.
It's a hype cycle with many of the hypers and deciders having zero idea about what AI actually is and how it works. ChatGPT, while amazing, is at its core a token predictor, it cannot ever get to an AGI level that you'd assume to be competitive to a human, even most animals. And just as every other hype cycle, this one will crash down hard. The crypto crashes were bad enough but at least gamers got some very cheap GP…
Those glorified token predictors are the missing piece in the puzzle of general intelligence. There is a long way to go still in putting all those pieces together, but I don't think any of the steps left are in the same order of "we need a miracle breakthrough".
That said, I believe that this is going one of two ways: we use AI to make things materially harder for humans, in a scale from "you don't get this job" to "oops, this is Skynet", with many unpleasant stops in the middle. By the amount of money going into AI right now and most of the applications I'm seeing being hyped, I don't think we have have any scruples with this direction.
The other way this can go, and Cerebras is a good example, is that we increase our compute capability and our AI-usefulness to a point where we can fight cancer and stop/revert aging, both being a computational problem at this point. Even if most people don't realize it, or most people have strong moral objections to this outcome and don't even want to talk about it, so it probably won't happen.
In simpler words, I think we want to use AI to commit species suicide :-)
Re: 100x defect tolerance: How we solved the yield problem
#119Earlier quoted context omitted.
> I don't think the current "AI" is special in any way As someone who loves to pour ice water on AI hype, I have to say: you can't be serious. The current AI tech has opened up paths to develop applications that were impossible just a few years ago. Even if the tech freezes in place, I think it will yield substantial economic value in the coming years. It's very different from crypto, the main use case for which appe…
>It's very different from crypto, the main use case for which appears to be money laundering. Which has substantial economic value (for certain groups of people).
According to this random estimate, black market economy alone in just the US is worth ~ $2 trillion/yr. [https://www.investopedia.com/terms/u/underground-economy.asp]
Roughly 11-12% of GDP.
In many countries, black+grey market is larger than the ‘white’ market. The US is notoriously ‘clean’ compared to most (probably top 10).
Even in the US, if you suddenly stopped 10-12% of GDP we’re talking ‘great depression’ levels of economic pain.
Honestly, the only reason Crypto isn’t bigger IMO is because there is such a large and established set of folks doing laundering in the ‘normal’ system, and those work well enough there is not nearly as much demand as you’d expect.
Re: 100x defect tolerance: How we solved the yield problem
#120TSMC also have a manufacturing process used by Tesla's Dojo where you can cut up the chips, throw away the defective ones, and then reassemble working ones into a sort of wafer scale device (5x5 chips for Dojo). Seems like a more logical design to me.
Is this similar to a chiplet design? Chiplets have been a thing for a while, so I assume Cerebras avoided them on purpose.
I don't think so - chiplets are much smaller and I think the process is different.