31–40 of 169 posts
Earlier quoted context omitted.
You do realize that AI crossed human expert performance in NLP / Vision tasks ?
Exactly how does one outperform a human expert in natural language processing?
How much memory is on the chip, and what kind is it? Under what circumstances does the chip need to access external memory? What type of communication interfaces does this chip have? Also, if the chip is the size of a wafer, is it appropriate to call it a Chip?
40GB of SRAM. Not quite big enough to fit the big models like GPT3
Most interesting aspect of wafer-scale manufacturing is yield. Even if we have 95% chip yield, as the chip size approaches the wafer-level dimensions, I don't know off top of my head what the math would be but it is going to plummet drastically. My guess is that they're handling this in the chip logic. Building resiliency by turning off cells in the wafer that didn't yield. That begs the question, how are they probin…
Not an expert in chip manufacturing but my guess is that they just disable the parts that don't work and their big numbers represent ~80% of the actual number of transistors in the wafer because they account for that manufacturing loss.
Most interesting aspect of wafer-scale manufacturing is yield. Even if we have 95% chip yield, as the chip size approaches the wafer-level dimensions, I don't know off top of my head what the math would be but it is going to plummet drastically. My guess is that they're handling this in the chip logic. Building resiliency by turning off cells in the wafer that didn't yield. That begs the question, how are they probin…
How much memory is on the chip, and what kind is it? Under what circumstances does the chip need to access external memory? What type of communication interfaces does this chip have? Also, if the chip is the size of a wafer, is it appropriate to call it a Chip?
Tom's Hardware has some nice tables comparing the specs: https://www.tomshardware.com/news/cerebras-wafer-scale-engin... (more than the IEEE) This thing pulls 15-20kW of juice!
If you look at the wafer she’s holding at the top, it’s seemingly segmented into a 12x7 grid of roughly chip-sized rectangles. That’s 84 “CPUs” at 200-240 watts each, which is pretty well in line with discrete server CPUs.
The amount of heat coming off this thing must be amazing, though.
How much memory is on the chip, and what kind is it? Under what circumstances does the chip need to access external memory? What type of communication interfaces does this chip have? Also, if the chip is the size of a wafer, is it appropriate to call it a Chip?
40GB of SRAM. Not quite big enough to fit the big models like GPT3
The 40GB of SRAM probably has tremendous bandwidth (it could all be updated every few cycles!), but the memory size is very small compared to the amount of compute available.
However, maybe a different way of looking at it is that this chip will allow the training steps on deep learning models to take a fraction of the time as a GPU. Perhaps what takes 1s on a GPU could take 10ms on this chip.
So, this product may be effective at making training happen very fast, but without substantial model size or efficiency gains.
That's still ground breaking -- you can't acheive this result on GPUs. You can't achieve this result by any parallelization or distributed training, either. The large batch sizes in distributed training do not result in the same model or one that generalizes as well.