41–45 of 45 posts
The memory and bandwidth numbers are mind blowing. Going to be very hard to catch Nvidia. It’s as if competitors are going through the motions for participation prizes.
The closest is Google with their TPUs.
Earlier quoted context omitted.
Does sparse mean anything other than we can not actually do as many FP8 operations per second as we just claimed? To me it sounds like they can do X matrix operations per second on sparse matrices using Y FP8 operations per second, but instead of just saying what Y is they tell us how many FP8 operations would be required if the matrices were not sparse. Is this pure marketing bullshit or is there some logic to this?…
It means a very specific sparsity pattern - 2:4, so 2 out of 4 values are not 0. It's not pure bullshit, because a matrix with 2:4 sparsity may represent more "information" than a matrix that is 50% smaller.
But this also made me wonder, how does one use this in practice? If the matrices are not tiny, then they will probably have to be incredible sparse in order to always have at least two out of four entries zero. So does this just set some entries to zero if there are not enough of them in each group of four? Does one have to ensure this on its own, reorder rows and columns and introduce zeros where required and acceptable?
I would be interesting to know what kind of next-gen models this can train. On the LLM frontier, we’re starting to hit the limits of reasoning abilities in the current gen.
The memory and bandwidth numbers are mind blowing. Going to be very hard to catch Nvidia. It’s as if competitors are going through the motions for participation prizes.
NVDA has no real competitors in DL space. The closest is Google with their TPUs.
Agree for now, but long do we think this will last though.
There really hasn’t been that great of a financial incentive to compete on DL. Nvidia themselves only recently made this a major priority.
However, now that heaps of money are being thrown at massive training runs I expect we’ll see more competition popping up. Particularly if Intel pulls off IFS and catches up on the next node increasing availability.
Why is this called a whitepaper, as this is more of a documentation and architecture overview of the cluster? Wow a CLOS topology for networking, very innovative. Details on NVLink would be great. For example, the needs and problems solved by their custom cables seemingly required by NVLink would be worth a whitepaper. Don't get me wrong, this is still great the general public can get a glimpse into Grace Hopper. And…
> Why is this called a whitepaper, as this is more of a documentation and architecture overview of the cluster? That’s what a marketing white paper is and does. It’s not an academic paper.