31–40 of 63 posts
I find it weird that not everyone agree Meta and Facebook and social networks in general are doing some good the the society and our democracies; yet they manage to spend incredible amount of money/energy/time to develop solutions to problems we aren't exactly sure are worth solving…
I assume this helps reduce their server and electricity costs. At a certain scale these things pay off.
Intel Gaudi 3 has more interconnect bandwidth than this has memory bandwidth. By a lot. I guess they can't be fairly compared without knowing the TCO for each. I know in the past Google's TPU per-chip specs lagged Nvidia but the much lower TCO made them a slam dunk for Google's inference workloads. But this seems pretty far behind the state of the art. No FP8 either.
Intel Gaudi 3 has more interconnect bandwidth than this has memory bandwidth. By a lot. I guess they can't be fairly compared without knowing the TCO for each. I know in the past Google's TPU per-chip specs lagged Nvidia but the much lower TCO made them a slam dunk for Google's inference workloads. But this seems pretty far behind the state of the art. No FP8 either.
Also its at 90 watts vs 900 watts for gaudi 3, the flops/mem bw per watt is much more comparable.
Can't imagine any other reason other than cost as to why they went with LPDDR5, LPDDR5X has more bandwidth and GDDR6 has even more.
Still seems pretty primitive. Very cool though. I can only imagine the lack of fear Jensen experiences when reading this.
I thought MTIA v2 would use the mx formats https://arxiv.org/pdf/2302.08007.pdf , guess they were too far along in the process to get it in this time. Still this looks like it would make for an amazing prosumer home ai setup. Could probably fit 12 accelerators on a wall outlet with change for a cpu, would have enough memory to serve a 2T model at 4bit and reasonable dense performance for small training runs and image…
I suppose it might, there are not a lot of details (what kind of sparsity for example?) about what they mean in terms of INT8 support - it could be MXINT8, or something else. Glad someone was thinking the same thing I was though!
Wishful thinking maybe they'll announce selling it with the giant llama3 cause there's no good, cheap way to inference something like that at home at the moment and this could change that.
Earlier quoted context omitted.
Also its at 90 watts vs 900 watts for gaudi 3, the flops/mem bw per watt is much more comparable.
It would be interesting if this could be made into a reasonably priced (lmao) card for home inference if they intend to mass produce it. Can't imagine any other reason other than cost as to why they went with LPDDR5, LPDDR5X has more bandwidth and GDDR6 has even more.
Earlier quoted context omitted.
To me, it’s bizarre to see the HPC mindset taking hold again after the cloud/commodity mindset dominated the last 16 years. You don’t always need a Ferrari to go to the store
WDYM by HPC mindset?
Sorry if this make anyone feels bad. It certainly made myself uncomfortable typing it out though.
Still seems pretty primitive. Very cool though. I can only imagine the lack of fear Jensen experiences when reading this.
I like the interactive 3D widget showing off the chip. Yep, that sure is a metal rectangle.
Really annoys me that the loading animation of these before-/after-images doesn't finish on firefox and that it won't let me drag the knob with the separator. ...no "Under the hood" for me.