What are the row of green rectangles in the middles of the longe edges?
Maybe VRMs (although I've never seen VRMs that look like that).
Judging by the pictures at the end of the article, it looks like the VRM are under the board o_O
Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
181–188 of 188 posts
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#182Earlier quoted context omitted.
Did you watch the Nvidia 2024 GTC keynote[1]? The CEO of NVIDIA was pretty clear: Nvidia's focus is now on selling Blackwell AI Factories. According to some napkin math, each of these Blackwell AI factories will have an annual power bill of roughly $8,000,000 USD and if you do a little apples to oranges comparison will outperform the computer that's currently #1 on the supercomputing top 500 (Frontier at Oak Ridge) b…
When moving into a market where there are no competitors one should be wary that this may be because there are no customers.
The topic of customer interest is addressed at timestamp 59 minutes and zero seconds of the keynote.
By my count, 49 corporate logos were shown as launch partners.
At one point in the talk the CEO claims that “Blackwell will be Nvidia’s most successful product ever”.
—
And again, the product he’s talking about is a data centers packed with 56 racks…
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#183About time. NVIDIA needs some serious competition.
Did you watch the Nvidia 2024 GTC keynote[1]? The CEO of NVIDIA was pretty clear: Nvidia's focus is now on selling Blackwell AI Factories. According to some napkin math, each of these Blackwell AI factories will have an annual power bill of roughly $8,000,000 USD and if you do a little apples to oranges comparison will outperform the computer that's currently #1 on the supercomputing top 500 (Frontier at Oak Ridge) b…
Not to rain on the parade here but nvidia is not going to haul a 1 to 2 order of magnitude in a single jump. Top500 measures fp64, nvidia likes to quote fp8. Blackwell is actually a regression on fp64 performance to provide more die space for AI compute. This isn't to say the silicon isn't good just different goals being hit.
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#184Earlier quoted context omitted.
Did you watch the Nvidia 2024 GTC keynote[1]? The CEO of NVIDIA was pretty clear: Nvidia's focus is now on selling Blackwell AI Factories. According to some napkin math, each of these Blackwell AI factories will have an annual power bill of roughly $8,000,000 USD and if you do a little apples to oranges comparison will outperform the computer that's currently #1 on the supercomputing top 500 (Frontier at Oak Ridge) b…
Not to rain on the parade here but nvidia is not going to haul a 1 to 2 order of magnitude in a single jump. Top500 measures fp64, nvidia likes to quote fp8. Blackwell is actually a regression on fp64 performance to provide more die space for AI compute. This isn't to say the silicon isn't good just different goals being hit.
I’m aware.
In the keynote Nvidia is quoting FP4 performance for Blackwell specifically. Frontier and every computer that has ever hit the supercomputer top 500 is measuring fp64. This is why I qualified my statement above by saying if you’re willing to compare apples and oranges.
But I think this is where things open up to debate (and/or personal interpretation). My view is, if all you care about is AI workload (Specifically LLMs), then you really are seeing a full two orders of magnitude improvement. And if we are trying to get a feel for the space, and what that even means, then there really is nothing else to compare Blackwell to other than frontier, in terms of scale alone.
Above I say “one or two orders of magnitude”, the “one order of magnitude” is a value I got by taking Blackwell’s FP4 values and dividing by 16, to try to conceptually convert back to a value that can be compared to fp64. — I’m perfectly happy to admit that all I’m really capable of doing here is using estimates to compare a completely new thing (the new compute architecture of Blackwell) to what came before it (the majority of the history of the supercomputing 500). But that’s okay, because that’s my only goal.
—-
Believe me, I get it. Blackwell will never post a linpack score. Blackwell will never run big scientific computing jobs like… global weather simulations.
But as computer science nerds, how could we not be excited to see such a big move: changing the computer architecture to better suit a specific compute workload (LLMs)?
I know Bitcoin has its asic processors, but to me Blackwell’s mission statement of “computing intelligence” is a lot more interesting.
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#185Earlier quoted context omitted.
pokes OpenCL's corpse with a stick C'mon, do something...
Its very unfortunate. One of our vendors has spent a hell of an effort making spatial analysis tools work via OpenCL instead of being just processor bound, and its made those industry standard libraries 1-2 orders of magnitude faster. That is something the spatial industry desperately needs in order to improve iteratively. With the state of OpenCL it's frustrating. So much to be had, but so little improvement and sup…
In 50 years, I feel like GPGPU compute will be told as a Greek tragedy. Nvidia, pariah of Apple, would create a parallel compute ecosystem that was so powerful that even the combined effort of the industry couldn't topple it. At every corner where they could have thwarted them, Nvidia's competitors refused to up the ante or chided CUDA's efforts as silly and unnecessary. One crypto/AI craze later, everyone and their mother is berating their OEM for ignoring such basic functionality and refusing to collaborate on a competing alternative.
Frustrating is a good way to put it, but there's a certain causal satisfaction I feel from watching it all unfold. Of course everyone loses to CUDA when they refuse to sponsor an Open Source alternative. It's fascinating to me that hardware manufacturers would rather let CUDA dominate than establish a basic working relationship.
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#186Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#187Earlier quoted context omitted.
What does?
NVLink
That’s not how NVIDIA’s solutions work; if you connect a DGX system to a NVLINK switch it will talk NVLINK if you connect it to another via Infiniband it will speak Infiniband, if you do PCIe Direct Attach it would talk that too and if you connect it via Ethernet it you guessed it talk Ethernet with RDMA and all the bells and whistles to boot.
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#188NVDA customers are willing to pay to much for AI chips so they get the money back and more from the pumped up NVDA stock price . This is inflationary and we are still stuck with high interest rates because of those schmucks!