So, where can I use instances of Gaudi 3 and what is the hourly price for these instances?
Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
11–20 of 188 posts
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#12The "in the wild" part of the title is misleading. These chips are being presented in a very controlled environment. An interesting aspect of Intel's design is they use Ethernet for connectivity. If they can get the performance on par with NVLink, that by itself could be a win because everybody knows how to manage Ethernet. Very few people know how to manage an NVLink network. To be clear, this is data center hardwar…
You don’t need to manage NVLink.
NVLink either talks native NVlink to itself when you are using NVlink switches either intra-server or intra-rack or;
It can talk PCIe over NVlink when talking to a PCIe endpoint.
Or you can run Infiniband or Ethernet on top of it and talk to w/e is on the other side.
Gaudi isn’t that different remember Ethernet != TCP/IP.
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#13The "in the wild" part of the title is misleading. These chips are being presented in a very controlled environment. An interesting aspect of Intel's design is they use Ethernet for connectivity. If they can get the performance on par with NVLink, that by itself could be a win because everybody knows how to manage Ethernet. Very few people know how to manage an NVLink network. To be clear, this is data center hardwar…
[deleted]
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#14About time. NVIDIA needs some serious competition.
AMD is doing several billion in ai processor sales already and the new chip is selling as fast as they can make them. At least with AMD, a customer can actually get them now as opposed to the nearly 1 year lead time from nVidia.
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#15About time. NVIDIA needs some serious competition.
The only meaningful hardware competition, meaning lower prices, will come from Chinese designed, Chinese manufactured parts. This is still a long ways out.
Is it inevitable? I think so. Before 2019 there wasn't an opportunity, now there is.
For software, Chinese universities, Alibaba, Tencent and Bytedace are already releasing models, training code and in rare cases datasets that are competitive with private offerings. CogVLM/CogAgent is one that I use. It's very promising.
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#16The "in the wild" part of the title is misleading. These chips are being presented in a very controlled environment. An interesting aspect of Intel's design is they use Ethernet for connectivity. If they can get the performance on par with NVLink, that by itself could be a win because everybody knows how to manage Ethernet. Very few people know how to manage an NVLink network. To be clear, this is data center hardwar…
You don’t need to manage NVLink. NVLink either talks native NVlink to itself when you are using NVlink switches either intra-server or intra-rack or; It can talk PCIe over NVlink when talking to a PCIe endpoint. Or you can run Infiniband or Ethernet on top of it and talk to w/e is on the other side. Gaudi isn’t that different remember Ethernet != TCP/IP.
So it works with Ethernet switches?
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#17About time. NVIDIA needs some serious competition.
pokes OpenCL's corpse with a stick C'mon, do something...
https://github.com/ROCm/ROCm/issues/2754
Wow and I thought that the latest generation of GPUs was better.
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#18About time. NVIDIA needs some serious competition.
Competition doesn't do much when all production everywhere is already taken in preorders. It'll only change when there is surplus production.
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#19The "in the wild" part of the title is misleading. These chips are being presented in a very controlled environment. An interesting aspect of Intel's design is they use Ethernet for connectivity. If they can get the performance on par with NVLink, that by itself could be a win because everybody knows how to manage Ethernet. Very few people know how to manage an NVLink network. To be clear, this is data center hardwar…
Very few people actually know how to provision and manage a lossless Ethernet fabric and I’d wager someone who had literally never touched infiniband would have an easier time accomplishing it from zero with IB than with Ethernet on real vendor gear. Ethernet has so, so many gotchas. Maybe if it was a layer 3 only network it would work. Maybe.
I'm guessing it's RDMA over ethernet too which often has a lot of gotchas depending on the exact hardware being used.
Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild
#20Earlier quoted context omitted.
You don’t need to manage NVLink. NVLink either talks native NVlink to itself when you are using NVlink switches either intra-server or intra-rack or; It can talk PCIe over NVlink when talking to a PCIe endpoint. Or you can run Infiniband or Ethernet on top of it and talk to w/e is on the other side. Gaudi isn’t that different remember Ethernet != TCP/IP.
So it works with Ethernet switches?
What does?