Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GigaByte CXL memory expansion card with up to 512GB DRAM

gigabyte.com

Post reply on HN
Print view View original

31–40 of 67 posts

Re: GigaByte CXL memory expansion card with up to 512GB DRAM

#31

Post by Dylan16807 »

reilly3000 wrote: post #26
cjensen wrote: post #3

Earlier quoted context omitted.

Both of the supported motherboards support installation of 2TB of DRAM.

Presumably this is about adding more memory channels via pcie lanes. I’m very curious to know what kind of bandwidth one could expect with such a setup, as that is the primary bottleneck for inference speed.

The raw speed of PCIe 5.0 x16 is 63 billion bytes per second each way. Assuming we transfer several cache lines at a time the overhead should be pretty small, so expect 50-60GB/s. Which is on par with a single high-clocked channel of DRAM.

Re: GigaByte CXL memory expansion card with up to 512GB DRAM

#32

Post by immibis »

Twirrim wrote: post #7

CXL is going to be really interesting. On the positive side, you can scale out memory quite a lot, fill up PCI slots, even have memory external to your chassis. Memory tiering has a lot of potential. On the negative side, you've got latency costs to swallow up. You don't get distance from CPU for free (there's a reason the memory on your motherboard is as close as practical to the CPU) https://www.nextplatform.com/20…

What kind of motherboard, CPU, cables, switches, and end devices would I need to buy to have a CXL network?

Re: GigaByte CXL memory expansion card with up to 512GB DRAM

#33

Post by JonChesterfield »

wmf wrote: post #30
JonChesterfield wrote: post #29

I don't get it. The point of (ddrN) memory is latency. If its on the far side of pcie latency is much worse than the system memory. In what sense is this better than ssd on the far side of pcie?

It's only ~2x worse latency than main memory but 100x lower than SSD.

I'm finding ~50ns best case for pcie, ~10ns for system. Which is a lot closer than I expected.

Re: GigaByte CXL memory expansion card with up to 512GB DRAM

#34

Post by packetlost »

tanelpoder wrote: post #10
Twirrim wrote: post #7

CXL is going to be really interesting. On the positive side, you can scale out memory quite a lot, fill up PCI slots, even have memory external to your chassis. Memory tiering has a lot of potential. On the negative side, you've got latency costs to swallow up. You don't get distance from CPU for free (there's a reason the memory on your motherboard is as close as practical to the CPU) https://www.nextplatform.com/20…

Yup, for best results you wouldn't just dump your existing pointer-chasing and linked-list data structures to CXL (like the Optane's transparent mode was, whatever it was called). But CXL-backed memory can use your CPU caches as usual and the PCIe 5.0 lane throughput is still good, assuming that the CXL controller/DRAM side doesn't become a bottleneck. So you could design your engines and data structures to account f…

> You probably don't want to have global shared locks and frequent atomic operations on CXL-backed shared memory (once that becomes possible in theory with CXL3.0).

I'd bet contested locks spend more time in cache than most other lines of memory so in practice a global lock might not be too bad.

Re: GigaByte CXL memory expansion card with up to 512GB DRAM

#35

Post by bobmcnamara »

kvemkon wrote: post #16
jonhohle wrote: post #14

Why did something like this take so long to exist? I’ve always wanted swap or tmpfs available on old RAM I have lying around.

I'd have rather a question why we had single (or already dual) core CPUs with dual-channel memory controller and now we have 16-core CPUs but still with only dual-channel RAM.

Intel and AMD I'd reckon. Apple went wide with their busses.

Re: GigaByte CXL memory expansion card with up to 512GB DRAM

#36

Post by bobmcnamara »

bri3d wrote: post #6

CXL is a standard for compute and I/O extension over PCIe signaling which has been around for a few years, with a couple of available RAM boards (from SMART and others). I think the main bridge chipsets come from Microchip (this one) and Montage. This Gigabyte product is interesting since it’s a little lower end than most AXL solutions - so far AXL memory expansion has mostly appeared in esoteric racked designs like…

CXL seems so much cleaner than the old AMD way of plumbing an FPGA through the second CPU socket.

Re: GigaByte CXL memory expansion card with up to 512GB DRAM

#38

Post by marcopolis »

jauntywundrkind wrote: post #37

I wonder whose controller they are using. For a memory controller, that thing looks hot!

From the manual, it looks like a Microchip PM8712 [1].

[1] PDF Data sheet: https://ww1.microchip.com/downloads/aemDocuments/documents/D...

Re: GigaByte CXL memory expansion card with up to 512GB DRAM

#39

Post by tanelpoder »

packetlost wrote: post #34
tanelpoder wrote: post #10

Earlier quoted context omitted.

Yup, for best results you wouldn't just dump your existing pointer-chasing and linked-list data structures to CXL (like the Optane's transparent mode was, whatever it was called). But CXL-backed memory can use your CPU caches as usual and the PCIe 5.0 lane throughput is still good, assuming that the CXL controller/DRAM side doesn't become a bottleneck. So you could design your engines and data structures to account f…

> You probably don't want to have global shared locks and frequent atomic operations on CXL-backed shared memory (once that becomes possible in theory with CXL3.0). I'd bet contested locks spend more time in cache than most other lines of memory so in practice a global lock might not be too bad.

Yep agreed, for single-host with CXL scenarios. I wrote this comment thinking about a hypothetical future CXL3.x+ scenario with multi-host fabric coherence where one could in theory put locks and control structures that protect shared access to CXL memory pools into the same shared CXL memory (so, no need for coordination over regular network at least).

Re: GigaByte CXL memory expansion card with up to 512GB DRAM

#40

Post by wmf »

immibis wrote: post #32
Twirrim wrote: post #7

CXL is going to be really interesting. On the positive side, you can scale out memory quite a lot, fill up PCI slots, even have memory external to your chassis. Memory tiering has a lot of potential. On the negative side, you've got latency costs to swallow up. You don't get distance from CPU for free (there's a reason the memory on your motherboard is as close as practical to the CPU) https://www.nextplatform.com/20…

What kind of motherboard, CPU, cables, switches, and end devices would I need to buy to have a CXL network?

CXL networking is still in the R&D stage.
Post reply on HN