GigaByte CXL memory expansion card with up to 512GB DRAM
1–10 of 67 posts
Re: GigaByte CXL memory expansion card with up to 512GB DRAM
#2That is amazing. Most consumer boards will only have 32 or 64. To have 512 is great!
Re: GigaByte CXL memory expansion card with up to 512GB DRAM
#3That is amazing. Most consumer boards will only have 32 or 64. To have 512 is great!
Both of the supported motherboards support installation of 2TB of DRAM.
Re: GigaByte CXL memory expansion card with up to 512GB DRAM
#4> Buy From One of the Regions Below
> Egypt
:-/
But, because I'm a good sport, I actually chased a couple of those links figuring that I could convert Egyptian Pound into USD but https://www.sigma-computer.com/en/search?q=CXL%20R5X4> is "No results", and similar for the other ones that I could get to even load
Re: GigaByte CXL memory expansion card with up to 512GB DRAM
#5That is amazing. Most consumer boards will only have 32 or 64. To have 512 is great!
... and if you have the money, you can use 3 out of 4 PCIe5 slots for CXL expansion. So that could be 2TB DRAM + 1.5TB DRAM-over-CXL, all cache coherent thanks to CXL.mem.
I guess there are some use cases for this for local users, but I think the biggest wins could come from the CXL shared memory arrays in smaller clusters. So you could, for example, cache the entire build-side of a big hash join in the shared CXL memory and let all other nodes performing the join see the single shared dataset. Or build a "coherent global buffer cache" using CPU+PCI+CXL hardware, like Oracle Real Application Clusters has been doing with software+NICs for the last 30 years.
Edit: One example of the CXL shared memory pool devices is Samsung CMM-B. Still just an announcement, haven't seen it in the wild. So, CXL arrays might become something like the SAN arrays in the future - with direct loading to CPU cache (with cache coherence) and being byte-addressable.
https://semiconductor.samsung.com/news-events/tech-blog/cxl-...
Re: GigaByte CXL memory expansion card with up to 512GB DRAM
#6CXL is a standard for compute and I/O extension over PCIe signaling which has been around for a few years, with a couple of available RAM boards (from SMART and others).
I think the main bridge chipsets come from Microchip (this one) and Montage.
This Gigabyte product is interesting since it’s a little lower end than most AXL solutions - so far AXL memory expansion has mostly appeared in esoteric racked designs like the particularly wild https://www.servethehome.com/cxl-paradigm-shift-asus-rs520qa... .
Re: GigaByte CXL memory expansion card with up to 512GB DRAM
#7CXL is going to be really interesting.
On the positive side, you can scale out memory quite a lot, fill up PCI slots, even have memory external to your chassis. Memory tiering has a lot of potential.
On the negative side, you've got latency costs to swallow up. You don't get distance from CPU for free (there's a reason the memory on your motherboard is as close as practical to the CPU) https://www.nextplatform.com/2022/12/05/just-how-bad-is-cxl-.... CXL spec for 2.0 is at about 200ns of latency added to all calls to what is stored in memory, so when using it you've got to think carefully about how you approach using it, or you'll cripple yourself.
There's been work on the OS side around data locality, but CXL stuff hasn't been widely available, so there's an element of "Well, we'll have to see".
Azure has some interesting whitepapers out as they've been investigating ways to use CXL with VMs, https://www.microsoft.com/en-us/research/wp-content/uploads/....
Re: GigaByte CXL memory expansion card with up to 512GB DRAM
#8> Buy From One of the Regions Below > Egypt :-/ But, because I'm a good sport, I actually chased a couple of those links figuring that I could convert Egyptian Pound into USD but https://www.sigma-computer.com/en/search?q=CXL%20R5X4 > is "No results", and similar for the other ones that I could get to even load
Yeah I saw the same. I've been keeping an eye on the CXL world for ~5 years and so far it's 99% announcements, unveilings and great predictions. But the only CXL cards a consumer/small business can buy are some experimental-ish 64GB/128GB cards that you can actually buy today. Haven't seen any of my larger clients use it either. Both Intel Optane and DSSD storage efforts got discontinued after years of fanfare, from technical point of view, I hope that the same doesn't happen to CXL.
Re: GigaByte CXL memory expansion card with up to 512GB DRAM
#9My god - a CXL product! That's really surprising anything go that far.
I'd been expecting external CXL boxes, not internal stuff.
Re: GigaByte CXL memory expansion card with up to 512GB DRAM
#10CXL is going to be really interesting. On the positive side, you can scale out memory quite a lot, fill up PCI slots, even have memory external to your chassis. Memory tiering has a lot of potential. On the negative side, you've got latency costs to swallow up. You don't get distance from CPU for free (there's a reason the memory on your motherboard is as close as practical to the CPU) https://www.nextplatform.com/20…
Yup, for best results you wouldn't just dump your existing pointer-chasing and linked-list data structures to CXL (like the Optane's transparent mode was, whatever it was called).
But CXL-backed memory can use your CPU caches as usual and the PCIe 5.0 lane throughput is still good, assuming that the CXL controller/DRAM side doesn't become a bottleneck. So you could design your engines and data structures to account for these tradeoffs. Like fetching/scanning columnar data structures, prefetching to hide latency etc. You probably don't want to have global shared locks and frequent atomic operations on CXL-backed shared memory (once that becomes possible in theory with CXL3.0).
Edit: I'll plug my own article here - if you've wondered whether there were actual large-scale commercial products that used Intel's Optane as intended then Oracle database took good advantage of it (both the Exadata and plain database engines). One use was to have low latency durable (local) commits on Optane:
https://tanelpoder.com/posts/testing-oracles-use-of-optane-p...
VMware supports it as well, but using it as a simpler layer for tiered memory.