Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Even a laptop can run RAM externally thanks to CXL

techradar.com

Post reply on HN
Print view View original

71–80 of 85 posts

Re: Even a laptop can run RAM externally thanks to CXL

#71

Post by cmrdporcupine »

abound wrote: post #66
cmrdporcupine wrote: post #62

Earlier quoted context omitted.

For myself, I've got 3gbps fiber to the house, and a need to host something. Pondering buying a used server instead of dishing money out to AWS or GCP. Question is if the power bills make sense.

I've spent a lot of time down this rabbit hole recently. I have a Dell M1000e blade server (16 blades, each with dual CPUs). A comical amount of compute for a home, sounds like a leaf blower. Hosted a couple dozen small apps on it. Power usage was 1.5kW (ouch), made less painful by large rooftop solar. Just today, I replaced the whole thing with a single mini PC with a laptop Ryzen chip (6900HX). Pulls like 60W at th…

Yeah I think this likely where I'll end up going as well. What MiniPC did you end up working with?

The thing I'm building/hosting can benefit from plenty of cores, but for the start it's just prototype with a limited audience, so I don't need to go hardcore.

But I have a friend who has rackspace in a DC, and my thought was if I bought something in a rack form I could move it there and pay him to host it later. But I suspect it doesn't make a lot of sense, really.

Re: Even a laptop can run RAM externally thanks to CXL

#72

Post by abound »

cmrdporcupine wrote: post #71
abound wrote: post #66

Earlier quoted context omitted.

I've spent a lot of time down this rabbit hole recently. I have a Dell M1000e blade server (16 blades, each with dual CPUs). A comical amount of compute for a home, sounds like a leaf blower. Hosted a couple dozen small apps on it. Power usage was 1.5kW (ouch), made less painful by large rooftop solar. Just today, I replaced the whole thing with a single mini PC with a laptop Ryzen chip (6900HX). Pulls like 60W at th…

Yeah I think this likely where I'll end up going as well. What MiniPC did you end up working with? The thing I'm building/hosting can benefit from plenty of cores, but for the start it's just prototype with a limited audience, so I don't need to go hardcore. But I have a friend who has rackspace in a DC, and my thought was if I bought something in a rack form I could move it there and pay him to host it later. But I…

I ended up with a Beelink GTR6, picked up used on eBay for $430. It's been great so far, but it's too early to vouch for it. You lose the nice IPMI/other server management features using consumer hardware, but that's part of the tradeoff.

Rack stuff can be efficient (<100W idling), especially recent model year stuff, but it gets pricey quickly. You could spec a used 1U single socket Epyc Rome or Milan for under $2,000 if you're willing to shop around, but if you don't have other rackmount equipment, I'd stay away as it's such a price (and power) premium

Re: Even a laptop can run RAM externally thanks to CXL

#73

Post by cmrdporcupine »

abound wrote: post #72
cmrdporcupine wrote: post #71

Earlier quoted context omitted.

Yeah I think this likely where I'll end up going as well. What MiniPC did you end up working with? The thing I'm building/hosting can benefit from plenty of cores, but for the start it's just prototype with a limited audience, so I don't need to go hardcore. But I have a friend who has rackspace in a DC, and my thought was if I bought something in a rack form I could move it there and pay him to host it later. But I…

I ended up with a Beelink GTR6, picked up used on eBay for $430. It's been great so far, but it's too early to vouch for it. You lose the nice IPMI/other server management features using consumer hardware, but that's part of the tradeoff. Rack stuff can be efficient (<100W idling), especially recent model year stuff, but it gets pricey quickly. You could spec a used 1U single socket Epyc Rome or Milan for under $2,00…

There's plenty of 10-yearish old PowerEdge servers around here for $500ish CAD full of SAS drives and high core count Xeons and a pile of memory. But yeah, power and noise. And I don't have ethernet running down to the basement yet, so not super enthused about having that share office space with me.

The laptop I'm typing on is a Ryzen 6850H, so not too far off from the 6900 you're talking about and it has plenty of oomph. I might consider such a thing.

Re: Even a laptop can run RAM externally thanks to CXL

#74

Post by bogantech »

abound wrote: post #66
cmrdporcupine wrote: post #62

Earlier quoted context omitted.

For myself, I've got 3gbps fiber to the house, and a need to host something. Pondering buying a used server instead of dishing money out to AWS or GCP. Question is if the power bills make sense.

I've spent a lot of time down this rabbit hole recently. I have a Dell M1000e blade server (16 blades, each with dual CPUs). A comical amount of compute for a home, sounds like a leaf blower. Hosted a couple dozen small apps on it. Power usage was 1.5kW (ouch), made less painful by large rooftop solar. Just today, I replaced the whole thing with a single mini PC with a laptop Ryzen chip (6900HX). Pulls like 60W at th…

> I have a Dell M1000e blade server (16 blades, each with dual CPUs). A comical amount of compute for a home, sounds like a leaf blower.

That's pretty awesome! I would have loved that at one point but they're extremely loud and I remember them needing 16 amp IEC leads

Re: Even a laptop can run RAM externally thanks to CXL

#75

Post by jauntywundrkind »

arandomhuman wrote: post #16

How's the latency with these vs on board memory?

170-250ns was what they were saying a year ago, which they were saying was 2x native latency. https://www.servethehome.com/compute-express-link-cxl-latenc...

But with Genoa for example, socket to socket latency has climbed up to 220ns, and going across nodes on a socket is 110ns. I feel like CXL will be less than 2x a hit, if only because cores themselves are having higher and higher latencies. https://chipsandcheese.com/2023/07/17/genoa-x-server-v-cache...

Re: Even a laptop can run RAM externally thanks to CXL

#76

Post by jauntywundrkind »

simcop2387 wrote: post #41
e63f67dd-065b wrote: post #22

CXL is useful for many more things beyond just memory expansion (which is a "type 3 device"). The purpose of CXL is to allow for memory coherency between different CXL devices. To quote the spec on type 2 devices: > CXL Type 2 devices, in addition to fully coherent cache, also have memory, for example DDR, High-Bandwidth Memory (HBM), etc., attached to the device. These devices execute against memory, but their perfo…

I've got to imagine that at some point in the future when it's been common on server stuff for a while it'll end up as part of a next-gen "thunderbolt" style connection, simply because the silicon by then will be a commodity and it'll be cheaper to use the same thing everywhere. I'm imagining a docking station that doubles the ram, and gives it a powerful GPU/NPU/TPU for a workstation. Since the ram would be close to…

CXL 3.0 is where it gets interesting, where you can start to have switched fabric where many hosts can talk to many devices. Having that pool of ram & GPUs be on demand usable by whomever has some attractive possibilities. Also, one can just imagine having some pools of data in CXL memory that a host can just attach to & read, which seems like a cool possibility.

It does kind of upset me a bit that CXL 3.0 still seems purely host-to-switch-to-device oriented. If you have your formerly PCIe slots on your cores speaking CXL and doing directory memory over fabric, I'd really really love to be able to talk to other hosts. Maybe that happens & is possible in 3.0, but it feels like CXL isnt paving that cowpath, isn't making is obvious, and that there will be a bunch of proprietary nasty ways to bridge computers & chat over CXL that are all non-standard, & I wish CXL had been more direct about making themselves & their upcoming switched fabric viable & interesting for host-to-host.

Re: Even a laptop can run RAM externally thanks to CXL

#77

Post by gbraad »

sva_ wrote: post #60
hadlock wrote: post #9

This wasn't mentioned in the article but CXL Stands for Compute Express Link which is an open standard, CPU-to-Device Interconnect, I guess they're on v3.0 now. First I had heard of it as well. 3.9GB/s on the low end and as high as 121.0GB/s with the latest 16x 3.0 spec over serial connection https://www.computeexpresslink.org/about-cxl https://en.wikipedia.org/wiki/Compute_Express_Link

Is it possible to run an eGPU with this? Like the TH3P4G3 one can buy on AliExpress

This is based on serial PCI Express (PCIe), and allows block based transfer, cache coherence, etc. In the Device Types they talk about "Accelerators with Memory", type 2: https://www.logic-fruit.com/blog/cxl/compute-express-link-cx....

Re: Even a laptop can run RAM externally thanks to CXL

#78

Post by mr_toad »

LoganDark wrote: post #18

To my eyes, it looks like this article consists almost solely of "external RAM exists". OK? What laptops even support it? How much capacity is there ("over 1TB" is kind of vague)? So many questions...

The current devices are full size PCIe. And you need the latest Xeon CPU.

Re: Even a laptop can run RAM externally thanks to CXL

#79

Post by rldjbpin »

Dell has been pushing a new memory standard for laptops (https://news.ycombinator.com/item?id=33048948) and even there you face the realities of latency tradeoffs when physically moving the memory away from the CPU.

i would like to see whether the high memory use cases (like running LLMs) can benefit more from direct memory access to GPU that efforts such as DirectStorage is working on (https://news.ycombinator.com/item?id=33265666) and just faster, low-latency drives can bridge this gap faster.

Re: Even a laptop can run RAM externally thanks to CXL

#80

Post by LoganDark »

mr_toad wrote: post #78
LoganDark wrote: post #18

To my eyes, it looks like this article consists almost solely of "external RAM exists". OK? What laptops even support it? How much capacity is there ("over 1TB" is kind of vague)? So many questions...

The current devices are full size PCIe. And you need the latest Xeon CPU.

Are these the same Xeons that require a recurring subscription to unlock the silicon you've already purchased?
Post reply on HN