Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RAM is the new disk – and how to measure its performance (2015)

tanelpoder.com

Post reply on HN
Print view View original

11–20 of 64 posts

Re: RAM is the new disk – and how to measure its performance (2015)

#11

Post by MayeulC »

tanelpoder wrote: post #5
rektide wrote: post #3

meanwhile, disk is potentially getting to be as fast as ram, throughput wise. 128 lanes of pcie 4.0 is 256GBps iirc. epyc's 8 channel ddr4-3200 otoh is good for 208GBps. Let's Encrypt stopped a little short, using 24x nvme disks (it fits in a 2U though so that's nice)[1]. that could be up to 96 of 128 pcie links in use. with the right ssds, working on large data-objects, that'd be somewhere a bit under 192GBps versus…

Author (of the RAM article) here: Indeed, you can go further, but got to plan for other bandwidth needed by other peripherals, data movement and inter-CPU bandwidth (NUMA) and intra-CPU-core bandwidth limitations too (AMD's infinity fabric is point to point between chiplets, but intel has some ring-bus architecture for moving bits between CPU cores). I got my Lenovo ThinkStation P620 workstation (with AMD Zen-2 Threa…

That bandwidth limitation could be due to Infinity Fabric, which seems to be rated at 42GBps (x2 as full duplex, though)?

https://en.wikichip.org/wiki/amd/infinity_fabric

Re: RAM is the new disk – and how to measure its performance (2015)

#12

Post by geocar »

nomel wrote: post #10
cogman10 wrote: post #6

Earlier quoted context omitted.

Interestingly enough, this is why Simultaneous multithreading [1] exists! The revelation that "The CPU could be doing useful work while stalled waiting on a load" has lead to CPU designers "faking" the number of cores available to the OS to allow the CPU to do more useful work while different "threads" are paused waiting on memory to come back with the data they need. [1] https://en.wikipedia.org/wiki/Simultaneous_mu…

I think the point is that this still eats into CPU memory bandwidth. Offloading could let the CPU use that memory for better purposes, especially since the stall is from memory access anyways.

Linked-list chasing (as in vlovich123's example) isn't bandwidth-limited as much as it is latency: SMT helps here because you're able to enqueue multiple wait-states "simultaneously".

Re: RAM is the new disk – and how to measure its performance (2015)

#13

Post by jeffbee »

geocar wrote: post #12
nomel wrote: post #10

Earlier quoted context omitted.

I think the point is that this still eats into CPU memory bandwidth. Offloading could let the CPU use that memory for better purposes, especially since the stall is from memory access anyways.

Linked-list chasing (as in vlovich123's example) isn't bandwidth-limited as much as it is latency: SMT helps here because you're able to enqueue multiple wait-states "simultaneously".

Anything you can do with SMT you can do without SMT using ILP instead. SMT does not grant a CPU extra cache-filling resources.

Re: RAM is the new disk – and how to measure its performance (2015)

#14

Post by tanelpoder »

nomel wrote: post #10
cogman10 wrote: post #6

Earlier quoted context omitted.

Interestingly enough, this is why Simultaneous multithreading [1] exists! The revelation that "The CPU could be doing useful work while stalled waiting on a load" has lead to CPU designers "faking" the number of cores available to the OS to allow the CPU to do more useful work while different "threads" are paused waiting on memory to come back with the data they need. [1] https://en.wikipedia.org/wiki/Simultaneous_mu…

I think the point is that this still eats into CPU memory bandwidth. Offloading could let the CPU use that memory for better purposes, especially since the stall is from memory access anyways.

When doing pointer chasing, then it's gonna be more of a latency problem, there can be plenty of memory bandwidth available, but we don't know which memory line we want to go to next, before the previous line (where the pointer resides) has been loaded. So, the CPUs spend a lot of time in "stalled backend" mode due to the big difference in CPU cycle latency vs RAM access latency.

Offloading some operations closer to the RAM would be a hardware solution (like the Oracle/Sun SPARC DAX I mentioned in a separate comment).

Or you could design your software to rely more on the memory throughput (scanning through columnar structures) vs memory latency (pointer chasing).

Btw, even with pointer-chasing, you could optimize the application to work mostly from the CPU cache (assuming that the CPUs don't do concurrent writes into these cache lines all the time), but this would require not only different application code, but different underlying data structures too. That's pretty much what my article series is about - fancy CPU throuhgput features like SIMD would not be very helpful, if the underlying data (and memory) structures don't support their way of thinking.

Re: RAM is the new disk – and how to measure its performance (2015)

#15

Post by geocar »

jeffbee wrote: post #13
geocar wrote: post #12

Earlier quoted context omitted.

Linked-list chasing (as in vlovich123's example) isn't bandwidth-limited as much as it is latency: SMT helps here because you're able to enqueue multiple wait-states "simultaneously".

Anything you can do with SMT you can do without SMT using ILP instead. SMT does not grant a CPU extra cache-filling resources.

SMT is easier to program.

Walking a single linked-list is nothing; A garbage collector walks lots of them that tend to fan out for a bit.

Re: RAM is the new disk – and how to measure its performance (2015)

#16

Post by tanelpoder »

MayeulC wrote: post #11
tanelpoder wrote: post #5

Earlier quoted context omitted.

Author (of the RAM article) here: Indeed, you can go further, but got to plan for other bandwidth needed by other peripherals, data movement and inter-CPU bandwidth (NUMA) and intra-CPU-core bandwidth limitations too (AMD's infinity fabric is point to point between chiplets, but intel has some ring-bus architecture for moving bits between CPU cores). I got my Lenovo ThinkStation P620 workstation (with AMD Zen-2 Threa…

That bandwidth limitation could be due to Infinity Fabric, which seems to be rated at 42GBps (x2 as full duplex, though)? https://en.wikichip.org/wiki/amd/infinity_fabric

Yes, that's what I'm suspecting too, although with higher clocked RAM, I should have somewhat more bandwidth. My DIMMs are 3200 MT, so should be running at 1600 MHz. But I saw a note (not sure where) that Infinity Fabric can run up to 2933 MT on my machine and it would run in sync with memory with DIMMs only up to 2933 MT. Unfortunately my BIOS doesn't allow to downgrade the RAM "clock" from 3200 MT to 2933, thus Ininity Fabric is running "out of sync" with my RAM.

This should mean non-ideal memory access latency at least, not sure how it affects throughput of large sequential transfers.

I'm planning to come up with some additional tests and hopefully write up a "part 2" too.

Re: RAM is the new disk – and how to measure its performance (2015)

#17

Post by wcerfgba »

vlovich123 wrote: post #2

I think how DMA operates needs another serious look. Right now we have to fetch everything into the CPU before we can make decisions. What if we had asynchronous HW embedded within the memory that could be given a small (safe) executable program to process the memory in-place rather than evaluating it on the CPU. In other words, a linked list would be much faster and simpler to traverse. A lot of the software archite…

This reminds me of the Connection Machine architecture [1]

> Each CM-1 microprocessor has its own 4 kilobits of random-access memory (RAM), and the hypercube-based array of them was designed to perform the same operation on multiple data points simultaneously, i.e., to execute tasks in single instruction, multiple data (SIMD) fashion. The CM-1, depending on the configuration, has as many as 65,536 individual processors, each extremely simple, processing one bit at a time.

[1] https://en.wikipedia.org/wiki/Connection_Machine

Re: RAM is the new disk – and how to measure its performance (2015)

#18

Post by jeffbee »

geocar wrote: post #15
jeffbee wrote: post #13

Earlier quoted context omitted.

Anything you can do with SMT you can do without SMT using ILP instead. SMT does not grant a CPU extra cache-filling resources.

SMT is easier to program. Walking a single linked-list is nothing; A garbage collector walks lots of them that tend to fan out for a bit.

SMT gives you some probability that thread X and thread Y are under-using the resources of the CPU but if you lose that bet you get antagonism. On Intel CPUs in particular there are only 2 slots for filling cache lines, and filling them randomly from main memory takes many cycles, so two threads can easily get starved.

If thread X is chasing a linked list and thread Y is compressing an MPEG then it's brilliant.

Re: RAM is the new disk – and how to measure its performance (2015)

#19

Post by d_tr »

vlovich123 wrote: post #2

I think how DMA operates needs another serious look. Right now we have to fetch everything into the CPU before we can make decisions. What if we had asynchronous HW embedded within the memory that could be given a small (safe) executable program to process the memory in-place rather than evaluating it on the CPU. In other words, a linked list would be much faster and simpler to traverse. A lot of the software archite…

A few months ago I wanted to take a look at the Gen-Z fabric specifications, but unfortunately they still have a lame members-only download request form in place.

Re: RAM is the new disk – and how to measure its performance (2015)

#20

Post by 1996 »

tanelpoder wrote: post #5
rektide wrote: post #3

meanwhile, disk is potentially getting to be as fast as ram, throughput wise. 128 lanes of pcie 4.0 is 256GBps iirc. epyc's 8 channel ddr4-3200 otoh is good for 208GBps. Let's Encrypt stopped a little short, using 24x nvme disks (it fits in a 2U though so that's nice)[1]. that could be up to 96 of 128 pcie links in use. with the right ssds, working on large data-objects, that'd be somewhere a bit under 192GBps versus…

Author (of the RAM article) here: Indeed, you can go further, but got to plan for other bandwidth needed by other peripherals, data movement and inter-CPU bandwidth (NUMA) and intra-CPU-core bandwidth limitations too (AMD's infinity fabric is point to point between chiplets, but intel has some ring-bus architecture for moving bits between CPU cores). I got my Lenovo ThinkStation P620 workstation (with AMD Zen-2 Threa…

> I had to move SSD cards around so they'd use separate PCIe root complexes to avoid a PCIe CPU data transfer bottleneck

I am doing similar things. Have you considered looking at how to control by software the PCI lanes assignment?

Intel HSIO seems to be software configurable - except that usually, it's all done just by the bios.

But as PCI specs allow for both device-side and host-side negotiations, it should be doable without "moving SSDs around"

> The throughput differs depending on which specific CPU cores happen to run the processes doing I/Os against different SSDs.

That strikes me as odd. I would check the detail of the PCI lanes and their routing. You could have something funky going on. My first guess would be that it's slow on one core because it's also handling something else, by design or by accident.

There're some bad hardware designs out there. But thanks to stuff like HSIO, it should now be possible to fix the worst ones by software (how else would the bios do it otherwise!) just like in the old days of isapnptools!

Post reply on HN