RAM is the new disk – and how to measure its performance (2015) • Hacker News Board

Re: RAM is the new disk – and how to measure its performance (2015)

#21

Post by dragontamer » Thu, Jan 21, 2021, 9:31 PM UTC

vlovich123 wrote: post #2

I think how DMA operates needs another serious look. Right now we have to fetch everything into the CPU before we can make decisions. What if we had asynchronous HW embedded within the memory that could be given a small (safe) executable program to process the memory in-place rather than evaluating it on the CPU. In other words, a linked list would be much faster and simpler to traverse. A lot of the software archite…

> Right now we have to fetch everything into the CPU before we can make decisions.

Think about virtual memory. The memory at location #6000 is NOT at where the program thinks is at memory location #6000.

In fact, the program might be reading / writing to memory location 0x08004000 (or close to that, whatever the magic start address was on various OS like Linux / Windows). And then your CPU virtually-translates that address to memory-stick #0 column#4000 or whatever.

Because of virtual memory: all memory operations must be translated by the CPU before you actually go to RAM (and that translation may require a page-table walk in the worst case)

Re: RAM is the new disk – and how to measure its performance (2015)

#22

Post by fctorial » Thu, Jan 21, 2021, 9:36 PM UTC

cogman10 wrote: post #6

vlovich123 wrote: post #2

I think how DMA operates needs another serious look. Right now we have to fetch everything into the CPU before we can make decisions. What if we had asynchronous HW embedded within the memory that could be given a small (safe) executable program to process the memory in-place rather than evaluating it on the CPU. In other words, a linked list would be much faster and simpler to traverse. A lot of the software archite…

Interestingly enough, this is why Simultaneous multithreading [1] exists! The revelation that "The CPU could be doing useful work while stalled waiting on a load" has lead to CPU designers "faking" the number of cores available to the OS to allow the CPU to do more useful work while different "threads" are paused waiting on memory to come back with the data they need. [1] https://en.wikipedia.org/wiki/Simultaneous_mu…

They aren't faking the number. A hyperthreaded core is two separate cores that share alu, fpu etc.

Re: RAM is the new disk – and how to measure its performance (2015)

#23

Post by 1996 » Thu, Jan 21, 2021, 9:36 PM UTC

tanelpoder wrote: post #16

MayeulC wrote: post #11

Earlier quoted context omitted.

That bandwidth limitation could be due to Infinity Fabric, which seems to be rated at 42GBps (x2 as full duplex, though)? https://en.wikichip.org/wiki/amd/infinity_fabric

Yes, that's what I'm suspecting too, although with higher clocked RAM, I should have somewhat more bandwidth. My DIMMs are 3200 MT, so should be running at 1600 MHz. But I saw a note (not sure where) that Infinity Fabric can run up to 2933 MT on my machine and it would run in sync with memory with DIMMs only up to 2933 MT. Unfortunately my BIOS doesn't allow to downgrade the RAM "clock" from 3200 MT to 2933, thus Ini…

> Unfortunately my BIOS doesn't allow to downgrade the RAM "clock"

How deep are you willing to go?

The RAM clock is controlled by the memory training algorithms. They use data from the XMP, which can be edited.

The simplest is to reflash your memory sticks to alter their XMP, so the training algorithm will reach the conclusions you want. There's some Windows software to do that.

You could also implement your own MRC, something done by coreboot and the likes.

Re: RAM is the new disk – and how to measure its performance (2015)

#24

Post by gpsar » Thu, Jan 21, 2021, 9:45 PM UTC

geocar wrote: post #15

jeffbee wrote: post #13

Earlier quoted context omitted.

Anything you can do with SMT you can do without SMT using ILP instead. SMT does not grant a CPU extra cache-filling resources.

SMT is easier to program. Walking a single linked-list is nothing; A garbage collector walks lots of them that tend to fan out for a bit.

Software SMT is even better for concurrent traversals as you are not limited by the number of hw contexts https://www.linkedin.com/pulse/dont-stall-multitask-georgios...

Re: RAM is the new disk – and how to measure its performance (2015)

#25

Post by tanelpoder » Thu, Jan 21, 2021, 9:45 PM UTC

1996 wrote: post #23

tanelpoder wrote: post #16

Earlier quoted context omitted.

Yes, that's what I'm suspecting too, although with higher clocked RAM, I should have somewhat more bandwidth. My DIMMs are 3200 MT, so should be running at 1600 MHz. But I saw a note (not sure where) that Infinity Fabric can run up to 2933 MT on my machine and it would run in sync with memory with DIMMs only up to 2933 MT. Unfortunately my BIOS doesn't allow to downgrade the RAM "clock" from 3200 MT to 2933, thus Ini…

> Unfortunately my BIOS doesn't allow to downgrade the RAM "clock" How deep are you willing to go? The RAM clock is controlled by the memory training algorithms. They use data from the XMP, which can be edited. The simplest is to reflash your memory sticks to alter their XMP, so the training algorithm will reach the conclusions you want. There's some Windows software to do that. You could also implement your own MRC,…

Ha, thanks for the idea! I was briefly thinking of buying 2933 "MHz" RAM for the test (as I later would put it into my other workstation that can go up to 2600 "MHz" only), but then I realized I don't have time for this right now (will do my throughput, performance stability tests first and maybe look into getting the most out of the latency later).

Re: RAM is the new disk – and how to measure its performance (2015)

#26

Post by tanelpoder » Thu, Jan 21, 2021, 9:49 PM UTC

1996 wrote: post #20

tanelpoder wrote: post #5

Earlier quoted context omitted.

Author (of the RAM article) here: Indeed, you can go further, but got to plan for other bandwidth needed by other peripherals, data movement and inter-CPU bandwidth (NUMA) and intra-CPU-core bandwidth limitations too (AMD's infinity fabric is point to point between chiplets, but intel has some ring-bus architecture for moving bits between CPU cores). I got my Lenovo ThinkStation P620 workstation (with AMD Zen-2 Threa…

> I had to move SSD cards around so they'd use separate PCIe root complexes to avoid a PCIe CPU data transfer bottleneck I am doing similar things. Have you considered looking at how to control by software the PCI lanes assignment? Intel HSIO seems to be software configurable - except that usually, it's all done just by the bios. But as PCI specs allow for both device-side and host-side negotiations, it should be doa…

As this is an AMD machine - and as it's a workstation, not server, perhaps this is why they've restricted it in BIOS.

I'm not too much of an expert in PCI express - but if this workstation has 4 PCIe root complexes/host bridges, each capable of x32 PCIe 4.0 lanes - and there are no multi-root PCIe switches, wouldn't a lane physically have to communicate with just one PCIe root complex/CPU "port"?

Re: RAM is the new disk – and how to measure its performance (2015)

#27

Post by ben509 » Thu, Jan 21, 2021, 9:50 PM UTC

dragontamer wrote: post #21

vlovich123 wrote: post #2

I think how DMA operates needs another serious look. Right now we have to fetch everything into the CPU before we can make decisions. What if we had asynchronous HW embedded within the memory that could be given a small (safe) executable program to process the memory in-place rather than evaluating it on the CPU. In other words, a linked list would be much faster and simpler to traverse. A lot of the software archite…

> Right now we have to fetch everything into the CPU before we can make decisions. Think about virtual memory. The memory at location #6000 is NOT at where the program thinks is at memory location #6000. In fact, the program might be reading / writing to memory location 0x08004000 (or close to that, whatever the magic start address was on various OS like Linux / Windows). And then your CPU virtually-translates that a…

Well, all memory operations must be translated by an MMU. So can the MMU be a distributed beast that lives in the CPU, the memory and the IO?

Re: RAM is the new disk – and how to measure its performance (2015)

#28

Post by tanelpoder » Thu, Jan 21, 2021, 9:50 PM UTC

fctorial wrote: post #22

cogman10 wrote: post #6

Earlier quoted context omitted.

Interestingly enough, this is why Simultaneous multithreading [1] exists! The revelation that "The CPU could be doing useful work while stalled waiting on a load" has lead to CPU designers "faking" the number of cores available to the OS to allow the CPU to do more useful work while different "threads" are paused waiting on memory to come back with the data they need. [1] https://en.wikipedia.org/wiki/Simultaneous_mu…

They aren't faking the number. A hyperthreaded core is two separate cores that share alu, fpu etc.

I think it's more correct to say that a single core has two "sets of registers" (and I guess instruction decoders/dispatchers perhaps)... so it's a single core with 2 "execution entry points"?

Re: RAM is the new disk – and how to measure its performance (2015)

#29

Post by BenoitP » Thu, Jan 21, 2021, 9:56 PM UTC

vlovich123 wrote: post #2

I think how DMA operates needs another serious look. Right now we have to fetch everything into the CPU before we can make decisions. What if we had asynchronous HW embedded within the memory that could be given a small (safe) executable program to process the memory in-place rather than evaluating it on the CPU. In other words, a linked list would be much faster and simpler to traverse. A lot of the software archite…

> HW embedded within the memory that could be given a small (safe) executable program to process the memory in-place rather than evaluating it on the CPU

Well, that seems to be the exact definition of what UPMEM is doing:

https://www.upmem.com/upmem-announces-silicon-based-processi...

Between the M1, GPUs, TPUs, RISC-V interesting times are coming in hardware. I blame the physical limits, which are putting the Duke Nukem development method to an end (you promise the client 2x the same performance in 18 months, then play Duke Nukem the whole time). The only way to better performance now is through hardware specialization. That and avoiding Electron.

Re: RAM is the new disk – and how to measure its performance (2015)

#30

Post by dragontamer » Thu, Jan 21, 2021, 9:57 PM UTC

ben509 wrote: post #27

dragontamer wrote: post #21

Earlier quoted context omitted.

> Right now we have to fetch everything into the CPU before we can make decisions. Think about virtual memory. The memory at location #6000 is NOT at where the program thinks is at memory location #6000. In fact, the program might be reading / writing to memory location 0x08004000 (or close to that, whatever the magic start address was on various OS like Linux / Windows). And then your CPU virtually-translates that a…

Well, all memory operations must be translated by an MMU. So can the MMU be a distributed beast that lives in the CPU, the memory and the IO?

> So can the MMU be a distributed beast that lives in the CPU, the memory and the IO?

That "blah = blah->next" memory operation could be:

* Going to swap thanks to swap / overcommit behavior on Linux/Windows.

* Or going to a file thanks to mmap

* Going to Ethernet and to another computer thanks to RDMA

So... no. I'm pretty sure the CPU is the proper place for that kind of routing.