Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I/O is no longer the bottleneck? (2022)

stoppels.ch

Post reply on HN
Print view View original

21–30 of 133 posts

Re: I/O is no longer the bottleneck? (2022)

#21

Post by geoctl »

throwaway94275 wrote: post #14
geoctl wrote: post #10

Earlier quoted context omitted.

With modern CXL/PCIe, I guess it's not going to be that stupid to claim that RAM/memory controller is slowly becoming I/O on its own.

Old IBM's term for RAM was "storage."

I wonder whether the current huge funding in AI will ever lead to a revolution in computer architecture. Modern PCIe/CXL is already starting to blur the difference between memory and I/O. Maybe the future is going to be that CPUs, RAM, storage devices, GPUs and other devices are going to directly address one another like a mesh network. Maybe the entire virtual memory model will change to include everything to be addressed via a unified virtual memory space from a process/CPU perspective with simple read/write syscalls that translate into network packets flowing between the CPU and the destination (e.g. RAM/GPU/NVMe) and vice versa.

Re: I/O is no longer the bottleneck? (2022)

#22

Post by gary_0 »

Not a new idea, but it's intriguing to think about an architecture that's just: CPU caches nonvolatile storage

What if you could take it for granted that mmap()ing a file has the exact same performance characteristics as malloc(), aside from the data not going away when you free the address space? What if arbitrary program memory could be given a filename and casually handed off to the OS to make persistent? A lot of basic software design assumptions are still based on the constraints of the spinning rust era...

Re: I/O is no longer the bottleneck? (2022)

#23

Post by nine_k »

lunixbochs wrote: post #17
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

your single core numbers seem way too low for peak throughput on one core, unless you stipulate that all cores are active and contending with each other for bandwidth e.g. dual channel zen 1 showing 25GB/s on a single core https://stackoverflow.com/a/44948720 I wrote some microbenchmarks for single-threaded memcpy zen 2 (8-channel DDR4) naive c: 17GB/s non-temporal avx: 35GB/s Xeon-D 1541 (2-channel DDR4, my weakest…

As much as I can understand a Zen 5 CPU core can run two AVX512 operations per clock (1024 bits) + 4 integer operations per clock (which use up FPU circuitry in the process), so additional 256 bits. At 4 GHz, this is 640 GB/s.

I suppose that in real life such ideal condition do not occur, but it shows how badly the CPU is limited by its memory bandwidth for streaming tasks. Its maximum memory-read bandwidth is 768 bits per clock. only 60% of its peak bit-crunching performance. DRAM bandwidth is even more limiting. And this is a single core of at least 12 (and at most 64).

Re: I/O is no longer the bottleneck? (2022)

#24

Post by eliasdejong »

johncolanduoni wrote: post #9
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

What is the nature of the architectural limit here? The bus between an individual core and the caches and/or memory controller?

The limit is the number of outstanding cache line requests to the memory controller. CPUs have a fixed number of slots for this, around 10-12 usually. Intel calls them LFBs (Line Fill Buffers) and AMD MSHRs (Miss Status Holding Registers). When the slots are filled, the CPU can issue no more requests and has to wait for them to complete. Apple M chips (probably) have more slots and the memory is physically packaged together with the CPU, so they get better numbers.

Re: I/O is no longer the bottleneck? (2022)

#26

Post by nine_k »

ThreatSystems wrote: post #16

*Unless your in the cloud, then it's a metric to nickel and dime with throttling! On a more serious note, the performance of hardware today is mind boggling from what we all encountered way back when. What I struggle to comprehend though is how some software (particularly Windows as an OS, instant messaging applications etc.) feel less performant now than they ever were.

The answer, I suspect, is is the same as always: waiting for I/O in the GUI thread.

Both Telegram and FB messenger are snappy; I didn't use anything else seriously as of late. (Especially not Teams, nor the late Skype.)

Re: I/O is no longer the bottleneck? (2022)

#27

Post by npn »

kevmo314 wrote: post #11

This was my instinct when NVMe SSDs first came out: I'd joke that now we have 2 TB of RAM. The real joke is on me though, some of these GPU servers actually have 2 TB of RAM now. Crazy engineering!

Now? I had found some used epyc servers with 2TB ddr4 ram for around 5k usd yesteryear. Too bad I didn't purchase it.

Re: I/O is no longer the bottleneck? (2022)

#28

Post by hamandcheese »

eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

Lite claims that it can be modified in-place, but I'm curious how that works with variable-length structures like strings?

Re: I/O is no longer the bottleneck? (2022)

#29

Post by kvemkon »

nine_k wrote: post #23
lunixbochs wrote: post #17

Earlier quoted context omitted.

your single core numbers seem way too low for peak throughput on one core, unless you stipulate that all cores are active and contending with each other for bandwidth e.g. dual channel zen 1 showing 25GB/s on a single core https://stackoverflow.com/a/44948720 I wrote some microbenchmarks for single-threaded memcpy zen 2 (8-channel DDR4) naive c: 17GB/s non-temporal avx: 35GB/s Xeon-D 1541 (2-channel DDR4, my weakest…

As much as I can understand a Zen 5 CPU core can run two AVX512 operations per clock (1024 bits) + 4 integer operations per clock (which use up FPU circuitry in the process), so additional 256 bits. At 4 GHz, this is 640 GB/s. I suppose that in real life such ideal condition do not occur, but it shows how badly the CPU is limited by its memory bandwidth for streaming tasks. Its maximum memory-read bandwidth is 768 bi…

> CPU is limited by its memory bandwidth for streaming tasks

That must be the reason, why EPYC 9175F exists. It is only 16-core CPU, but all 16 8-core CCDs are populated and only one core on each is active.

The next gen EPYC is rumored to have 16 instead of 12 memory channels (which were 8 only 4-5 years ago).

Re: I/O is no longer the bottleneck? (2022)

#30

Post by sroussey »

npn wrote: post #27
kevmo314 wrote: post #11

This was my instinct when NVMe SSDs first came out: I'd joke that now we have 2 TB of RAM. The real joke is on me though, some of these GPU servers actually have 2 TB of RAM now. Crazy engineering!

Now? I had found some used epyc servers with 2TB ddr4 ram for around 5k usd yesteryear. Too bad I didn't purchase it.

He said GPU servers
Post reply on HN