Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I/O is no longer the bottleneck? (2022)

stoppels.ch

Post reply on HN
Print view View original

91–100 of 133 posts

Re: I/O is no longer the bottleneck? (2022)

#91

Post by torginus »

lunixbochs wrote: post #17
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

your single core numbers seem way too low for peak throughput on one core, unless you stipulate that all cores are active and contending with each other for bandwidth e.g. dual channel zen 1 showing 25GB/s on a single core https://stackoverflow.com/a/44948720 I wrote some microbenchmarks for single-threaded memcpy zen 2 (8-channel DDR4) naive c: 17GB/s non-temporal avx: 35GB/s Xeon-D 1541 (2-channel DDR4, my weakest…

Thanks for the detailed writeup! This made me think of an interesting conundrum - with how much RAM modern computers come with (16GB is considered to be on the small side), having the CPU read the entire contents of RAM takes a nontrivial amount of time.

A single threaded Zen2 program could very well take 1 second to scan through your RAM, during which it's entirely trivial to read stuff from disk, so the modern practice of keeping a ton of stuff in RAM might be actually hurting performance.

Algorithms, such as garbage collection, which scan the entire heap, where the single-threaded version is probably slower than the naive zen 2 memcpy might run for more than a second even on a comparatively modest 16GB heap, which might not be even acceptable.

Re: I/O is no longer the bottleneck? (2022)

#93

Post by imtringued »

zozbot234 wrote: post #84
gary_0 wrote: post #22

Not a new idea, but it's intriguing to think about an architecture that's just: CPU caches nonvolatile storage What if you could take it for granted that mmap()ing a file has the exact same performance characteristics as malloc(), aside from the data not going away when you free the address space? What if arbitrary program memory could be given a filename and casually handed off to the OS to make persistent? A lot of…

> A lot of basic software design assumptions are still based on the constraints of the spinning rust era... fsync() is still slow, and you need that for real persistence. It's not just about spinning rust, there's very good reasons for wanting a different treatment of clearly ephemeral/scratchpad storage.

Isn't the problem with fsync() that you can't wait for specific IO operations? You're waiting for every single operation on the file descriptor. If anything happens between your write and the call to fsync(), you're going to get delayed by the additional operation that managed to get through.

Re: I/O is no longer the bottleneck? (2022)

#94

Post by zozbot234 »

torginus wrote: post #91
lunixbochs wrote: post #17

Earlier quoted context omitted.

your single core numbers seem way too low for peak throughput on one core, unless you stipulate that all cores are active and contending with each other for bandwidth e.g. dual channel zen 1 showing 25GB/s on a single core https://stackoverflow.com/a/44948720 I wrote some microbenchmarks for single-threaded memcpy zen 2 (8-channel DDR4) naive c: 17GB/s non-temporal avx: 35GB/s Xeon-D 1541 (2-channel DDR4, my weakest…

Thanks for the detailed writeup! This made me think of an interesting conundrum - with how much RAM modern computers come with (16GB is considered to be on the small side), having the CPU read the entire contents of RAM takes a nontrivial amount of time. A single threaded Zen2 program could very well take 1 second to scan through your RAM, during which it's entirely trivial to read stuff from disk, so the modern prac…

It's true that GC is a huge drain on memory throughput, even though it doesn't have to literally scan the entire heap (only potential references have to be scanned). The Golang folks are running into this issue, it's reached a point where the impact of GC traffic is itself a meaningful bottleneck on performance, and GC-free languages have more potential for most workloads.

Re: I/O is no longer the bottleneck? (2022)

#95

Post by zipy124 »

coryrc wrote: post #85
ThreatSystems wrote: post #16

*Unless your in the cloud, then it's a metric to nickel and dime with throttling! On a more serious note, the performance of hardware today is mind boggling from what we all encountered way back when. What I struggle to comprehend though is how some software (particularly Windows as an OS, instant messaging applications etc.) feel less performant now than they ever were.

CRTs get data to the screen faster. Some LCDs have 500ms delays.

What non-ancient LCD's have response times that high. Even e-ink/e-paper displays are better than that!

Re: I/O is no longer the bottleneck? (2022)

#96

Post by vjerancrnjak »

nine_k wrote: post #23
lunixbochs wrote: post #17

Earlier quoted context omitted.

your single core numbers seem way too low for peak throughput on one core, unless you stipulate that all cores are active and contending with each other for bandwidth e.g. dual channel zen 1 showing 25GB/s on a single core https://stackoverflow.com/a/44948720 I wrote some microbenchmarks for single-threaded memcpy zen 2 (8-channel DDR4) naive c: 17GB/s non-temporal avx: 35GB/s Xeon-D 1541 (2-channel DDR4, my weakest…

As much as I can understand a Zen 5 CPU core can run two AVX512 operations per clock (1024 bits) + 4 integer operations per clock (which use up FPU circuitry in the process), so additional 256 bits. At 4 GHz, this is 640 GB/s. I suppose that in real life such ideal condition do not occur, but it shows how badly the CPU is limited by its memory bandwidth for streaming tasks. Its maximum memory-read bandwidth is 768 bi…

It is interesting that despite this we still have programming languages and libraries that cannot exploit pipelining to actually demonstrate IO is the bottleneck and not CPU

Re: I/O is no longer the bottleneck? (2022)

#97

Post by zozbot234 »

imtringued wrote: post #93
zozbot234 wrote: post #84

Earlier quoted context omitted.

> A lot of basic software design assumptions are still based on the constraints of the spinning rust era... fsync() is still slow, and you need that for real persistence. It's not just about spinning rust, there's very good reasons for wanting a different treatment of clearly ephemeral/scratchpad storage.

Isn't the problem with fsync() that you can't wait for specific IO operations? You're waiting for every single operation on the file descriptor. If anything happens between your write and the call to fsync(), you're going to get delayed by the additional operation that managed to get through.

Modern storage hardware has multiple IO queues with operations in flight at any given time, so I'm not sure to what extent this applies. It may be that only operations that are directly relevant to any given range of data need to be waited for.

Re: I/O is no longer the bottleneck? (2022)

#98

Post by auselen »

eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

How do you measure/calculate 6GB/s?

Re: I/O is no longer the bottleneck? (2022)

#99

Post by mgaunard »

eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

Quite easy to outperform a parsing library when you're not actually doing any parsing work and just memory-mapping pre-parsed data...

That being said storing trees as serializable flat buffers is definitely useful, if only because you can release them very cheaply.

Re: I/O is no longer the bottleneck? (2022)

#100

Post by 15155 »

Analemma_ wrote: post #43
tiffanyh wrote: post #36

Earlier quoted context omitted.

> On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. What makes M-series have 3x the bandwidth (per core), over x86?

The sibling comment has the correct, more detailed answer, but the high-level answer is that the M-series chips are SoCs with all the RAM on-die. That lets you push way more data than you can over a bus out to socketed memory. The tradeoff is that it's non-upgradeable, but (contra some people who claim this is only a cash-grab by Apple to prevent RAM upgrades) it's worth it for the bandwidth.

The memory is in-package, not on-die - on-die would mean that the DRAM is being manufactured on the same 1-2-3-4-whatever nanometer process - DDR is much larger.
Post reply on HN