Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I/O is no longer the bottleneck? (2022)

stoppels.ch

Post reply on HN
Print view View original

41–50 of 133 posts

Re: I/O is no longer the bottleneck? (2022)

#41

Post by josephg »

nine_k wrote: post #26
ThreatSystems wrote: post #16

*Unless your in the cloud, then it's a metric to nickel and dime with throttling! On a more serious note, the performance of hardware today is mind boggling from what we all encountered way back when. What I struggle to comprehend though is how some software (particularly Windows as an OS, instant messaging applications etc.) feel less performant now than they ever were.

The answer, I suspect, is is the same as always: waiting for I/O in the GUI thread. Both Telegram and FB messenger are snappy; I didn't use anything else seriously as of late. (Especially not Teams, nor the late Skype.)

> waiting for I/O in the GUI thread

The problem is sloppy programming. We knew how to make small, fast, programs 20+ years ago that would just scream on modern hardware. But now everything is bloated and slow. CPUs can retire billions of instructions per second. Discord takes 10+ seconds to open. I’m simply not creative enough to think up how to keep the cpu busy that long opening IRC.

Re: I/O is no longer the bottleneck? (2022)

#42

Post by MindSpunk »

tiffanyh wrote: post #36
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

> On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. What makes M-series have 3x the bandwidth (per core), over x86?

M-series have a substantially wider memory bus allowing much higher throughput. It's not really an x86/M-series thing, rather it's a packaging limitation. Apple integrates the memory into the same package as the CPU, the vast majority of x86 CPUs are socketed with socketed memory.

Apple are able to push a wider bus at higher frequencies because they aren't limited by signal integrity problems you encounter trying to do the same over sockets and motherboard traces. x86 CPUs like the Ryzen AI Max 395+, when packaged without socketed memory, are able to push equally wide busses at high frequencies.

Re: I/O is no longer the bottleneck? (2022)

#43

Post by Analemma_ »

tiffanyh wrote: post #36
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

> On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. What makes M-series have 3x the bandwidth (per core), over x86?

The sibling comment has the correct, more detailed answer, but the high-level answer is that the M-series chips are SoCs with all the RAM on-die. That lets you push way more data than you can over a bus out to socketed memory.

The tradeoff is that it's non-upgradeable, but (contra some people who claim this is only a cash-grab by Apple to prevent RAM upgrades) it's worth it for the bandwidth.

Re: I/O is no longer the bottleneck? (2022)

#44

Post by eliasdejong »

lunixbochs wrote: post #17
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

your single core numbers seem way too low for peak throughput on one core, unless you stipulate that all cores are active and contending with each other for bandwidth e.g. dual channel zen 1 showing 25GB/s on a single core https://stackoverflow.com/a/44948720 I wrote some microbenchmarks for single-threaded memcpy zen 2 (8-channel DDR4) naive c: 17GB/s non-temporal avx: 35GB/s Xeon-D 1541 (2-channel DDR4, my weakest…

> A key feature of this code is that it skips CPU cache when copying

Are those numbers also measured while skipping the CPU cache?

Re: I/O is no longer the bottleneck? (2022)

#45

Post by yusyusyus »

eliasdejong wrote: post #15
dehrmann wrote: post #8

Earlier quoted context omitted.

> 6 GB/s Samsung is selling NVMe SSDs claiming 14 GB/s sequential read speed.

> 14 GB/s Yes, those numbers are real but only in very short bursts of strictly sequential reads, sustained speeds will be closer to 8-10 GB/s. And real workloads will be lower than that, because they contain random access. Most NVMe drivers on Linux actually DMA the pages directly into host memory over the PCIe link, so it is not actually the CPU that is moving the data. Whenever the CPU is involved in any data move…

DMAing as opposed to what?

Re: I/O is no longer the bottleneck? (2022)

#46

Post by lunixbochs »

eliasdejong wrote: post #44
lunixbochs wrote: post #17

Earlier quoted context omitted.

your single core numbers seem way too low for peak throughput on one core, unless you stipulate that all cores are active and contending with each other for bandwidth e.g. dual channel zen 1 showing 25GB/s on a single core https://stackoverflow.com/a/44948720 I wrote some microbenchmarks for single-threaded memcpy zen 2 (8-channel DDR4) naive c: 17GB/s non-temporal avx: 35GB/s Xeon-D 1541 (2-channel DDR4, my weakest…

> A key feature of this code is that it skips CPU cache when copying Are those numbers also measured while skipping the CPU cache?

naive c is just a memcpy. non-temporal uses the streaming instructions.

Re: I/O is no longer the bottleneck? (2022)

#47

Post by Scaevolus »

Analemma_ wrote: post #43
tiffanyh wrote: post #36

Earlier quoted context omitted.

> On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. What makes M-series have 3x the bandwidth (per core), over x86?

The sibling comment has the correct, more detailed answer, but the high-level answer is that the M-series chips are SoCs with all the RAM on-die. That lets you push way more data than you can over a bus out to socketed memory. The tradeoff is that it's non-upgradeable, but (contra some people who claim this is only a cash-grab by Apple to prevent RAM upgrades) it's worth it for the bandwidth.

Apple had soldered DDR ram for a long time that was no faster than any other laptop. It's only with the Apple Silicon M1 that it started being notably higher bandwidth.

Re: I/O is no longer the bottleneck? (2022)

#48

Post by eru »

Analemma_ wrote: post #43
tiffanyh wrote: post #36

Earlier quoted context omitted.

> On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. What makes M-series have 3x the bandwidth (per core), over x86?

The sibling comment has the correct, more detailed answer, but the high-level answer is that the M-series chips are SoCs with all the RAM on-die. That lets you push way more data than you can over a bus out to socketed memory. The tradeoff is that it's non-upgradeable, but (contra some people who claim this is only a cash-grab by Apple to prevent RAM upgrades) it's worth it for the bandwidth.

> The tradeoff is that it's non-upgradeable, but (contra some people who claim this is only a cash-grab by Apple to prevent RAM upgrades) it's worth it for the bandwidth.

That, and if you come at it from the phone / tablet or even laptop angle: most people are quite ok just buying their computing devices pre-assembled and not worrying about upgrading them. You just buy a new one when the old one fails or you want an upgrade.

Similar to how cars these days are harder to repair for the layman, but they also need much less maintenance. The guy who was always tinkering away with his car in old American sitcoms wasn't just a trope, he was truth-in-television. Approximately no one has to do that anymore with modern cars.

Re: I/O is no longer the bottleneck? (2022)

#49

Post by dpc_01234 »

It's not about memory/CPU/IO, but latency vs throughput. Most software is slow because it ignores the latency. If you program serially waiting for _whatever_ it is going to be slow. If you scatter your data around memory, or read from disk in small chunks, or make tons of tiny queries to the DB serially your software will be 99.9% waiting idle for something to finish. That's it. If you can organize your data linearly in memory and/or work on batches of it at the time and/or parallelize stuff and/or batch your IO, it is going to be fast.

Re: I/O is no longer the bottleneck? (2022)

#50

Post by eru »

eliasdejong wrote: post #32
hamandcheese wrote: post #28

Earlier quoted context omitted.

Lite claims that it can be modified in-place, but I'm curious how that works with variable-length structures like strings?

If the new value is equal size or smaller, it will overwrite the old value in-place. If it is larger, then the new value is appended to the buffer and the index structure is updated to point to the new location. In the case of append, the old value still lives inside the buffer but is zeroed out. This means that if you keep replacing variable-sized elements, over time the buffer will fragment. You can vacuum a messag…

Interesting. Sounds like you are getting copy-on-write-with-sharing for growing sizes and in-place updates when your data shrinks?
Post reply on HN