Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I/O is no longer the bottleneck? (2022)

stoppels.ch

Post reply on HN
Print view View original

31–40 of 133 posts

Re: I/O is no longer the bottleneck? (2022)

#31

Post by foota »

eliasdejong wrote: post #24
johncolanduoni wrote: post #9

Earlier quoted context omitted.

What is the nature of the architectural limit here? The bus between an individual core and the caches and/or memory controller?

The limit is the number of outstanding cache line requests to the memory controller. CPUs have a fixed number of slots for this, around 10-12 usually. Intel calls them LFBs (Line Fill Buffers) and AMD MSHRs (Miss Status Holding Registers). When the slots are filled, the CPU can issue no more requests and has to wait for them to complete. Apple M chips (probably) have more slots and the memory is physically packaged t…

I assume these must be really expensive? Otherwise it seems like a great way to improve throughput on low concurrency tasks.

Re: I/O is no longer the bottleneck? (2022)

#32

Post by eliasdejong »

hamandcheese wrote: post #28
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

Lite claims that it can be modified in-place, but I'm curious how that works with variable-length structures like strings?

If the new value is equal size or smaller, it will overwrite the old value in-place. If it is larger, then the new value is appended to the buffer and the index structure is updated to point to the new location.

In the case of append, the old value still lives inside the buffer but is zeroed out. This means that if you keep replacing variable-sized elements, over time the buffer will fragment. You can vacuum a message by recursively writing it from the root to a new buffer. This will clear out the unused space. This operation can be delayed for as long as you like.

If you are only setting fixed-size values like integers or floats, then the buffer never grows as they are always updated in-place.

Re: I/O is no longer the bottleneck? (2022)

#33

Post by rattray »

eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

Does capn proto have similar properties?

Re: I/O is no longer the bottleneck? (2022)

#34

Post by tignaj »

Nathanba wrote: post #13
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

cool, do you think it's possible to add a schema mode to lite3 to remove the message size tradeoff? I think most people will still want to use lite3 with hard schemas during both serialization and deserialization. It's nice that it also works in a schemaless mode though.

If you are looking for a fast format with schema support see STEF: https://www.stefdata.net/

Disclosure: I am the author.

Re: I/O is no longer the bottleneck? (2022)

#35

Post by fragmede »

nine_k wrote: post #26
ThreatSystems wrote: post #16

*Unless your in the cloud, then it's a metric to nickel and dime with throttling! On a more serious note, the performance of hardware today is mind boggling from what we all encountered way back when. What I struggle to comprehend though is how some software (particularly Windows as an OS, instant messaging applications etc.) feel less performant now than they ever were.

The answer, I suspect, is is the same as always: waiting for I/O in the GUI thread. Both Telegram and FB messenger are snappy; I didn't use anything else seriously as of late. (Especially not Teams, nor the late Skype.)

They could be way faster. They're snappy enough but still, so slow.

Re: I/O is no longer the bottleneck? (2022)

#36

Post by tiffanyh »

eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

> On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips.

What makes M-series have 3x the bandwidth (per core), over x86?

Re: I/O is no longer the bottleneck? (2022)

#38

Post by eliasdejong »

rattray wrote: post #33
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

Does capn proto have similar properties?

Yes, any zero-copy format in general will have this advantage because reading a value is essentially just a pointer dereference. Most of the message data can be completely ignored, so the CPU never needs to see it. Only the actual data accessed counts towards the limit.

Btw: in my project README I have benchmarks against Cap'N Proto & Google Flatbuffers.

Re: I/O is no longer the bottleneck? (2022)

#39

Post by tanelpoder »

kvemkon wrote: post #29
nine_k wrote: post #23

Earlier quoted context omitted.

As much as I can understand a Zen 5 CPU core can run two AVX512 operations per clock (1024 bits) + 4 integer operations per clock (which use up FPU circuitry in the process), so additional 256 bits. At 4 GHz, this is 640 GB/s. I suppose that in real life such ideal condition do not occur, but it shows how badly the CPU is limited by its memory bandwidth for streaming tasks. Its maximum memory-read bandwidth is 768 bi…

> CPU is limited by its memory bandwidth for streaming tasks That must be the reason, why EPYC 9175F exists. It is only 16-core CPU, but all 16 8-core CCDs are populated and only one core on each is active. The next gen EPYC is rumored to have 16 instead of 12 memory channels (which were 8 only 4-5 years ago).

This also leaves more power & thermal allowance for the IO Hub on the CPU chip and I guess the CPU is cheaper too.

If your workload is mostly about DMAing large chunks of data around between devices and you still want to examine the chunk/packet headers (but not touch all payload) on the CPU, this could be a good choice. You should have the full PCIe/DRAM bandwidth if all CCDs are active.

Edit: Worth noting that a DMA between PCIe and RAM still goes through the IO Hub (Uncore on Intel) inside the CPU.

Re: I/O is no longer the bottleneck? (2022)

#40

Post by wmf »

dehrmann wrote: post #8
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

> 6 GB/s Samsung is selling NVMe SSDs claiming 14 GB/s sequential read speed.

Any code that's reading/writing to SSD needs to use multiple cores. The SSD is faster than a single CPU core.
Post reply on HN