Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I/O is no longer the bottleneck? (2022)

stoppels.ch

Post reply on HN
Print view View original

111–120 of 133 posts

Re: I/O is no longer the bottleneck? (2022)

#111

Post by cake-rusk »

StilesCrisis wrote: post #106
eliasdejong wrote: post #102

Earlier quoted context omitted.

Imagine if you measured the speed of beer delivery by the rate at which beer cans can be packed/unpacked from truck pallets. But then somebody shows up with a tanker truck and starts pumping beer directly in and out. You might argue this is 'unfair' because the tanker is not doing any packing or unpacking. But then you realize it was never about packing speed in the first place. It was about delivering beer.

This is actually a good analogy; the beer cans are self-contained and ready for the customer with zero extra work. The beer delivered by the tanker still needs to be poured into individual glasses by the bartender, which is slow and tedious.

He probably meant beer kegs. Memory map-able data is closer to sending beer cans since both are in a ready to use format.

Re: I/O is no longer the bottleneck? (2022)

#112

Post by wtallis »

imtringued wrote: post #90
yusyusyus wrote: post #45

Earlier quoted context omitted.

DMAing as opposed to what?

Letting the CPU burn cycles by accessing memory mapped device memory instead of using DMA like in the good old days? What even is this question?

> accessing memory mapped device memory instead of using DMA like in the good old days

That's not actually an option for NVMe devices, is it? The actual NAND storage isn't available for direct memory mapping, and the only way to access it is to set up DMA transfers.

Re: I/O is no longer the bottleneck? (2022)

#113

Post by quadrature »

eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

For what it’s worth simdjson now has an on demand api that lets you skip over keys that you don’t need.

Re: I/O is no longer the bottleneck? (2022)

#114

Post by perching_aix »

brunoborges wrote: post #105
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

> This limit also applies to (de)serializing data like JSON and Protobuf, because those formats must typically be fully parsed before a single field can be read. Which file formats allow partial parsing?

Anything that is both streamable and seekable?

Re: I/O is no longer the bottleneck? (2022)

#115

Post by mort96 »

quadrature wrote: post #113
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

For what it’s worth simdjson now has an on demand api that lets you skip over keys that you don’t need.

Which is great, but a JSON parser fundamentally can't avoid looking at every byte. You can't jump to the next key, you have to parse your way to the next key.

Re: I/O is no longer the bottleneck? (2022)

#116

Post by mort96 »

brunoborges wrote: post #105
eliasdejong wrote: post #7

Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect…

> This limit also applies to (de)serializing data like JSON and Protobuf, because those formats must typically be fully parsed before a single field can be read. Which file formats allow partial parsing?

You just need to encode the size of values in bytes to make it possible to partially parse the format.

Imagine the following object:

    {
        "attributes": [ .. some really really long array of whatever .. ],
        "name": "Bob"
    }
In JSON, if you want to extract the "name" property, you need to parse the whole "attributes" array. However, if you encoded the size of the "attributes" array in bytes, a parser could look at the key "attributes", decide that it's not interested, and jump past it without parsing anything.

You'd typically want some kind of binary format, but for illustration purposes, here's an imaginary XML representation of the JSON data model which achieves this:

    
      
        .. some 8 MiB large array of values ..
      
      Bob
If this was stored in a file, you could use fseek to seek past the "attributes" array. If it was compressed, or coming across a socket, you'd need more complicated mechanisms to seek past irrelevant parts of the object.

Re: I/O is no longer the bottleneck? (2022)

#117

Post by coryrc »

zipy124 wrote: post #95
coryrc wrote: post #85

Earlier quoted context omitted.

CRTs get data to the screen faster. Some LCDs have 500ms delays.

What non-ancient LCD's have response times that high. Even e-ink/e-paper displays are better than that!

TVs can do a bunch of filtering which adds long latency based on the setting about the type of content (sorry, can't remember the exact term ATM).

Re: I/O is no longer the bottleneck? (2022)

#118

Post by eliasdejong »

digdugdirk wrote: post #59
eliasdejong wrote: post #18

Earlier quoted context omitted.

Being schemaless is deliberate design decision as it eliminates the need for managing and building schema files. By not requiring schema, messages are always readable to arbitrary consumers. If you want schema, it must be done by the application through runtime type checking. All messages contain type information. Though I do see the value of adding pydantic-like schema checking in the future. EDIT: Regarding message…

Pydantic was the first thing I thought of when I saw this. The possibilities are very intriguing. Do you have any thoughts/recommendations for someone if they were to try making a pydantic interface layer for lite3?

The library contains iterator functions that can be used to recursively traverse a message from the root and discover the entire structure. Then every value can be checked for defined constraints appropriately while at the same time validating the traversal paths. Also a schema description format is needed, perhaps something like JSONSchema can be used.

Doing this by definition requires processing an entire message which can be a little slow, so it is best done in C and exposed through CPython bindings / CFFI.

Re: I/O is no longer the bottleneck? (2022)

#119

Post by pjdesno »

Since no one else seems to have pointed this out - the OP seems to have misunderstood the output of the 'time' command.

  $ time ./wc-avx2
"System" time is the amount of CPU time spent in the kernel on behalf of your process, or at least a fairly good guess at that. (e.g. it can be hard to account for time spent in interrupt handlers) With an old hard drive you would probably still see about 117ms of system time for ext4, disk interrupts, etc. but real time would have been much longer.

    $ time ./optimized  /dev/null

    real    0m1.525s
    user    0m1.477s
    sys     0m0.048s
Here we're bottlenecked on CPU time - 1.477s + 0.048s = 1.525s. The CPU is busy for every millisecond of real time, either in user space or in the kernel.

In the optimized case:

  real    0m0.395s
  user    0m0.196s
  sys     0m0.117s
0.196 + 0.117 = 0.313, so we used 313ms of CPU time but the entire command took 395ms, with the CPU idle for 82ms.

In other words: yes, the author managed to beat the speed of the disk subsystem. With two caveats:

1. not by much - similar attention to tweaking of I/O parameters might improve I/O performance quite a bit.

2. the I/O path is CPU-bound. Those 117ms (38% of all CPU cycles) are all spent in the disk I/O and file system kernel code; if both the disk and your user code were infinitely fast, the command would still take 117ms. (but those I/O tweaks might reduce that number)

Note that the slow code numbers are with a warm cache, showing 48ms of system time - in this case only the ext4 code has to run in the kernel, as data is already cached in memory. In the cold cache case it has to run the disk driver code, as well, for a total of 117ms.

Post reply on HN