31–40 of 143 posts
Shouldn't this be "io_uring is faster than mmap"? I guess that would not get much engagement though! That said, cool write up and experiment.
If anyone can suggest a better title (i.e. more accurate and neutral) we can change it again.
Earlier quoted context omitted.
MAP_HUGETLB can't be used for mmaping files on disk, it can only be used with MAP_ANONYMOUS, with a memfd, or with a file on a hugetlbfs pseudo-filesystem (which is also in memory).
> MAP_HUGETLB can't be used for mmaping files on disk False. I've successfully used it to memory-map networked files.
Earlier quoted context omitted.
> MAP_HUGETLB can't be used for mmaping files on disk False. I've successfully used it to memory-map networked files.
Share your code?
Do you have kernel documentation that says that hugetlb doesn't work for files? I don't see that stated anywhere.
Earlier quoted context omitted.
MAP_HUGETLB can't be used for mmaping files on disk, it can only be used with MAP_ANONYMOUS, with a memfd, or with a file on a hugetlbfs pseudo-filesystem (which is also in memory).
> MAP_HUGETLB can't be used for mmaping files on disk False. I've successfully used it to memory-map networked files.
Earlier quoted context omitted.
> MAP_HUGETLB can't be used for mmaping files on disk False. I've successfully used it to memory-map networked files.
That doesn’t sound like the intended meaning of “on disk”.
The "on disk" distinction is a simplification.
Because PCIe bandwidth is higher than memory bandwidth
Some newer HBM enabled, like a Xeon Max 9480 can go over 1.6TBs for HBM (up to 64GB) and DDR5 can reach > 300 GB/s.
Even saturating all PCIe lanes (196 on a dual socket Xeon 6), you could at most theoretically get ~784GB/s, which coincidentally is the max memory bandwidth of such CPUs (12 Channels x 8,800 MT/s = 105,600 MT/s total bandwidth or roughly ~784GB/s).
I mean, solid state IO is getting really close, but it's not so fast on non-sequential access patterns.
I agree that many workloads could be shifted to SSDs but it's still quite nuanced.
Because PCIe bandwidth is higher than memory bandwidth This doesn't sound right, a PCIe 5.0 x16 slot offers up to 64 GB/s. That's fully saturated, a fairly old Xeon server can sustain >100 GB/s memory reads per numa node without much trouble. Some newer HBM enabled, like a Xeon Max 9480 can go over 1.6TBs for HBM (up to 64GB) and DDR5 can reach > 300 GB/s. Even saturating all PCIe lanes (196 on a dual socket Xeon 6),…
Earlier quoted context omitted.
I just saw this post so am starting with Part 1. Could you replace the charts with ones on some sort of log scale? It makes it look like nothing happened til 2010, but I'd wager its just an optical illusion... And, even better, put all the lines on the same chart, or at least with the same y axis scale (perhaps make them all relative to their base on the left), so that we can the relative rate of growth?
I tried with the log scale before. They failed to express the exponential hockey stick growth unless you really spend the time with the charts and know what log scale is. I'll work on incorporating log scale due to popular demand. They do show the progress has been nice and exponential over time. When I put the lines on the same chart it made the y axis impossible to understand. The units are so different. Maybe I'll…
Log axis solves this, and turns meaningless hockey sticks into generally a straightish line that you can actually parse. If it still deviates from straight, then you really know there's true changes in the trendline.
Lines on same chart can all be divided by their initial value, anchoring them all at 1. Sometimes they're still a mess, but it's always worth a try.
You're enormously knowledgeable and the posts were fascinating. But this is stats 101. Not doing this sort of thing, especially explicitly in favour of showing a hockey stick, undermines the fantastic analysis.
Just looked at the i9-14900k and I guess it's true, but only if you add all the PCIe lanes together. I'm sure there are other chips where it's even more true. Crazy!