61–70 of 143 posts
Wait, PCIe bandwidth is higher than memory bandwidth now? That's bonkers, when did that happen? I haven't been keeping up. Just looked at the i9-14900k and I guess it's true, but only if you add all the PCIe lanes together. I'm sure there are other chips where it's even more true. Crazy!
Hmm.
Somebody make me a PCIe card with RDIMM slots on it.
Wait, PCIe bandwidth is higher than memory bandwidth now? That's bonkers, when did that happen? I haven't been keeping up. Just looked at the i9-14900k and I guess it's true, but only if you add all the PCIe lanes together. I'm sure there are other chips where it's even more true. Crazy!
> Wait, PCIe bandwidth is higher than memory bandwidth now? Hmm. Somebody make me a PCIe card with RDIMM slots on it.
Earlier quoted context omitted.
MAP_HUGETLB can't be used for mmaping files on disk, it can only be used with MAP_ANONYMOUS, with a memfd, or with a file on a hugetlbfs pseudo-filesystem (which is also in memory).
> MAP_HUGETLB can't be used for mmaping files on disk False. I've successfully used it to memory-map networked files.
Based on this SO discussion [1], it is possibly a limitation with popular filesystems like ext4?
If anyone knows more about this, I'd love to know what exactly are the requirements for using hugepages this way.
[1] https://stackoverflow.com/questions/44060678/huge-pages-for-...
Earlier quoted context omitted.
The original blog post title is intentionally clickbaity. You know, to bait people into clicking. Also I do want to challenge people to really think here. Seeing if the cached file data can be accessed quickly is the point of the experiment. I can't get mmap() to open a file with huge pages. void* buffer = mmap(NULL, size_bytes, PROT_READ, (MAP_HUGETLB | MAP_HUGE_1GB), fd, 0); doesn't work. You can can see my code he…
MAP_HUGETLB can't be used for mmaping files on disk, it can only be used with MAP_ANONYMOUS, with a memfd, or with a file on a hugetlbfs pseudo-filesystem (which is also in memory).
This is wrong, because your mmap code is being stalled for page faults (including soft page faults that you have when the data is in memory, but not mapped to your process). The io_uring code looks like it is doing all the fetch work in the background (with 6 threads), then just handing the completed buffers to the counter. Do the same with 6 threads that would first read the first byte on each page and then hand tha…
That’s the point: “mmap” is slow because it is serial.
> A few notes for the "um actually" haters commenting on Hacker News Stay classy; any criticism is of course "hating", right? The fact that your title is clickbaity and your results suspect should encourage you to get the most accurate picture, not shoot the messenger.
This:
size_t count = 0;
/// ... code to actually count elided ...
printf("Found %ld 10s\n", count);
The (C17 draft) standard says "If any argument is not the correct type for the corresponding conversion specification, the behavior is undefined" so this is not intended as pointless language-lawyering, it's just that it can be important to get silly details like this right in C.
Maybe I'm misunderstanding, but after reading it sounds to me not like "io_uring is faster than mmap" but "raid0 with 8 SSDs has more throughput than 3 channel DRAM".
Earlier quoted context omitted.
Lol. Thanks.
Its not even about clickbait for me, but I really dont want to go parse an article to figure out what is meant by "Memory is slow, Disk is fast". You want "clickbait" to make people click and think, we want descriptive tittles to know what the article is about before we read it. That used to be original purpose of tittles, we like it that way. Its like as if youd label your food product "you wont believe this", and f…
Indeed[0].
[0] https://en.wikipedia.org/wiki/I_Can't_Believe_It's_Not_Butte...!