181–190 of 216 posts
Anyone running ripgrep on a an HPC cluster against a large cluster filesystem needs to stop and redesign their workflow. This generates high amounts of small I/O which is the Achilles heel of any large cluster filesystem. You are exporting your workload onto the metadata mechanisms of the filesystem rather keeping it within the higher bandwidth capable memory subsystem on your cluster. It doesn't take but a couple us…
Earlier quoted context omitted.
This is technical writing for kernel developers who already know how memory management works. If you want to understand it better, you should study up on the x86 MMU and the kernel memory management subsystem. I'm not a kernel expert but I found it pretty understandable, so I'll try to explain it. MMU: The memory management unit, part of the CPU designed to allow an OS kernel to flexibly control how memory addresses…
I understood the jargon just fine. The writing is still terrible and way too verbose.
Earlier quoted context omitted.
A good example is go. On linux you can use a from scratch image fairly easily because it only uses syscalls. But for windows or mac the moving target wasn't maintainable so they link against shared objects. Linus enforcing the don't break userspace rule is what made that possible. That definitely has tradeoffs. At some point relibc or something similar will allow the same (stably) for rust. But using posix as that co…
The Linux kernel defines syscalls as its stable ABI (note: there are also non-kernel ABIs on Linux, such as Wayland) while Windows defines the DLL calls as its stable ABI. One isn't better than the other. Linux's approach allows binaries to be fully statically linked, which is a more predictable environment for binaries, but Windows's approach composes better, as every process loads DLLs and this allows for things li…
However given how fundamental GPU acceleration is (as well as various other pieces of dedicated hardware in various scenarios) it would be nice if the kernel defined some basic portable semantics for linking with important drivers. These could exist independently of libc and the rest of userspace purely as an optional fallback.
... or we could all just include a glibc compatible dlopen routine in our statically linked binaries instead of worrying about pedantic hypotheticals.
Earlier quoted context omitted.
> Alternatively, it could be a hardware bug, since afaict it's only been repro'd on one hardware config. (I know there are a bunch of ARM cores with erratas around TLB invalidation) Which raises the question of whether any HN readers have successfully reproduced this? I've just tried (using his file generation script and the official rg binary he links) on a Ryzen 7 5800X / 7.1.5-arch1-2 with sufficient free ram as t…
I have never, in my entire personal history of kernel development, successfully reproduced a paging structure cache bug. I have stared at these sorts of bugs and puzzled them out. With this particular issue, if my theory is right, it will depend on all manner of microarchitectural issues, possibly address spare randomization, context switch and migration timing, random speculative accesses that pull things into cache…
When Bruce Dawson investigated a bug in the Xbox 360 cpu, the offending instructions didn't even need to "execute" in order to crash the system.
[0] https://randomascii.wordpress.com/2018/01/07/finding-a-cpu-d...
Earlier quoted context omitted.
The way I read it was that it wrote to a page and then did a read on that page within 10 instructions and that was not enough time for the memory system to have gone through the process of creating the page backed with real memory and that the timing bug widened in Linux 7.0 such that more things like this exposed the bug. Or there was flapping in the TLB for some reason, or the locking wasn’t correct, or whatever el…
But the write should have triggered a page fault immediately in that case, and should have been blocked on the page being actually allocated before resuming.
I get why people don't bother replacing the default allocator from musl all the time (it's there, convenient). But in an application whose purpose is to be FAST, I find it weird they haven't bothered replacing it with another more performant one. mallocng is bad at dealing with contention during multithreading. I've had applications that usually were I/O bound suddenly become "malloc" bound when building with musl in…
We use musl+mimalloc by default for our entire production operating systems: https://stagex.tools
The analysis of the kernel bug may be a better thing to link to: https://github.com/dfoxfranke/ripgrep-3494-analysis .
I tried reading and gave up at the "Headline"... Quoting from the bug analysis: >Headline. The crash is real and reproducible. With musl instrumentation we pin the in-process mechanism precisely: a thread's own store to a freshly- faulted anonymous page becomes invisible to that same thread's reload ~10 instructions later, because the page's backing is replaced mid-function. A pagemap read at the instant of the fault…
Actual technical writers know what to emphasize and what not to, based on the expected audience; good ones will gently add hints to expand that audience without compromising technical accuracy or content.
I get why people don't bother replacing the default allocator from musl all the time (it's there, convenient). But in an application whose purpose is to be FAST, I find it weird they haven't bothered replacing it with another more performant one. mallocng is bad at dealing with contention during multithreading. I've had applications that usually were I/O bound suddenly become "malloc" bound when building with musl in…
Heh, from the kernel patch: https://lore.kernel.org/all/CALCETrXbj__SFQMzPZhES5y6-sh4np-... > I saw a fun bug report in ripgrep and a studious but pretty bad AI-generated analysis Referring to https://github.com/dfoxfranke/ripgrep-3494-analysis which I indeed thought "that's an awful lot written to have been written by a human." Looks like that thread is from...today!
Until ~2000, "cellphone user in public" equated with "smug asshole." We're at that icky rejection stage with AI. Two years ago, that writeup would have been viewed as a generous gift of time to the community. Now we can't be bothered to read through it because we know where it came from, and that it's just worth $0.06 in tokens. Another reason (I think) is because we know what it portends. In a few years, digging man…
Earlier quoted context omitted.
Until ~2000, "cellphone user in public" equated with "smug asshole." We're at that icky rejection stage with AI. Two years ago, that writeup would have been viewed as a generous gift of time to the community. Now we can't be bothered to read through it because we know where it came from, and that it's just worth $0.06 in tokens. Another reason (I think) is because we know what it portends. In a few years, digging man…
If we know ‘where it came from’ in the sense that the author couldn’t even bother writing anything themselves, why should other people bother reading this deluge of nonsensical strung together words?
One exception: if the model is poor or on 'low' and that leads to a crappy report. That's the equivalent of shipping performance-critical code built with -O0 in the compiler analogy.