Live data from Hacker News

System call instrumentation on Linux/x86‑64 using memory‑indirect calls, part I

humprog.org

31–34 of 34 posts

Re: System call instrumentation on Linux/x86‑64 using memory‑indirect calls, part I

#31
post #28

Earlier quoted context omitted.

Yeah. On Linux it's just an optimization. What user space really wanted was a way to memory map some kernel data into the process address space in order to avoid switching to kernel mode while accessing it. Instead Linux memory mapped an entire ELF whose only purpose is to wrap the data. Newer system calls like io_uring are doing it right.

Strongly disagree that providing the vDSO in ELF file format is somehow harmful or inefficient. You'll need a compatibility mechanism in any case since the exposed features will change over time, and doing that through normal symbol resolution avoids a whole bunch of extra effort. And after ld.so is done with relations on executable startup, it makes no difference in performance either. Look at the Linux architecture…

> harmful or inefficient

It's not harmful, it just requires a lot of machinery. The normal system call entry point is perfectly simple, minimal, sufficient and language agnostic.

As for inefficient, program startup speed matters. It's probably not contributing much to the overall profile but it's still parsing binary formats and looking up strings in hash tables. This is definitely stuff that could be eliminated entirely. Code could conceivably mmap a gettimeofday buffer later on when it actually needs it instead of doing it on the startup procedure of every single process. Only reason it's not done this way is history.

> You'll need a compatibility mechanism in any case since the exposed features will change over time

Not really. Existing system calls stay as is. Linux won't break them, they remain supported. New users will use the latest version of the system call.

> And after ld.so is done with relations on executable startup

Assumes that there is an ld.so. It's a completely optional component. The system should be usable without it.

> Look at the Linux architectures that have a vDSO in non-ELF format. It's seriously ugly.

We agree on this. I have no objection to the ELF format. If a vDSO must be provided, then ELF is a good a format for it.

My point is that providing the vDSO should not actually be necessary, and that no system call should be forced to go through it.

> I don't think the comparison with io_uring is valid either, very different kind of API

Why? Both involve kernel/userspace shared memory. The difference is the vDSO hides the memory behind a function call while io_uring gives you the parameters you can pass to mmap to control it yourself, no ELF needed.

Re: System call instrumentation on Linux/x86‑64 using memory‑indirect calls, part I

#32

Earlier quoted context omitted.

> "Freestanding", as in "standing on top of an OS but nothing else"? Freestanding as in freestanding C. > Then using the OS-provided shared object that is the documented interface between the userspace and the kernel doesn't violate your free stand. Correct. I'm just saying it shouldn't be required. > I mean, I too had written small interpreters that had only LoadLibraryW/GetProcAddress from kernel32.dll as their imp…

> I'm just saying it shouldn't be required. I'm still not entirely sure why you want to use instructions from the privilleged subset of your ISA instead of plain old "call fun_addr". > And where are LoadLibraryW and GetProcAddress coming from? They're provided by the OS. Their addresses are patched into your executable's image during the loading. It's a very ancient technology, one of the very first software technolo…

> instructions from the privilleged subset of your ISA

Absolutely nothing "privileged" about the syscall instruction. Every Linux process can run that instruction without any problems whatsoever.

> They're provided by the OS.

Not on Linux.

> You don't have to parse it.

You do on Linux. Otherwise you cannot get to the vDSO's table of symbols to function pointers.

System calls are themselves a table of function pointers, indexed by the system call number rather than by symbol. The difference is the kernel resolves it so you don't have to. That's what makes it simple and easy to use.

Re: System call instrumentation on Linux/x86‑64 using memory‑indirect calls, part I

#33
post #28

Earlier quoted context omitted.

Strongly disagree that providing the vDSO in ELF file format is somehow harmful or inefficient. You'll need a compatibility mechanism in any case since the exposed features will change over time, and doing that through normal symbol resolution avoids a whole bunch of extra effort. And after ld.so is done with relations on executable startup, it makes no difference in performance either. Look at the Linux architecture…

> harmful or inefficient It's not harmful, it just requires a lot of machinery. The normal system call entry point is perfectly simple, minimal, sufficient and language agnostic. As for inefficient, program startup speed matters. It's probably not contributing much to the overall profile but it's still parsing binary formats and looking up strings in hash tables. This is definitely stuff that could be eliminated enti…

> Why? Both involve kernel/userspace shared memory.

The vDSO linking happens once, at startup. io_uring is a continuous service.

With this fundamental of a difference in perspective between us I don't think it's useful to sink further time into this discussion thread.

Re: System call instrumentation on Linux/x86‑64 using memory‑indirect calls, part I

#34
post #33

Earlier quoted context omitted.

> harmful or inefficient It's not harmful, it just requires a lot of machinery. The normal system call entry point is perfectly simple, minimal, sufficient and language agnostic. As for inefficient, program startup speed matters. It's probably not contributing much to the overall profile but it's still parsing binary formats and looking up strings in hash tables. This is definitely stuff that could be eliminated enti…

> Why? Both involve kernel/userspace shared memory. The vDSO linking happens once, at startup. io_uring is a continuous service. With this fundamental of a difference in perspective between us I don't think it's useful to sink further time into this discussion thread.

> The vDSO linking happens once, at startup.

io_uring buffers are also created once, and only if necessary.

> io_uring is a continuous service.

As is the vDSO. It will be used continuously if linked. Possibly many times per second.

> With this fundamental of a difference in perspective between us

I don't think we actually disagree that much. We are probably misunderstanding each other. I agreed with you on ELF.

> I don't think it's useful to sink further time into this discussion thread.

Could be useful to people who are just reading the comments though.

Post reply on HN