Live data from Hacker News

The Journey Before main()

amit.prasad.me

91–100 of 145 posts

Re: The Journey Before main()

#91

Earlier quoted context omitted.

> But adult projects should support Windows imho and consider Windows support from the start. Hope whatever "adult" is working on the project this is getting paid handsomely. They'd certainly need to pay me big bucks to care about Windows support. In any case, Linux system call ABI is becoming a lingua franca of systems programming. BSDs have implemented Linux system calls. Windows has straight up included Linux in t…

Try playing audio or displaying image on the screen using only documented syscalls. And make it work on all platforms you mentioned.

Displaying an image on the screen is not that difficult a task. Linux has framebuffer device files. You open them, issue an ioctl to get metadata like screen geometry and color depth, then mmap the framebuffer as an array of pixels you can CPU render to. It's eerily similar to the way terminal applications work.

It's also possible to use Linux KMS/DRM without any user space libraries.

https://github.com/laxyyza/drmlist/

The problem with hardware accelerated rendering is much of the associated functionality is actually implemented in user space and therefore not part of the kernel. They unfortunately force the libc on us. One would have to reimplement things like Mesa in order to do this. Not impossible, just incredibly time consuming.

Things could have been organized in a way that makes this feasible. Example: SQLite. You can plug in your own memory allocation functions and VFS layer. I've been slowly porting the SQLite Unix VFS to freestanding Linux in order to use it in my freestanding applications.

Re: The Journey Before main()

#92
post #68

Earlier quoted context omitted.

.interp to a glibc/libc you ship or static linking. These days it’s probably faster (in dev time) to just run a container than setting up a bespoke interp and a parallel set of libraries (and the associated toolchain changes or binary patching needed to support it).

Running a container is exactly my current solution as well. Are there any other solutions that don't depend on glibc?

Guix (and I assume Nix as well, but I only know Guix) can create a package that is completely self contained including glibc. You can even have it be an AppImage https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix...

Re: The Journey Before main()

#93

Earlier quoted context omitted.

Almost freestanding. It still requires you to link against kernel32 and use the functions it provides. This is because issuing system calls directly to the Windows kernel is not supported. The kernel developers reserve the right to change things like system call numbers, so they can't be hardcoded into the application.

> Almost freestanding. It still requires you to link against kernel32 Nitpick: the phrase “link against kernel32” feels like a Linux-ism. If you’re only calling a few function you need to load kernel32.dll and call some functions in it. But that’s a slightly different operation than linking against it. At least how I’ve always used the term link. You’re not wrong in principle. But Linux and Windows do a lot of things…

Loading means creating a memory image of the library. Linking means resolving the symbols to addresses within that memory image.

Loading a library and calling some functions from it is linking. The function pointer you receive is your link to the library function.

Re: The Journey Before main()

#94

Earlier quoted context omitted.

Part of it is the Glibc loader’s carnal knowledge of Glibc proper; there’s essentially no module boundary there. (That’s not completely unjustified, but Glibc is especially hostile there, like in its many other architectural choices.) Musl outright merges the two into a single binary. So if you want to do a loader then you’re also doing a libc. Part of it for desktop Linux specifically is that a lot of the graphics s…

Why do you need "vendor-specific userspace drivers"? I thought graphic acceleration uses OpenGL/Vulkan, and non-accelerated graphics uses DRM? And there are no "drivers" for Wayland compositors?

OpenGL and Vulkan are implemented as libraries in user space as the Mesa project.

Re: The Journey Before main()

#95
post #25

Its been a while since I've touched this stuff but my recollection is the ELF interpreter (ldso, not the kernel) is responsible for everything after mapping the initial ELF's segments. iirc execve maps pt_load segments from the program header, populates the aux vector on the stack, and jump straight to the ELF interpreter's entry point. Any linked objects are loaded in userspace by the elf interpreter. The kernel has…

That's right!

https://lwn.net/Articles/631631/

https://github.com/torvalds/linux/blob/master/fs/binfmt_elf....

Especially relevant for dynamic linkers is the AT_PHDR and AT_BASE auxiliary vector entries which provide the address of the executable's program header table and the address of the interpreter, respectively.

https://lwn.net/Articles/519085/

Re: The Journey Before main()

#96
post #38

Earlier quoted context omitted.

The "syscall API" is part of libc too. The read syscall is a trap, you put arguments in the right registers and issue the correct instruction[1] to enter the kernel. That's not something that can be expressed in C. The read() function that your C code actually uses is a C function provided by the C library. [1] "svc 0" on ARM, "int 0x80" on i386, etc...

> That's not something that can be expressed in C. I've often made the argument that compilers should add builtins for Linux system calls. Just emit code in the right calling convention and the system call instruction, and return the result. Even high level dynamic languages could have their JIT compilers generate this code. I actually tried to hack a linux_system_call builtin into GCC at some point. Lost that work i…

> I've often made the argument that compilers should add builtins for Linux system calls. Just emit code in the right calling convention and the system call instruction, and return the result. Even high level dynamic languages could have their JIT compilers generate this code.

You can only do that, when you compile for a specific machine. In general you are compiling for some abstract notion of an OS. JITs always compile for the machine they are running on, so they don't have that problem. There is code, that is compiled directly to your syscalls specific to your machine, so that abstract code can use this. It's called libc for the C language.

> One of the most annoying things in the Linux manuals is they conflate the glibc wrappers with the actual system calls in Linux. The C library does a lot more than just wrap these things, they dynamically choose the best variants and even implement cancellation/interruption mechanisms. Separating the Linux behavior from libc behavior can be difficult, and in my experience requires reading kernel source code.

In my experience there are often detailed explanation in the notes section. From readv(2):

  NOTES
       POSIX.1  allows  an  implementation  to  place a limit on the number of
       items that can be passed in iov.  An implementation can  advertise  its
       limit  by  defining IOV_MAX in  or at run time via the return
       value from sysconf(_SC_IOV_MAX).  On modern Linux systems, the limit is
       1024.  Back in Linux 2.0 days, this limit was 16.

   C library/kernel differences
       The  raw  preadv() and pwritev() system calls have call signatures that
       differ slightly from that of the corresponding GNU  C  library  wrapper
       functions  shown  in  the SYNOPSIS.  The final argument, offset, is un‐
       packed by the wrapper functions into two arguments in the system calls:

           unsigned long pos_l, unsigned long pos

       These arguments contain, respectively, the low order and high order  32
       bits of offset.

   Historical C library/kernel differences
       To  deal  with  the  fact  that IOV_MAX was so low on early versions of
       Linux, the glibc wrapper functions for readv() and  writev()  did  some
       extra  work  if  they  detected  that the underlying kernel system call
       failed because this limit was exceeded.  In the case  of  readv(),  the
       wrapper  function  allocated a temporary buffer large enough for all of
       the items specified by iov, passed that buffer in a  call  to  read(2),
       copied  data from the buffer to the locations specified by the iov_base
       fields of the elements of iov, and then freed the buffer.  The  wrapper
       function  for  writev()  performed the analogous task using a temporary
       buffer and a call to write(2).

       The need for this extra effort in the glibc wrapper functions went away
       with Linux 2.2 and later.  However, glibc continued to provide this be‐
       havior until version 2.10.  Starting with glibc version 2.9, the  wrap‐
       per  functions  provide  this behavior only if the library detects that
       the system is running a Linux kernel older than version 2.6.18 (an  ar‐
       bitrarily  selected  kernel  version).  And since glibc 2.20 (which re‐
       quires a minimum Linux kernel version of  2.6.32),  the  glibc  wrapper
       functions always just directly invoke the system calls.

Re: The Journey Before main()

#97
post #5

It's also possible to pack a whole codebase into "before main()" - or with no main() at all. I was recently experimenting doing this, as well as a whole codebase that only uses main() and calls itself over and over. Good fun: https://joshua.hu/packing-codebase-into-single-function-disr...

Just wondering, how did you get that domain name? I’ve been looking for registrars offering .hu

whois data points to https://www.domain.hu.

Re: The Journey Before main()

#98
post #5

It's also possible to pack a whole codebase into "before main()" - or with no main() at all. I was recently experimenting doing this, as well as a whole codebase that only uses main() and calls itself over and over. Good fun: https://joshua.hu/packing-codebase-into-single-function-disr...

Just wondering, how did you get that domain name? I’ve been looking for registrars offering .hu

https://nic.hu/index_en.html ?

Re: The Journey Before main()

#99
post #60
post #38

Earlier quoted context omitted.

The "syscall API" is part of libc too. The read syscall is a trap, you put arguments in the right registers and issue the correct instruction[1] to enter the kernel. That's not something that can be expressed in C. The read() function that your C code actually uses is a C function provided by the C library. [1] "svc 0" on ARM, "int 0x80" on i386, etc...

The libc syscall wrappers are part of the libc API, but on Linux, syscalls are part of the stable ABI and so you can freely do __asm__(...) to write your own version of syscall(2) and it is fully supported. Yeah, __asm__ is probably not in the C spec, but every compiler implements it... For instance, Go directly calls Linux system calls without going through libc (which has lead to lots of workarounds to emulate some…

Yes, you can. Then you don't write against the OS, but against the kernel. It sometimes works, because the kernel is a separate project, it sometimes doesn't, you gave an example yourself.

> In any case, file descriptors are definitely not a libc construct on Linux.

File descriptors come definitely from the kernel, but they do also exist as a concept in libc, and I was referring to them as such. I was saying that I depend on non-portable libc functions, even though I value portability, because the API is just so nice. I did not want to indicate, that I am doing syscalls directly.

Re: The Journey Before main()

#100
post #47

Earlier quoted context omitted.

I did mean file descriptors.

Then I'm confused by what you meant, because you can use fds with or without libc.

I don't want to bypass libc in general, because I care about portability, but fds are just a nice interface, so I still use them instead of FILE, which would be the portable choice. My calls are still subject to OS choices, that differ from the kernel, since I don't bypass libc.
Post reply on HN