Not going to lie, I thought this was going to be opining on the Christmas movie Elf
On ELF, Part 2 (2018)
21–30 of 36 posts
Re: On ELF, Part 2 (2018)
#22Just submitting part 2 of this as I stumbled across it after having played around with write ELF files by hand. Never knew about the Hunk format and I think the author makes some interesting points.
Might you have any interesting resources you could share regarding how you learned to write ELF files by hand?
These bootstrapping compilers [1][2] as well as the posts here [3].
[1] https://github.com/smtlaissezfaire/bcompiler
Re: On ELF, Part 2 (2018)
#23Reminds me that I have an unfinished project to write a 68k ELF binary loader for the Atari ST that I never finished. Now that I've quit my job maybe I can return to this.
Beats cleaning the garage.
Re: On ELF, Part 2 (2018)
#24Off topic — does anyone know more about what happened to the Kestrel project? It sounds from the note in the archived GitHub project[0] like the maintainer shut it down because a company released a similar product with the same name. Seems odd (and sad). [0] https://github.com/KestrelComputer/kestrel
https://www.youtube.com/watch?v=WHogxYNkzp8 - He actually presented about that at the svfig meeting a few days ago!
Re: On ELF, Part 2 (2018)
#25This is all well and good, but in reality everyone is using all the rich features of ELF -- dynamic linking obviously, but also symbol versioning, constructors, symbol interposition (probably the worst one for performance), symbol visibility, preload, etc. The question is how to make it fast in the common case, and actually the Linux and glibc authors are doing a pretty good job here.
One thing I saw in KDE applications (KDE has split its base libraries into a bazillion small libraries) is that looking up a symbol in a dynamic library is fast, but linearly trying all the libraries is costly - (IIRC) 20-40% of startup time. A possible solution would be to store with each symbol where it's expected to come from. Required libraries are listed as an array, so the "where" could be an index into the arr…
> A possible solution would be to store with each symbol where it's expected to come from.
Solaris extended their ELF format with "Direct Binding" in 2005. It messes with intentional interposition though. (https://blogs.oracle.com/solaris/post/direct-binding-now-the...>)
Michael Meeks did an implementation of Direct Binding for GNU binutils and glibc but it got rejected to be in the mainline with "prelink is more efficient". (https://lwn.net/Articles/192082/>)
I haven't found anything about the Direct Binding in GNU that is more recent than 2006. Maybe time to revisit?
Re: On ELF, Part 2 (2018)
#26So is there a layman “intro to executable loader formats”. Like what are things to consider when designing a format?
Re: On ELF, Part 2 (2018)
#27> "the big mistakes from Unix, besides the X Window System" The entire X11 system was "a mistake". Got it, we should have stuck to 7-bit text on a VT-100 because that was utter perfection. With that compelling intro I lost any interest in any other arguments the author presented.
Probably worth noting that his criticism comes from someone that builds systems and yours as a user.
Re: On ELF, Part 2 (2018)
#28So is there a layman “intro to executable loader formats”. Like what are things to consider when designing a format?
There are OS internals books that document this kind of thing (e.g., various books on FreeBSD, Solaris, MacOS, etc.). You can also learn a lot just by reading the specifications for various object file formats, or the source code to tools like linkers and debuggers. There are also some dated books that are specifically on linkers and loaders (I've only leafed through them in a bookstore, don't know how good they are).
But on the whole, I'd say you learn on the job. Project, anyway.
In days of yore these systems-level structures often arose out of a project's specific needs and the individual experience of the people on staff. For example, I sat next to the person who designed the GEMDOS executable format (we needed one, the old one in CP/M-68K was terrible), and I think it was done in a day or two. The engineer in question had maybe 15 years industry experience, including some time as a systems programmer at IBM; I think the format would have been different (maybe better, maybe worse, how would we know?) if a different engineer had decided to do that work.
I used a couple tricks from the GEMDOS executable format to do some rather nifty runtime work at Apple (it's not like the format was secret or anything). That's cross-pollination for you.
Re: On ELF, Part 2 (2018)
#29I worked with a.out on early unixes and elf on later ones - not addressed here is the main reason for switching: ELF allowed you manage pages contiguously on disk so that they could be paged in directly (stuff can be page aligned within the file at it's natural offsets), while a.out was designed for swapped kernels where you would just read() data into a text section, address 0 might be 32 bytes from the start of a f…
What’s the advantage of that? Would the kernel just load the entire file into contiguous memory and set up pages at the correct physical addresses?
There are also segmentation differences. Only one copy of the read-only data and code (aka text) segments are mapped into all programs which load the library. However, each program instance gets its own unique pages of read-write data. Those pages are always located at a fixed offset from their code segment. That way, modern CPUs can execute PC-relative instructions to generate their addresses. The global data segments for a module are always located at a fixed offset relative to the code pages.
See ARMv8 `adrp` (address of PC-relative page), RISC-V `auipc` (add upper immediate to program counter), and x86-64 PC-relative addressing for some modern examples.
Then go backwards in time and see how ARMv7 does it (literal pools) and how some earlier RISCs did it (Itanium and Alpha global pointer aka "gp", PowerPC table-of-contents register).
Re: On ELF, Part 2 (2018)
#30> "the big mistakes from Unix, besides the X Window System" The entire X11 system was "a mistake". Got it, we should have stuck to 7-bit text on a VT-100 because that was utter perfection. With that compelling intro I lost any interest in any other arguments the author presented.
Probably worth noting that his criticism comes from someone that builds systems and yours as a user.
System programming isn't a dark art, it's just programming with programmers as an audience. Application programming focuses on end-users who may not be programmers. I've done both.
I've used and developed on X11 since 1990. I'm well aware of its numerous limitations and I'm glad new technology such as Wayland is being developed to replace it. But it wasn't "a mistake".