Adopting the Parallel DWARF linker in dsymutil • Hacker News Board

Re: Adopting the Parallel DWARF linker in dsymutil

#11

Post by skissane » Wed, Jun 10, 2026, 6:24 AM UTC

eqvinox wrote: post #10

skissane wrote: post #9

Earlier quoted context omitted.

If Apple wants to add some brand new feature to Mach-O, they generally just define a new load command. There's generally just one way to do it. The main downside, is (in practice) only Apple can do it. [0] Whereas, with ELF – if you want to add a new feature, do you add it as a new program table entry type (PT_), or a new note type (NT_), or a separate note section (SHT_NOTE)? [0] Well, historically OSF/1 also used M…

> Whereas, with ELF – if you want to add a new feature, do you add it as a new program table entry type (PT_), or a new note type (NT_), or a separate note section (SHT_NOTE)? It won't be a note section since ELF binaries don't need to have any sections. You can remove all section headers from an existing ELF binary and it'll still work (sstrip [ https://github.com/aunali1/super-strip - not sure the one in ELFkickers…

Well, you'll have a SHT_NOTE section and then PT_NOTE pointing to its contents, right? So that way the notes are accessible both via section headers and program headers.

Mach-O does similarly have a distinction between sections as a link-time view, and segments as an execution-time view – a Mach-O contains one or more segments (LC_SEGMENT for 32-bit, LC_SEGMENT_64 for 64-bit), each of which contains a segment header followed by zero or more section headers, so the sections are subdivisions of the segment. Mach-O has names for both segments and sections; ELF only has names for sections.

Let me put it this way – if you wanted to add embedded digital signatures to ELF, so the kernel could verify them at runtime, similar to Mach-O LC_CODE_SIGNATURE – what is the extension point you'd use? The Linux kernel supports digital signatures for ELF executables in conjunction with IMA, but instead of storing the signature in the executable like Mach-O does, it stores it in a filesystem xattr.

Re: Adopting the Parallel DWARF linker in dsymutil

#12

Post by eqvinox » Wed, Jun 10, 2026, 4:11 PM UTC

skissane wrote: post #11

eqvinox wrote: post #10

Earlier quoted context omitted.

> Whereas, with ELF – if you want to add a new feature, do you add it as a new program table entry type (PT_), or a new note type (NT_), or a separate note section (SHT_NOTE)? It won't be a note section since ELF binaries don't need to have any sections. You can remove all section headers from an existing ELF binary and it'll still work (sstrip [ https://github.com/aunali1/super-strip - not sure the one in ELFkickers…

Well, you'll have a SHT_NOTE section and then PT_NOTE pointing to its contents, right? So that way the notes are accessible both via section headers and program headers. Mach-O does similarly have a distinction between sections as a link-time view, and segments as an execution-time view – a Mach-O contains one or more segments (LC_SEGMENT for 32-bit, LC_SEGMENT_64 for 64-bit), each of which contains a segment header…

> Well, you'll have a SHT_NOTE section and then PT_NOTE pointing to its contents, right?

No, the PT_NOTE points directly at parts of the file. The SHT_NOTE is still there because tools don't bother removing it, but it's essentially wrong to use any section information on a finished binary. The loader and dynamic linker don't.

> […] so the sections are subdivisions of the segment. […]

Ok, I'll say this is probably cleaner than the ELF n:m relationship between PT_LOAD blocks and sections. I guess the ELF way is a tiny bit more efficient (since there's fewer instructions for the loader to follow) but that really doesn't matter in face of the cost of doing relocations.

> Let me put it this way – if you wanted to add embedded digital signatures to ELF, […]

Hm. I'm not a libc/kernel developer, so take this with multiple grains of salt, but my train of thought would be this:

There's already GNU_RELRO which makes a statement about "spans" in the file. I'm not sure if it even makes sense to sign a binary partially, but let's say it does. In that case, I'd start with a new PT_SIGNED, of which there could be multiple, defining chunks of the file to be signed. Unfortunately, the signature itself can't be put into that same header, since it has no payload. (Actually... I guess e_phentsize could be bumped up. Hmm. Not sure how much breakage that would cause. But also, then you'd need to skip the signatures in the middle of program headers…) Left with having to put the signature(s) somewhere (one? multiple, if multiple chunks are getting signed? not sure.) - hmm, I'm torn between notes and a second new program header. I'm slightly leaning towards notes, but don't really know.

[Ed.:] oh, wait. The hypothetical PT_SIGNED program header would probably only use the PhysAddr / FileSize fields. The VirtAddr could point at the actual signature. (Or other way around?) A little hacky, but could maybe be executed well enough to not turn extremely ugly…

Re: Adopting the Parallel DWARF linker in dsymutil

#13

Post by skissane » Thu, Jun 11, 2026, 2:04 AM UTC

eqvinox wrote: post #12

skissane wrote: post #11

Earlier quoted context omitted.

Well, you'll have a SHT_NOTE section and then PT_NOTE pointing to its contents, right? So that way the notes are accessible both via section headers and program headers. Mach-O does similarly have a distinction between sections as a link-time view, and segments as an execution-time view – a Mach-O contains one or more segments (LC_SEGMENT for 32-bit, LC_SEGMENT_64 for 64-bit), each of which contains a segment header…

> Well, you'll have a SHT_NOTE section and then PT_NOTE pointing to its contents, right? No, the PT_NOTE points directly at parts of the file. The SHT_NOTE is still there because tools don't bother removing it, but it's essentially wrong to use any section information on a finished binary. The loader and dynamic linker don't. > […] so the sections are subdivisions of the segment. […] Ok, I'll say this is probably cle…

> hmm, I'm torn between notes and a second new program header. I'm slightly leaning towards notes, but don't really know.

This is my point-Mach-O is more elegant because it applies the “there should be one way to do it” dictum made famous by Python-the answer is load commands. ELF gives you multiple ways to do the same thing and it isn’t always clear which one people should use.

> Left with having to put the signature(s) somewhere (one? multiple, if multiple chunks are getting signed? not sure.)

The way macOS handles it-the binary contains a table of hashes, with one hash per a page. The table of hashes is itself hashed, and then that hash of hashes is signed. When loading the executable, the kernel saves the table of hashes in kernel memory, checking the hash on the table of hashes and its signature as it does so. Then, all the kernel has to do, is whenever it reads into memory a page from disk, it looks up the expected hash of the page in the hash table, and compares it it to the hash of the page data being read in. (This is a somewhat simplified explanation, not 100% accurate, but conveys the gist of how it works.)

Re: Adopting the Parallel DWARF linker in dsymutil

#14

Post by eqvinox » Thu, Jun 11, 2026, 4:00 AM UTC

skissane wrote: post #13

eqvinox wrote: post #12

Earlier quoted context omitted.

> Well, you'll have a SHT_NOTE section and then PT_NOTE pointing to its contents, right? No, the PT_NOTE points directly at parts of the file. The SHT_NOTE is still there because tools don't bother removing it, but it's essentially wrong to use any section information on a finished binary. The loader and dynamic linker don't. > […] so the sections are subdivisions of the segment. […] Ok, I'll say this is probably cle…

> hmm, I'm torn between notes and a second new program header. I'm slightly leaning towards notes, but don't really know. This is my point-Mach-O is more elegant because it applies the “there should be one way to do it” dictum made famous by Python-the answer is load commands. ELF gives you multiple ways to do the same thing and it isn’t always clear which one people should use. > Left with having to put the signatur…

Oh I didn't even consider that you'd want to hash individual pages.

…that does make the answer for ELF a program header & then you basically do the same thing. Notes are (as I understand it) for small pieces of information (e.g. a single hash, not a whole bunch of them.)

I see where you're coming from though. Thanks for the very informative discussion!