Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How is a binary executable organized? Let's explore it (2014)

jvns.ca

Post reply on HN
Print view View original

61–70 of 93 posts

Re: How is a binary executable organized? Let's explore it (2014)

#61

Post by matheusmoreira »

vlovich123 wrote: post #8
sva_ wrote: post #4

> When the program starts running, you might think it starts at main. It doesn’t! It actually goes to _start. This does a bunch of Very Important Things that I don’t understand very well, including calling main. So I won’t explain them. The way I understand it, the symbol main is a C-specific thing. The symbol _start is a language-agnostic entry point for the binary that will in this case call main. A convention of i…

> Things that I don’t understand very well, including calling main. So I won’t explain them. It depends on the language runtime, but a common task will be initializing global non-0 statics. For languages like Rust/C/C++ you can also inject variables to be initialized via linker flags. Before start if the program is dynamically linked then I believe the linker runtime is run to resolve the links and then transfer cont…

> initializing global non-0 statics

Does that mean this doesn't work in a freestanding environment? Yet another reason to assiduously avoid global variables. I suppose that's why I never ran into this issue.

> Basically hacks on hacks on hacks

And then there's this insanity right here:

https://blogs.oracle.com/solaris/post/init-and-fini-processi...

I like to imagine that I won't have to actually implement this when I write my ELF loader. Someone please tell me no modern software uses this.

Re: How is a binary executable organized? Let's explore it (2014)

#62

Post by matheusmoreira »

sva_ wrote: post #4

> When the program starts running, you might think it starts at main. It doesn’t! It actually goes to _start. This does a bunch of Very Important Things that I don’t understand very well, including calling main. So I won’t explain them. The way I understand it, the symbol main is a C-specific thing. The symbol _start is a language-agnostic entry point for the binary that will in this case call main. A convention of i…

> the symbol main is a C-specific thing

Absolutely. And only available on hosted C. Freestanding C lets you have any entry point you want.

> The symbol _start is a language-agnostic entry point for the binary that will in this case call main.

That's just the linker's default. You can set it to a nicer symbol with -Wl,--entry="${symbol}" and GCC even supports setting it directly with no need for the unsightly -Wl.

Also, the entry point is actually a pointer, not a symbol. The linker just takes the address of the symbol you specify and sets the ELF entry point to that.

> calling the entry point _start with main's argc/argv

In addition to argument count and argument vector, the stack also contains the environment vector and the auxiliary vector. The process startup code can be as simple as popping all that stuff off the stack and into the appropriate registers and then calling a C function of your choosing. Note that the entry point is not itself a function: there's nothing to return to. The entry point code finishes with an exit system call to ensure clean process termination when main returns a status code. This is how things work on Linux at least.

Re: How is a binary executable organized? Let's explore it (2014)

#63

Post by matheusmoreira »

Retr0id wrote: post #3
jstrieb wrote: post #2

As I've said in other threads https://news.ycombinator.com/item?id=38847750#38862450 , I highly recommend writing an ELF by hand at least once. It's a great exercise to understand the basic parts of an executable. It's also helpful if you want to go the opposite direction of this article - bottom-up instead of top-down. Lots of other great discussion in various threads on that other HN post.

Similarly, I'd recommend writing a simple ELF loader. There's a fair bit of implementation complexity in dynamic linking, but if you only support static ELFs then it's straight-forward.

Any good resources on the matter? I'm gonna need to write a fully featured ELF loader for my language soon. I need to prepare.

Re: How is a binary executable organized? Let's explore it (2014)

#64

Post by matheusmoreira »

akdas wrote: post #55
bruce343434 wrote: post #50

Earlier quoted context omitted.

That's an awesome interactive page! Did you write it by hand, or did you use some sort of generator/tool?

I wrote it all by hand :) Lately, I've been using Svelte for interactive visualizations (see my post on using a tool called Astro with Svelte: https://avikdas.com/2023/12/30/interactive-demos-using-astro... ). But this one is all hand-written JS!

You and your web site are huge inspirations for me.

Re: How is a binary executable organized? Let's explore it (2014)

#65

Post by matheusmoreira »

jstrieb wrote: post #2

As I've said in other threads https://news.ycombinator.com/item?id=38847750#38862450 , I highly recommend writing an ELF by hand at least once. It's a great exercise to understand the basic parts of an executable. It's also helpful if you want to go the opposite direction of this article - bottom-up instead of top-down. Lots of other great discussion in various threads on that other HN post.

Modifying existing ELFs can also be extremely educational and fun. It's a bit frustrating at first because it's more or less impossible to debug this stuff when it doesn't work but when things finally start working it's awesome. Turns out it's possible to patch ELFs in all sorts of interesting ways. With the auxiliary vector you can even have introspection at runtime: Linux gives us the address of the program header table and from there you can get to anywhere. Just gotta extend a LOAD segment to cover the whole binary.

For example I wrote tools to embed lisp modules and code right into my lisp's interpreter executable. The embedded segment is loaded from the ELF automatically, the interpreter just finds it and runs the code. I'm so proud of this little feature I wrote an article about it.

https://www.matheusmoreira.com/articles/self-contained-lone-...

Would be cool if mainstream languages adopted this method.

Re: How is a binary executable organized? Let's explore it (2014)

#67

Post by toast0 »

Retr0id wrote: post #3
jstrieb wrote: post #2

As I've said in other threads https://news.ycombinator.com/item?id=38847750#38862450 , I highly recommend writing an ELF by hand at least once. It's a great exercise to understand the basic parts of an executable. It's also helpful if you want to go the opposite direction of this article - bottom-up instead of top-down. Lots of other great discussion in various threads on that other HN post.

Similarly, I'd recommend writing a simple ELF loader. There's a fair bit of implementation complexity in dynamic linking, but if you only support static ELFs then it's straight-forward.

I had written a static ELF loader for reasons, but when I was no longer to compile a static version of the binary I wanted to load, I found it wasn't too hard to load the system's dynamic loader instead. That's kind of the best of both worlds --- I can run a dynamicly linked binary, and I didn't have to do the linking and relocations.

Re: How is a binary executable organized? Let's explore it (2014)

#68

Post by alkh »

For a person with a heavy Python background, can anyone suggest a resource/book that would be a good applied intro to practical low-level programming? I've recently started learning Rust and realized I need to catch up on many things. I haven't taken any compiler course, so maybe that's the reason I am missing so much information. For example, I had no idea that symbols in a binary were a thing or what the difference between ELF/MACH-O was

Re: How is a binary executable organized? Let's explore it (2014)

#69

Post by neerajsi »

matheusmoreira wrote: post #61
vlovich123 wrote: post #8

Earlier quoted context omitted.

> Things that I don’t understand very well, including calling main. So I won’t explain them. It depends on the language runtime, but a common task will be initializing global non-0 statics. For languages like Rust/C/C++ you can also inject variables to be initialized via linker flags. Before start if the program is dynamically linked then I believe the linker runtime is run to resolve the links and then transfer cont…

> initializing global non-0 statics Does that mean this doesn't work in a freestanding environment? Yet another reason to assiduously avoid global variables. I suppose that's why I never ran into this issue. > Basically hacks on hacks on hacks And then there's this insanity right here: https://blogs.oracle.com/solaris/post/init-and-fini-processi... I like to imagine that I won't have to actually implement this when I…

Wow, great read. I worked on the Windows DLL loader and we had to implement similar mechanics for similar reasons. The PE image format makes some part of this a little easier, but the complexity is essentially the same.

Re: How is a binary executable organized? Let's explore it (2014)

#70

Post by [deleted] »

matheusmoreira wrote: post #65
jstrieb wrote: post #2

As I've said in other threads https://news.ycombinator.com/item?id=38847750#38862450 , I highly recommend writing an ELF by hand at least once. It's a great exercise to understand the basic parts of an executable. It's also helpful if you want to go the opposite direction of this article - bottom-up instead of top-down. Lots of other great discussion in various threads on that other HN post.

Modifying existing ELFs can also be extremely educational and fun. It's a bit frustrating at first because it's more or less impossible to debug this stuff when it doesn't work but when things finally start working it's awesome. Turns out it's possible to patch ELFs in all sorts of interesting ways. With the auxiliary vector you can even have introspection at runtime: Linux gives us the address of the program header…

[deleted]
Post reply on HN