Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Embedding Binary Objects in C

flak.tedunangst.com

Post reply on HN
Print view View original

51–60 of 141 posts

Re: Embedding Binary Objects in C

#51

Post by gmueckl »

kazinator wrote: post #35

I wouldn't do it this way, within the toolchain. Better to just append the data to the finished executable, with a header that contains an identifying marker. The program can scan its own image and look for that marker to get at the data. That will work on any OS with any executable and linker format and can be done post-production (users or downstream distributors can receive the binary and add customized data to it…

You should check out how self-extracting archives work. All of those that I am aware of have a first part that is a regular executable and the archive to ve extracted is appended to that. File formats like zip have an advantage here as they keep their index at the end of the file after the compressed data. So there is no need to scan for the start of the archive.

Re: Embedding Binary Objects in C

#54

Post by KMnO4 »

hikarudo wrote: post #14

My favorite is simply using a tool to create a C file with your binary data: static uint8_t mydata[] = {0xDE, 0xAD, 0xBE, 0xEF, ... }; The advantage is that it works anywhere, with any compiler. The disadvantage is that it can increase compile time. I would limit each .c file to 10MB; it seems there's a quadratic increase in build time with file size, at least with gcc. Also, instead of "0x%02x" I use decimal notatio…

I've never noticed the "quadratic" build time increase you mention, so I did a test[0]. Files in size from 1mb to 50mb, 3 trials each. These are the results, and they look absolutely linear[1].

[0]: https://pastebin.com/Z9329xkc

[1]: https://i.imgur.com/I3XSBDg.png

Re: Embedding Binary Objects in C

#55

Post by pdw »

Unfortunately, this technique is a bit problematic with modern C compilers. Because the `start` and `end` symbols are unrelated objects, as far as the compiler is concerned, the subtraction `&end - &start` to get the length of the data invokes undefined behavior. Just for that reason, I feel the include file with a hex dump is the better method.

Re: Embedding Binary Objects in C

#57

Post by mhh__ »

damnyou wrote: post #56

This is simply the include_bytes! macro in Rust. There's also the include_str! macro which does a compile-time check for UTF-8 validity.

I'm pretty sure this is trivial in D too although you have to tell the compiler the file exists (you can't do arbitrary FS reads at compile time)

Re: Embedding Binary Objects in C

#58

Post by tyhoff »

tyingq wrote: post #5

There's also gnu's objcopy. This post covers a lot of different approaches, and has lots of tips: https://www.devever.net/~hl/incbin

Thanks for the link! That’s got a lot of goodies.

One of us from Memfault also wrote a post about tips with Binutils, and this approach is covered in it.

https://interrupt.memfault.com/blog/gnu-binutils#objcopy

Re: Embedding Binary Objects in C

#59

Post by zevv »

Totally not relevant for this post, but this is how we do this in Nim:

    const a = readFile("mydata")
The `const` makes the expression evaluate at compile time, conveniently slurping the file into `a`, where it will be available at run time.

Re: Embedding Binary Objects in C

#60

Post by ghostpepper »

pdw wrote: post #55

Unfortunately, this technique is a bit problematic with modern C compilers. Because the `start` and `end` symbols are unrelated objects, as far as the compiler is concerned, the subtraction `&end - &start` to get the length of the data invokes undefined behavior. Just for that reason, I feel the include file with a hex dump is the better method.

Can you elaborate on this? Does "unrelated" have any special meaning in this context? Why is this operation undefined?
Post reply on HN