There's also gnu's objcopy. This post covers a lot of different approaches, and has lots of tips: https://www.devever.net/~hl/incbin
[deleted]
Embedding Binary Objects in C
41–50 of 141 posts
Re: Embedding Binary Objects in C
#42I wouldn't do it this way, within the toolchain. Better to just append the data to the finished executable, with a header that contains an identifying marker. The program can scan its own image and look for that marker to get at the data. That will work on any OS with any executable and linker format and can be done post-production (users or downstream distributors can receive the binary and add customized data to it…
This isn't portable. Executable files aren't binary blobs, they're a structured format. If you append data to an executable there is no guarantee it's actually going to end up mapped in memory for you. You'd have to put it into an actual ELF segment, and at that point you're back to using the linker.
It's a lot more sensible to ask the linker to do this as in OP than to hack together something like you've described.
Re: Embedding Binary Objects in C
#43I wouldn't do it this way, within the toolchain. Better to just append the data to the finished executable, with a header that contains an identifying marker. The program can scan its own image and look for that marker to get at the data. That will work on any OS with any executable and linker format and can be done post-production (users or downstream distributors can receive the binary and add customized data to it…
No need to scan the image if the last few bytes has a "footer" describing the length that precedes it.
I still think I like adding it as a section better, though.
Re: Embedding Binary Objects in C
#44If we're in the realm of "non-standard linker tricks"... Compilers will concatenate sections of the same name. You can use this trick to produce a concatenation of arrays across several files: $ cat t1.c __attribute__((section("some_array"))) int a[] = {1, 2, 3}; $ cat t2.c __attribute__((section("some_array"))) int b[] = {4, 5, 6}; $ cat t.c #include extern const int __start_some_array; extern const int __stop_some_…
How exactly is the ordering of items across the section decided? Is it relatively random or is it based on the order that the object files are passed to the linker?
It's unspecified, and I've seen it change over time as the linker changes it's data structures.
Re: Embedding Binary Objects in C
#45I wouldn't do it this way, within the toolchain. Better to just append the data to the finished executable, with a header that contains an identifying marker. The program can scan its own image and look for that marker to get at the data. That will work on any OS with any executable and linker format and can be done post-production (users or downstream distributors can receive the binary and add customized data to it…
The advantage of the approach described in the article is that there's less work to do at startup time. For applications that run on end-users' machines, we should do whatever we can to minimze work done at startup, getting it as close as possible to just mapping a big blob into memory and jumping to the code that does the real work of the program.
> For applications that run on end-users' machines, we should do whatever we can to minimze work done at startup.
That is not an absolute given. If this is some tool that is launched a large number of times from frequently running shell scripts, or from a busy web server's CGI, I'd tend to agree.
The principle you are championing is not widely followed anyway. All sorts of applications that people use on their machines on a daily basis have atrocious startup times. Though "everyone's startup time sucks" is no excuse, it does remove the motivation to worry about single digit millisecond differences.
With my technique we don't actually have to do anything at all until the blob is required. If there are ways of executing the program that don't require the data, that initialization need not even take place. With lazy initialization we can move the cost out of the startup time.
Even if you work at minimizing startup time, there is a lot of activity in a modern application, like attaching shared libraries and opening various files. Contribution from this approach is a like rounding error in the least significant digit of startup time compared to the default stuff that happens before your program even gains control.
Re: Embedding Binary Objects in C
#46I wouldn't do it this way, within the toolchain. Better to just append the data to the finished executable, with a header that contains an identifying marker. The program can scan its own image and look for that marker to get at the data. That will work on any OS with any executable and linker format and can be done post-production (users or downstream distributors can receive the binary and add customized data to it…
This isn't portable. Executable files aren't binary blobs, they're a structured format. If you append data to an executable there is no guarantee it's actually going to end up mapped in memory for you. You'd have to put it into an actual ELF segment, and at that point you're back to using the linker. It's a lot more sensible to ask the linker to do this as in OP than to hack together something like you've described.
> If you append data to an executable there is no guarantee it's actually going to end up mapped in memory for you.
I didn't state it clearly enough, but I didn't say anything about it being mapped. It almost certainly isn't mapped. Loaders do not blindly map the whole thing to memory; then you would end up with debug info unconditionally mapped.
Even if it were mapped, the program wouldn't easily find it with the latter approach I described: the offset given in the structure is measured from the start of the file, not from some address in memory.
> ask the linker
It is not portable either. For instance, it doesn't work with Microsoft's linker which is called link.exe.
Re: Embedding Binary Objects in C
#47My favorite is simply using a tool to create a C file with your binary data: static uint8_t mydata[] = {0xDE, 0xAD, 0xBE, 0xEF, ... }; The advantage is that it works anywhere, with any compiler. The disadvantage is that it can increase compile time. I would limit each .c file to 10MB; it seems there's a quadratic increase in build time with file size, at least with gcc. Also, instead of "0x%02x" I use decimal notatio…
Typically such data can be 'const', too.
Also note that for bytes, hex literals are longer than decimal ones a lot of the time and never shorter due to the "0x" prefix.
Re: Embedding Binary Objects in C
#48I wouldn't do it this way, within the toolchain. Better to just append the data to the finished executable, with a header that contains an identifying marker. The program can scan its own image and look for that marker to get at the data. That will work on any OS with any executable and linker format and can be done post-production (users or downstream distributors can receive the binary and add customized data to it…
No need to scan the image if the last few bytes has a "footer" describing the length that precedes it. I still think I like adding it as a section better, though.
The footer could work. If you round up the file to some reasonable power-of-two block size and put the footer right at the end of the last block, that would be highly portable; it would work on systems/devices that only write binary files in multiples of a block size (even if you write a short block at the end).
Re: Embedding Binary Objects in C
#49There's also gnu's objcopy. This post covers a lot of different approaches, and has lots of tips: https://www.devever.net/~hl/incbin
The image background on that webpage is extremely distracting
Re: Embedding Binary Objects in C
#50I wouldn't do it this way, within the toolchain. Better to just append the data to the finished executable, with a header that contains an identifying marker. The program can scan its own image and look for that marker to get at the data. That will work on any OS with any executable and linker format and can be done post-production (users or downstream distributors can receive the binary and add customized data to it…
Sounds like a great vector for malware.
Any unsigned native executable is a vector for malware. Images with stuff appended to them can be signed.
For instance, think of signed Linux kernel images on secure embedded systems. The signature includes the embedded initramfs and device tree blob.