Live data from Hacker News

Embedding Binary Objects in C

flak.tedunangst.com

11–20 of 141 posts

Re: Embedding Binary Objects in C

#11

Where did they define the symbols `_binary_quine_c_start` and `_binary_quine_c_end`? I would expect that the symbol names would need to be passed to the command that produced the object file from the binary file: ld -r -b binary quine.c -o myself.o -m elf_amd64

They are assuming gnu ld, and that's the pattern it uses. It will change some characters, like "." to "_" to keep valid symbol names.

GNU objcopy has a way to change the generated names. As far as I can tell, ld does not.

Re: Embedding Binary Objects in C

#12
I wrote this tool a long time ago, which allows you to embed files and then access them as a "filesystem" with stdio. May be of interest:

https://git.sr.ht/~sircmpwn/koio

If you just want the contents of the file as a symbol, the approach described in the article is 100% the way to go.

Re: Embedding Binary Objects in C

#13
If we're in the realm of "non-standard linker tricks"...

Compilers will concatenate sections of the same name. You can use this trick to produce a concatenation of arrays across several files:

    $ cat t1.c
    __attribute__((section("some_array"))) int a[] = {1, 2, 3};
    $ cat t2.c
    __attribute__((section("some_array"))) int b[] = {4, 5, 6};
    $ cat t.c
    #include 
    
    extern const int __start_some_array;
    extern const int __stop_some_array;
    
    int main() {
      const int* ptr = &__start_some_array;
      const int n = &__stop_some_array - ptr;
      for (int i = 0; i 
This is the mechanism that linkers use "under the hood" to get a list of C++ object initializers that need to run pre-main().

It's unfortunate that there is no standard way of getting at this functionality in portable C, or to get it in C++ without actually running code pre-main(). Sometimes you really want a linker-initialized list things (of some sort) that have been linked in, without actually running code pre-main() (which has all kind of issues).

I would love to see a thing like this standardized in both C and C++. C++ compilers need it under the hood anyway.

Re: Embedding Binary Objects in C

#14
My favorite is simply using a tool to create a C file with your binary data:

  static uint8_t mydata[] = {0xDE, 0xAD, 0xBE, 0xEF, ... };
The advantage is that it works anywhere, with any compiler.

The disadvantage is that it can increase compile time. I would limit each .c file to 10MB; it seems there's a quadratic increase in build time with file size, at least with gcc.

Also, instead of "0x%02x" I use decimal notation and no spaces in order to decrease the .c file size:

  static uint8_t mydata[] = {
  42,7,105,0,0,...,
  8,0,1,20,...,
  ...,
  };

Re: Embedding Binary Objects in C

#16
I needed this last week! Building an embedded firmware image for a dashboard display, with lots of PNG files for icons etc. 61 of them.

The original developer wrote a tool to expand the PNGs to BMPs (arrays of 32-bit pixel values) and generate a C array definition as a text file. Which is lots bigger than the original PNG (13K => 100K sort of thing). Then included that C source in the build. Used up 700K of my firmware image space which was only 1MB to begin with.

So I wrote a tool to represent the raw PNG file as a C declaration, then added parts of libpng to the boot code to decompress the images during initialization. Even with libpng, I saved over 400K. Now the images use RAM instead of ROM, but that's ok I had buttloads of RAM.

Anyway, this is a much slicker way of including binary files in an image. I may go back and change my build.

Re: Embedding Binary Objects in C

#17

My favorite is simply using a tool to create a C file with your binary data: static uint8_t mydata[] = {0xDE, 0xAD, 0xBE, 0xEF, ... }; The advantage is that it works anywhere, with any compiler. The disadvantage is that it can increase compile time. I would limit each .c file to 10MB; it seems there's a quadratic increase in build time with file size, at least with gcc. Also, instead of "0x%02x" I use decimal notatio…

I wonder what causes it to be nonlinear?

Re: Embedding Binary Objects in C

#18

My favorite is simply using a tool to create a C file with your binary data: static uint8_t mydata[] = {0xDE, 0xAD, 0xBE, 0xEF, ... }; The advantage is that it works anywhere, with any compiler. The disadvantage is that it can increase compile time. I would limit each .c file to 10MB; it seems there's a quadratic increase in build time with file size, at least with gcc. Also, instead of "0x%02x" I use decimal notatio…

The article specifically leads with this as the default technique and then wanted to describe a fun alternative.

Re: Embedding Binary Objects in C

#19

My favorite is simply using a tool to create a C file with your binary data: static uint8_t mydata[] = {0xDE, 0xAD, 0xBE, 0xEF, ... }; The advantage is that it works anywhere, with any compiler. The disadvantage is that it can increase compile time. I would limit each .c file to 10MB; it seems there's a quadratic increase in build time with file size, at least with gcc. Also, instead of "0x%02x" I use decimal notatio…

If you click through from the article to the mailing list post that inspired it, you'll see that that's what they originally used, but it created a memory problem.
Post reply on HN