A new PNG spec

> We really shouldn't be making new standards with big endian byte order.

IFF isn't a wire protocol standard for efficient zero-copy; and nor is it intended for file formats amenable to being streaming-parsed.

And that's okay! Not every format needs to be suited to efficient, scalable, concurrent, [other lovely words] message passing!

IFF has two major use-cases:

1. documents that are "loaded" in some program, where "loading" is expected to occur against a random-access block device; where each chunk will be visited in turn, with either its contents being parsed into an in-memory representation; its contents' slicing bounds being stored to later stream or random-access within (or the part of the file within those bounds being mmap(2)ed — same thing); or that chunk discarded, thus allowing the load operation to skip issuing any read ops for it or its descendants entirely.

This is the PNG use-case.

(Though, interestingly enough, since PNG has only one large chunk — the image data — PNG can be made into an "effectively-streamed format" simply by keeping that big chunk at the end of the IFF document. Presuming the stream length of the PNG file is known [as in a regular HTTP fetch], the "skeleton load" process for PNG can terminate after just having parsed its way through all the other tiny chunks — perhaps with a few minimal buffer waits to skip over unknown chunks — but with no need to buffer the entire image data chunk. [It adds the image-data-chunk length to the file pointer, realizes there's no more room for chunks in the stream, and so doesn't bother to buffer+seek past that final chunk.] The IFF parser then returns to the caller, passing it the slicing bounds of [among other things] the (still not-yet-fully-received) image-data chunk. And the caller can then turn around, and hand the same FILE pointer and those slicing bounds to its streaming renderer, letting it go to town consuming the stream as needed.)

IFF in its skeleton-loading model, would also be ideal for something like e.g. a font file (which has lots of little tables, which are either eagerly parsed, or ignored, by any given renderer.)

2. simple "read-rarely" packfile documents, that act sort of like little databases, but without any sort of TOC header part; where, when you want to grab something from the packfile, you re-navigate down through it from the root, taking the IOPS hit from all the seeks to each nesting-parent chunk's preceeding sibling chunks before hitting the descendant you want to navigate into.

This is the use-case of most IFFv1 file formats — most of them were made for use by programs that would grab this or that for the program's use either once at startup, or when the thing became relevant. (Think of the types of things a Windows executable embeds as "resources" — icons, translated strings, XAML declarative-MVC-view documents, etc.)

For a parallel, IFF here is to "using an entire archive-format library like tar or zip to store these assets for random access", as "spitting CSV/XML out using template strings" is to using a library to encode a table to a Parquet/ORC/etc. table.

The parallel is that in both cases, you're trading some performance and robustness, for massively reduced complexity and ease of implementation. Like with emitting CSV, you can slop together an IFF encoder right there inside your data-emitting logic — in any language that can write out binary files, and without even having access to the Internet, let alone adding a dependency on an encoder package in some package ecosystem. You can do it in C; you can do it in assembly; you can do it in a bash script; you can do it in BASIC; you can do it in a Windows batch file; you can do it in your single-file Python or Ruby or Perl script that lives in your repo. You can probably do it in a Makefile!

(Also, given how IFF parsing works [i.e. given that any given chunk's contents is in superposition of being either an opaque binary slice or a potential stream of child chunks, with a streaming event-based parser able to decide at each juncture whether to take that step of decoding the child chunks or to leave them as an undecoded binary for now], if you start to care about performance, you can just stick some memoization in front of your "fetch a key-path-lens KP from document D" function, and now you're building a just-in-time TOC. And obviously you can put TOC chunks in your IFF-based file formats if you want — though IMHO doing so kind of goes against the spirit of IFF.)

---

In neither of those use-cases does it really matter that lengths require reading four bytes one-at-a-time with left-shifts, rather than being able to just plop the four bytes into a register. These aren't cases where the parse overhead of the the structural glue between the data, will ever be non-trivial relative to the time it takes to consume the data itself.

And even if you did want to use IFF for something crazy, like as a substitute for Protobuf: did you know that most modern CPU ISAs have a byte-shuffle instruction that can transform big-endian into little-endian [among an unbounded number of other potential transformations] in a single cycle? Endian-ness did matter in protocol design for a while... but these days, unless you're e.g. a Google engineer designing a new SAN protocol, and optimizing it for message-handling overhead on your custom SDN L7 network-switch silicon that doesn't have a shuffle op... endian-ness is mostly irrelevant again!