Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We Need A Standard Layered Image Format

shapeof.com

Post reply on HN
Print view View original

71–80 of 145 posts

Re: We Need A Standard Layered Image Format

#71

Post by pmahoney »

Someone wrote: post #34
derefr wrote: post #27

Earlier quoted context omitted.

> For example this is how Android (.apk) and iOS (.ipa) applications are distributed. A more relevant example: this is how .docx files work, for very much the same reasons (one XML file at the root, and then a bunch of referenced resources.)

Apple also uses zip for Keynote, Pages and Numbers documents, again for very much the same reasons, and Java's .jar, .war, .ear, .whatevar files also use zip. .zip seems to be the universal 'combine some stuff in a single file' format, probably because it is old enough to be guaranteed patent free (OTOH: http://broadcast.oreilly.com/2010/06/is-zip-in-the-public-do... and http://www.pkware.com/support/zip-app-note . h…

It also, unlike tar.gz, provides an index for easily seeking to and extracting just one file from the zip archive. My understanding is that doing this in a .tar.gz means unzipping and processing the whole tarball as a stream.

Re: We Need A Standard Layered Image Format

#72

Post by bane »

One nice thing about using SQLite is that various compressed versions of the flattened image could just be stored as blobs in another table in the same file. For example, my camera shoots with a RAW+JPG mode that produces two files, why not store the RAW data as described here and the JPG as an easy to grab/display item for quick image viewers etc. Hell store various common versions of thumbnails and icons in there as well.

Re: We Need A Standard Layered Image Format

#73

Post by bane »

thristian wrote: post #17
aidos wrote: post #5

This whole idea seems totally mad from a distance but as a developer I'd love to be able to open complex files as DBs. What would be the downfalls of passing around a DB as an image file? Does the compression suck? Does the performance suck? What makes this a terrible idea? EDIT: to clarify, by compression I'm referring to the non-lossy type - I'm assuming that within the db you'd have already processed assets.

One downside would be that tools like file(1) that try to guess a file's format by examining its contents would return "SQLite database" for every image file, which isn't the most helpful result. Since SQLite aims to be a generic way for applications to store data, it'd be nice if its "open database" and "create database" functions allowed you to supply, say, a 4- or 8-byte magic value that would be stored at a fixed…

why not reserve a specially named table with the format identifier in there that file(1) could poke into and take a look at?

Re: We Need A Standard Layered Image Format

#74

Post by mahmud »

SQLite wrote: post #45
thristian wrote: post #17

Earlier quoted context omitted.

One downside would be that tools like file(1) that try to guess a file's format by examining its contents would return "SQLite database" for every image file, which isn't the most helpful result. Since SQLite aims to be a generic way for applications to store data, it'd be nice if its "open database" and "create database" functions allowed you to supply, say, a 4- or 8-byte magic value that would be stored at a fixed…

There are currently 24 contiguous bytes of unused space in the SQLite header. If need be, and if SQLite catches on for use as a portable image format, I will be willing to allocate some or all of those 24 bytes to an identifier string for file(1).

What an awesome response.

Re: We Need A Standard Layered Image Format

#75

Post by derefr »

pmahoney wrote: post #71
Someone wrote: post #34

Earlier quoted context omitted.

Apple also uses zip for Keynote, Pages and Numbers documents, again for very much the same reasons, and Java's .jar, .war, .ear, .whatevar files also use zip. .zip seems to be the universal 'combine some stuff in a single file' format, probably because it is old enough to be guaranteed patent free (OTOH: http://broadcast.oreilly.com/2010/06/is-zip-in-the-public-do... and http://www.pkware.com/support/zip-app-note . h…

It also, unlike tar.gz, provides an index for easily seeking to and extracting just one file from the zip archive. My understanding is that doing this in a .tar.gz means unzipping and processing the whole tarball as a stream.

Indeed--squashfs was basically created to deal with the inadequacies of random-access-extraction from compressed tarballs.

Re: We Need A Standard Layered Image Format

#76

Post by Semiapies »

gilgoomesh wrote: post #68
Semiapies wrote: post #62

This is an interesting idea, though I cringe a little at using a relational database as a key/value store (as in the case of layer_attributes). I wonder whether it would be more useful to just represent the whole thing as a big JSON structure with base64 blobs. Each layer can be an object in the "layers" array with the attributes as keys.

Layered images can easily grow to gigabytes in size. You do not want to store that much information in a format like JSON that must be opened into memory, parsed and stored in native objects. SQLite is RAM efficient and quick to open, even at large file sizes.

Fair enough - I rarely work much with images, so didn't even think of size.

Re: We Need A Standard Layered Image Format

#77

Post by aceofspades19 »

arielpts wrote: post #64
aceofspades19 wrote: post #4

Whats wrong with SVG? Doesn't it accomplish what the OP wants to do already and is already a standard supported by a lot of applications? I must be missing something here

XML is a really bad format for containing large chunks of binary data (bitmap)

And why are we storing binary data? The whole point of SVG is everything is in XML unless I'm very mistaken

Re: We Need A Standard Layered Image Format

#80

Post by supercoder »

aceofspades19 wrote: post #77
arielpts wrote: post #64

Earlier quoted context omitted.

XML is a really bad format for containing large chunks of binary data (bitmap)

And why are we storing binary data? The whole point of SVG is everything is in XML unless I'm very mistaken

How else would you propose to store a bitmap.
Post reply on HN