Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Biggest image in the smallest space

bamsoftware.com

Post reply on HN
Print view View original

61–70 of 106 posts

Re: Biggest image in the smallest space

#61

Post by wiredfool »

jerf wrote: post #30
michaelmior wrote: post #2

Actually, it decompresses to a 5.8MB PNG. However, many graphics programs may choose to use three bytes per pixel when rendering the image and because it has incredibly large dimensions, this representation would take up 141GB of RAM.

One of the rules of secure programming is that any program that is used in an even remotely security-sensitive context, and anything displaying a Portable Network Graphic is likely to be used in such a context, must be able to specify resource usage limits. In this case that could be dimensions or a limit on the total RAM allowed to be used. Limits need not be hard, either, but could produce a query, for instance, th…

Limits, yes. Callback, maybe not so much.

https://github.com/python-pillow/Pillow/blob/master/PIL/PngI...

Re: Biggest image in the smallest space

#62

Post by inglor »

raffomania wrote: post #28
inglor wrote: post #10

This does wonders when used in favicons :D

I just tried it on a locally served page, and my browser handles it quite well (although it won't really display it).

Only on firefox and chrome since they fixed it https://github.com/benjamingr/favicon-bug

Re: Biggest image in the smallest space

#64

Post by semi-extrinsic »

saalweachter wrote: post #33
semi-extrinsic wrote: post #31

If you follow the "related reading" link on the bottom of TFA, you come to a page by Glenn Randers-Pehrson discussing how libpng deals with decompression bombs. On the bottom of that page you find the following curious note; anyone know what to make of it? """ [Note for any DHS people who have stumbled upon this site, be aware that this is a cybersecurity issue, not a physical security issue. Feel free to contact me…

He's presumably had problems with people confusing "decompression bombs" with the blowy-up kind and sending him panicky e-mails.

Ah, of course. That didn't even cross my mind, for some reason; bomb in this context was so obviously not a physical device.

Re: Biggest image in the smallest space

#65

Post by [deleted] »

DanBC wrote: post #8

That's impressive. Here are some other compression curiosities. http://www.maximumcompression.com/compression_fun.php A 24 byte file that uncompresses to 5 MB; another file with good compression under RAR but almost no compression under ZIP; and a compressed file that decompresses to itself.

[deleted]

Re: Biggest image in the smallest space

#69

Post by dahart »

phkahler wrote: post #50
dahart wrote: post #43

Having dealt with and printed a lot of very large images, e.g., 60k x 60k pixels, I have been on the lookout for image processing software that never decompresses the entire image into ram, but instead works on blocks or scan lines or blocks of scan lines, but stays in constant memory and streams to and from disk. For example, the ImageMagick fork GraphicsMagick does a much better job of this than ImageMagick. What o…

The key is not to store it in raster form in RAM. Either tiles (like GIMP) or I prefer Z-ordering. Then a user can zoom in and pan around easily - you let the system swap and it won't be bad at all. If they zoom out though, you probably want to store MIP maps of it. Swap works well for this as long as your data has good locality. huge raster images don't. But no, I'm not aware of any software that handles stuff like…

What does Z-ordering mean in this context?

I definitely want to avoid swap at all costs and find things that are designed to tile & stream instead. The difference between GraphicsMagick resizing an image by streaming and ImageMagick resizing an image that hits swap is orders of magnitude - seconds versus hours.

Re: Biggest image in the smallest space

#70

Post by dahart »

userbinator wrote: post #52
michaelmior wrote: post #2

Actually, it decompresses to a 5.8MB PNG. However, many graphics programs may choose to use three bytes per pixel when rendering the image and because it has incredibly large dimensions, this representation would take up 141GB of RAM.

Better graphics programs will not attempt to put the whole image into RAM, but only decompress the pieces needed for processing it. I remember working with multi-megapixel images on systems with far less than 1MB of RAM, many years ago. Perhaps this is a good example of how more hardware resources can lead to them being wasted - the fact that RAM has grown so much that most images fit completely in it, has also meant…

This! Command line programs have no excuse, they should never need to decompress the entire file to memory. GUI image editors and web browsers probably generally do need to, but there definitely are such options for dealing with more pixels than you can display.

Anyway, do you know some of these "better" graphics programs that actually behave this way, especially command line processing? I am interested in finding more of them.

EDIT: Okay, I have to add & admit that by "no excuse", I actually mean somewhat the opposite. ;) I mean that its possible to do streaming image processing on compressed formats, not that its trivial to do or as easy as decompressing the file in a single call. I just wish that programs would handle very large images more, and it sucks when they don't even though I know its possible. Especially programs intended for dealing with large images like Hugin. Now, I know its a PITA to tile & stream compressed formats because I've done it, but I'm sure I've written image I/O that decompresses the entire file to RAM 100x more frequently than anything that tiles and/or streams, because I've only handled tiling or streaming myself once, and it was harder. :P

Post reply on HN