Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

techcrunch.com

Post reply on HN
Print view View original

11–20 of 262 posts

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#11

Post by revelation »

DCT is already lossy [1], so the statements around 8 megapixels are completely pointless, and worst of all, its 1990 lossy technology. Wavelet transformations completely destroy any DCT.

That said, if their emphasis is on producing pictures with minimal time delta at highest resolution, algorithms used for still pictures are out of place. Video compression algorithms still use DCT and wavelets, but they do so only after they have reduced redundancies between series of pictures, a process that tends to work significantly better than anything you can get out of these lossy transformations when you want to preserve quality.

Of course, eliminating redundancy in a series of pictures might have tipped them off to the fact that the image sensor isn't actually producing fresh pictures at the rate they want.

1: as used in JPEG. The transformation itself is perfectly invertible, assuming infinite precision arithmetic.

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#12

Post by jpap »

MikeTLive wrote: post #10

At 20fps, could you make a 3d camera app by the user moving their camera in space and then correcting for stabilization with the accelerometers etc telling you point in space and using the multiple view points as individual cameras.

That's a really interesting machine vision problem and a lot more complex than a JPEG codec. :-)

I wonder how long before we start to see Kinect-like infrared cameras mounted on phones to make the depth problem easier to solve. That would be cool!

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#13

Post by 0x09 »

jpap wrote: post #5
wmf wrote: post #3

Earlier quoted context omitted.

Yeah, I'm kinda skeptical of the "science" here. Edit: A new algorithm counts as science, but the TechCrunch article really gave no justification for the claim.

I've given a bit more background to the fast JPEG codec on my engineering blog: http://www.snappylabs.com/blog/snappycam/2013/07/31/iphone-k... If you like signal processing, fixed point arithmetic, SIMD cores, and assembly, then this is for you. :-)

So the summary is "JPEG encoder written in assembly with NEON instructions saves images faster than Apple's encoder."

That's a cool feat and is a little damning for Accelerate.framework, although the way techcrunch writes it I expected a new kind of fast cosine transform.

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#14

Post by Strilanc »

oakwhiz wrote: post #8
Dylan16807 wrote: post #2

Wow, I never thought I'd see a software optimization be talked about in such breathless amazement.

"discrete cosine transform JPG science" Here's a more interesting link directly from the app developers: http://www.snappylabs.com/blog/snappycam/2013/07/31/iphone-k...

Thanks, that's a much better article.

> extended some of that research to create a new algorithm

> 10,000 lines of hand-tuned assembly code

> optimized out pipeline bubbles using a cycle counter tool

Color me impressed. It sounds like they really pulled out all the stops.

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#15

Post by femto »

wmf wrote: post #3
Dylan16807 wrote: post #2

Wow, I never thought I'd see a software optimization be talked about in such breathless amazement.

Yeah, I'm kinda skeptical of the "science" here. Edit: A new algorithm counts as science, but the TechCrunch article really gave no justification for the claim.

Having shared a few beers with John Papandriopoulos (at an AusCTW workshop), I can vouch that he is capable of doing great things in signal processing. He's a smart guy [1].

G'day from across the ditch John and I'm glad to see things are going well! (from John D. in Sydney)

[1] http://www.rmit.edu.au/browse/Current%20students%2FAdmin%20e...

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#16

Post by jpap »

revelation wrote: post #11

DCT is already lossy [1], so the statements around 8 megapixels are completely pointless, and worst of all, its 1990 lossy technology. Wavelet transformations completely destroy any DCT. That said, if their emphasis is on producing pictures with minimal time delta at highest resolution, algorithms used for still pictures are out of place. Video compression algorithms still use DCT and wavelets, but they do so only af…

You are right on the loss: it's purposefully introduced as a quantization step after performing the DCT, and before losslessly compressing the resulting coefficients with Huffman and encoding to the final JPEG bitstream.

Despite all of that, JPEG has now become computationally tractable. I remember the days where it took tens of seconds to encode a JPEG on a commodity machine. Now, with the help of SIMD, we can encode a high quality image in msec on a mobile device.

Fortunately you can choose the quantization matrix that determines the amount of loss. Even if you were to choose a unitary matrix, no human, not even superman with his laser eyes, can "detect" the quantization noise.

For SnappyCam, I chose to invest in JPEG a little more because it's a ubiquitous standard for still image compression.... and with the right hardware and algorithms, quite tractable.

I'll consider adding a JPEG "quality setting" so you can choose the amount of loss introduced... sounds like a great idea to me.

The idea behind SnappyCam was also to code each picture independently, and not rely on motion prediction or video codecs. If you try and pull a single frame from a HD video you might be disappointed: they compress the YUV dynamic range (studio swing) and it looks washed out, even if you land on an i-frame.

Lastly, as far as I can tell, the image sensor is yielding complete scans with each frame. I'd hazard a guess to say that any motion prediction or frame deltas might actually slow the whole chain down.

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#17

Post by aerolite »

wmf wrote: post #3
Dylan16807 wrote: post #2

Wow, I never thought I'd see a software optimization be talked about in such breathless amazement.

Yeah, I'm kinda skeptical of the "science" here. Edit: A new algorithm counts as science, but the TechCrunch article really gave no justification for the claim.

How come it doesn't let me downvote you?

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#18

Post by jpap »

femto wrote: post #15
wmf wrote: post #3

Earlier quoted context omitted.

Yeah, I'm kinda skeptical of the "science" here. Edit: A new algorithm counts as science, but the TechCrunch article really gave no justification for the claim.

Having shared a few beers with John Papandriopoulos (at an AusCTW workshop), I can vouch that he is capable of doing great things in signal processing. He's a smart guy [1]. G'day from across the ditch John and I'm glad to see things are going well! (from John D. in Sydney) [1] http://www.rmit.edu.au/browse/Current%20students%2FAdmin%20e...

Hey, thanks!

The Australian Communications Theory Workshop was such a long time ago--what great memories. :-)

What are you up to these days?

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#19

Post by beambot »

MikeTLive wrote: post #10

At 20fps, could you make a 3d camera app by the user moving their camera in space and then correcting for stabilization with the accelerometers etc telling you point in space and using the multiple view points as individual cameras.

Or you could just use individual frames and bundle adjustment.... even via free online software like 123D Catch. ;) http://www.123dapp.com/catch
Post reply on HN