Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

techcrunch.com

Post reply on HN
Print view View original

41–50 of 262 posts

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#41

Post by Andrenid »

jpap wrote: post #32
dvt wrote: post #30

Why not have a deferred compressor? I assume that just straight-up saving the raw data in memory would be much faster than compressing every frame as you get it. Couldn't you get significant FPS increases (given that you still had free space/memory available)?

Actually, I do both on dual core devices. One core is dedicated to host the capture/buffer, the other will encode shots in the background. When you see the big circle percent animation, both cores are dedicated to compression to clear the encoder queue so you can take back to back living photos quickly.

I've just gotta say:

This is one of the main reasons I keep coming back to HN. A story gets posted about some cool new tech, and the creator is in the comments answering questions. Simply awesome.

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#42

Post by fluidcruft »

jpap wrote: post #9
peter_l_downs wrote: post #6

Any chance of this coming to Android soonish? This is seriously cool!

The fast JPEG codec was written for the ARM NEON SIMD coprocessor found in the iPhone. Most Android devices also sport the same architecture, so it is indeed possible. The code for the codec is written in mixed C and assembly, so it can be "easily" ported to Android by making use the JNI. While the R&D for the fast JPEG codec took about a year to perfect, the iOS app took just about the same time to get polished (inc…

Don't overlook the fact that the source for the stock Android camera is available under a commercial-use-friendly open source license and has a quite nice native android UI. You don't have to reinvent all the wheels unless you're stubborn.

https://android.googlesource.com/platform/packages/apps/Came...

I would buy that in a heartbeat.

Unrelated, how quickly can you alter exposure settings? Can you get 30 pictures per second with three interleaved exposure brackets? (i.e. burst of 10 HDR photos / second) That would be very, very, very, very cool.

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#43

Post by jasomill »

dvt wrote: post #30

Why not have a deferred compressor? I assume that just straight-up saving the raw data in memory would be much faster than compressing every frame as you get it. Couldn't you get significant FPS increases (given that you still had free space/memory available)?

Assuming approximately 8MB worth of uncompressed raw data from the image sensor, 20 frames per second would require writing images to flash storage at 160MB/sec, which no iPhone can do.

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#44

Post by jpap »

Andrenid wrote: post #41
jpap wrote: post #32

Earlier quoted context omitted.

Actually, I do both on dual core devices. One core is dedicated to host the capture/buffer, the other will encode shots in the background. When you see the big circle percent animation, both cores are dedicated to compression to clear the encoder queue so you can take back to back living photos quickly.

I've just gotta say: This is one of the main reasons I keep coming back to HN. A story gets posted about some cool new tech, and the creator is in the comments answering questions. Simply awesome.

haha, cool. :-)

To be honest, I don't often post here because I'm busy working, but am enjoying the discussion on a baby I've nursed for two years now. :-) Thanks for your post!

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#46

Post by dvt »

jasomill wrote: post #43
dvt wrote: post #30

Why not have a deferred compressor? I assume that just straight-up saving the raw data in memory would be much faster than compressing every frame as you get it. Couldn't you get significant FPS increases (given that you still had free space/memory available)?

Assuming approximately 8MB worth of uncompressed raw data from the image sensor, 20 frames per second would require writing images to flash storage at 160MB/sec, which no iPhone can do.

But writing to RAM (at 3200+MB/s) is certainly possible. You could cache around 2-3 seconds (on lower-end iPhones) as you compress. Again, the idea wouldn't be that you could indefinitely do this, but merely that the compressor would be deferred (and would lag at a ratio that would still yield x FPS).

That's how I would do it. But apparently they already do this :)

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#47

Post by jpap »

damian2000 wrote: post #37

I'm interested to know how their method compares to how dedicated digital cameras and DSLRs do it? are cameras running dedicated hardware/firmware to achieve the same result? Or have they optimised their software in the same way that SnappyCam has done it?

I can't say, as SnappyCam is my first foray into image signal processing. (Though DSP isn't new to me.)

I'd guess that DSLRs use a combination of hardware acceleration on the "tricky" bits (like DCT) with firmware to control the compute hardware.

Huffman is a particularly difficult beast, as it can't be parallelized. The JPEG bitstream is inherently serial, though there has been some proposals to improve that.

If you run a SnappyCam JPEG that you pluck from iTunes File Sharing through djpeg (from libJpeg) you will notice that each of the YCbCr planes are not interleaved.

I once experimented with a parallel JPEG encoder, encoding the Y, Cb, and Cr planes in parallel but the threading overhead was more than just queuing up each JPEG encode separately in a multithread queue.

Bonus points if you notice another marker in the JPEG. That's intended for parallel JPEG decoding but hasn't yet been implemented in SnappyCam as yet. (The existing decoder is fast enough for 8Mpx shots.)

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#48

Post by jpap »

mappu wrote: post #45

That's fantastic, and a very cool demo. How does the encoder performance compare to libjpeg-turbo? That also has some SIMD work for NEON.

Yes, Nokia contributed the NEON code for the DCT in libjpeg-turbo.

I haven't had a chance to do a side-by-side comparison as yet, but I suspect the SnappyCam encoder is faster for many reasons, including choice of algorithm and the way they use two multiplies (low, high) at times, and their image row-by-row nature with function call overhead in favour of code maintainability.

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#49

Post by jpap »

dvt wrote: post #46
jasomill wrote: post #43

Earlier quoted context omitted.

Assuming approximately 8MB worth of uncompressed raw data from the image sensor, 20 frames per second would require writing images to flash storage at 160MB/sec, which no iPhone can do.

But writing to RAM (at 3200+MB/s) is certainly possible. You could cache around 2-3 seconds (on lower-end iPhones) as you compress. Again, the idea wouldn't be that you could indefinitely do this, but merely that the compressor would be deferred (and would lag at a ratio that would still yield x FPS). That's how I would do it. But apparently they already do this :)

That's how it's done, where on dual core devices, a JPEG encode can be done in parallel to capture.

Try it on an iPhone 5 with "infinite shutter" disabled and you will see the dropped frames in the filenames once you import them to your machine from iTunes App File Sharing (or over SSH if you're jailbroken).

Re: Scientific Breakthrough Lets SnappyCam App Take 20 Full-Res Photos Per Second

#50

Post by marcamillion »

nwh wrote: post #31

Looks like they've an instagram-type site set up too. http://snappyc.am/2LdRMF28U0 http://snappyc.am/4HHxyCad7D http://snappyc.am/3G3i6QCJUk

Do these work for you? When I load the site, I see the photo scrub on the right hand side of the screen - and the slider moves up and down...but I don't see the big image loaded in the center of the screen where I expect the image to be.

Unless it takes a while to load - in which case I was just being uber impatient.

Post reply on HN