Live data from Hacker News

Parallax effect from Google Lens Blur photos

depthy.stamina.pl

41–50 of 61 posts

Re: Parallax effect from Google Lens Blur photos

#41
post #5

Surprisingly well faked in face of the fact that there's no real 3D information in the pictures. There simply cannot be new 'pixels' appearing that were (i.e. before applying the effect) hidden from objects in the cameras line-of-sight. This is evident upon closer inspection (look at edges around the approximate middle of the DOF). The trompe-l'œil then falls a bit apart. Interestingly, I did not notice above with th…

It's exactly that - single image, and a depth map calculated from a series of shots made upwards. Both these are bundled in a fake-bokeh image. There is obviously no more pixels. However... If you choose your subject wisely, the effect can be pretty believeable, using a simple displacement map. On iOS though it's something different. There's no depth map in there, just a flat image moving counterwise to your hand mov…

It sounds like there are two issues here. One is how the depth map is generated, and the other is how the resulting image file is formatted. For the former, several still images are collected while the camera is moving, which provides parallax which can be used to generate the depth map. For the latter, I don't know, but it would certainly be possible to bundle both the depth map and multiple "layers" of photograph that could be used to actually expose new pixels in the background when the foreground moves.

Re: Parallax effect from Google Lens Blur photos

#46
post #32

Earlier quoted context omitted.

There already is a library for extracting the depth map here: https://github.com/spite/android-lens-blur-depth-extractor (internet rocks, isn't it ;) ) As for the Shader - it's just a few lines of actual code. Depthy is a quick weekend hack. There's a loong way before it.

Has anyone managed to replicate the lens blur effect that's being utilized in the new Android camera app? Or at least know what research paper it's based on?

Yes, here: http://jabtunes.com/labs/lensblur/index3.html and here: http://quasimondo.com/QuasimondoLibsJS/demos/DepthMapLensBlu...

Re: Parallax effect from Google Lens Blur photos

#47
post #41

Earlier quoted context omitted.

It's exactly that - single image, and a depth map calculated from a series of shots made upwards. Both these are bundled in a fake-bokeh image. There is obviously no more pixels. However... If you choose your subject wisely, the effect can be pretty believeable, using a simple displacement map. On iOS though it's something different. There's no depth map in there, just a flat image moving counterwise to your hand mov…

It sounds like there are two issues here. One is how the depth map is generated, and the other is how the resulting image file is formatted. For the former, several still images are collected while the camera is moving, which provides parallax which can be used to generate the depth map. For the latter, I don't know, but it would certainly be possible to bundle both the depth map and multiple "layers" of photograph t…

There is an app for iOS - seene.co. The amount of movements you have to do to capture enough pixels is prohibitive for my taste. I think that google has nailed it - it's super simple.

As for storing the layers - you would only have the "from above" pixels, and only a few. Probably that's why there is only a LensBlur in their app in the first place.

If you just want a small displacement effect like on depthy - then the key is no sharp edges in the depthmap. We will try to tackle this in one of the upcoming updates...

Re: Parallax effect from Google Lens Blur photos

#49
post #11

Nice work! Are you doing any 'content-aware fill' style magic on the invented occluded pixels? I wonder what else can be done with the depth data... Fog is the obvious one. You could potentially insert elements into the scene that knew when they were behind foreground items. I'm sure some kind of point-cloud or mesh could also be derived but not sure how good it would be. Funnily enough I nearly posted to /r/android/…

If you're interested in Google's metadata spec for what the camera actually captures:

https://developers.google.com/depthmap-metadata/

Post reply on HN