Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parallax effect from Google Lens Blur photos

depthy.stamina.pl

Post reply on HN
Print view View original

11–20 of 61 posts

Re: Parallax effect from Google Lens Blur photos

#11

Post by andybak »

Nice work! Are you doing any 'content-aware fill' style magic on the invented occluded pixels?

I wonder what else can be done with the depth data... Fog is the obvious one. You could potentially insert elements into the scene that knew when they were behind foreground items.

I'm sure some kind of point-cloud or mesh could also be derived but not sure how good it would be.

Funnily enough I nearly posted to /r/android/ earlier "I wish you could save the generated z-buffer" - it didn't occur to me to actually look!

Re: Parallax effect from Google Lens Blur photos

#12

Post by panrafal »

arscan wrote: post #8

It might be helpful to separate the code that creates the parallax effect from the angular app that displays the website & examples. This is a really cool trick and it would be nice to be able to easily reuse it. For those looking, it seems like the "magic" happens in a couple of spots (I think): https://github.com/panrafal/depthy/blob/master/app/scripts/p... and in https://github.com/panrafal/depthy/blob/master/app/…

There already is a library for extracting the depth map here: https://github.com/spite/android-lens-blur-depth-extractor (internet rocks, isn't it ;) )

As for the Shader - it's just a few lines of actual code.

Depthy is a quick weekend hack. There's a loong way before it.

Re: Parallax effect from Google Lens Blur photos

#13

Post by roeme »

panrafal wrote: post #10
roeme wrote: post #5

Surprisingly well faked in face of the fact that there's no real 3D information in the pictures. There simply cannot be new 'pixels' appearing that were (i.e. before applying the effect) hidden from objects in the cameras line-of-sight. This is evident upon closer inspection (look at edges around the approximate middle of the DOF). The trompe-l'œil then falls a bit apart. Interestingly, I did not notice above with th…

It's exactly that - single image, and a depth map calculated from a series of shots made upwards. Both these are bundled in a fake-bokeh image. There is obviously no more pixels. However... If you choose your subject wisely, the effect can be pretty believeable, using a simple displacement map. On iOS though it's something different. There's no depth map in there, just a flat image moving counterwise to your hand mov…

Now that you explained the IOS7 trick, it's obvious. (Given the environment: background filling image on a physical device movement as opposed to a image on a website).

Re: Parallax effect from Google Lens Blur photos

#14

Post by panrafal »

andybak wrote: post #11

Nice work! Are you doing any 'content-aware fill' style magic on the invented occluded pixels? I wonder what else can be done with the depth data... Fog is the obvious one. You could potentially insert elements into the scene that knew when they were behind foreground items. I'm sure some kind of point-cloud or mesh could also be derived but not sure how good it would be. Funnily enough I nearly posted to /r/android/…

Fog, refocus, background separation, chroma shifting.

As for the fill, if the depthmap is of good quality and without sharp edges - there is no need for that, unless you go berzerk with the scale of displacement.

Re: Parallax effect from Google Lens Blur photos

#15

Post by panrafal »

roeme wrote: post #13
panrafal wrote: post #10

Earlier quoted context omitted.

It's exactly that - single image, and a depth map calculated from a series of shots made upwards. Both these are bundled in a fake-bokeh image. There is obviously no more pixels. However... If you choose your subject wisely, the effect can be pretty believeable, using a simple displacement map. On iOS though it's something different. There's no depth map in there, just a flat image moving counterwise to your hand mov…

Now that you explained the IOS7 trick, it's obvious. (Given the environment: background filling image on a physical device movement as opposed to a image on a website).

On iOS you have the icons as a first plane - that makes the trick.

If you have an Android device around, try depthy on it - it's way better this way imo.

Re: Parallax effect from Google Lens Blur photos

#16

Post by ctdonath »

crazygringo wrote: post #7
roeme wrote: post #5

Surprisingly well faked in face of the fact that there's no real 3D information in the pictures. There simply cannot be new 'pixels' appearing that were (i.e. before applying the effect) hidden from objects in the cameras line-of-sight. This is evident upon closer inspection (look at edges around the approximate middle of the DOF). The trompe-l'œil then falls a bit apart. Interestingly, I did not notice above with th…

Indeed -- there are a bunch of spots where, if you look closely, you can see the foreground texture being "repeated" in the background texture, when you've moved your cursor to the edge. I wonder if a better kind of pixel-filling-in algorithm could make this more believable, or if the problem is with the inherent inaccuracies of the depth map. But still, as a whole it's remarkable effective. I'd like to be able to ap…

I'd like to be able to appreciate it without moving my mouse

Oculus Rift?

Not just different left/right angles for each eye, but also can rotate the angles by tilting head. That would be a spectacular way to view still photos in VR.

Re: Parallax effect from Google Lens Blur photos

#17

Post by aleyan »

Well done. I have been working with animating 2d images with a hand constructed depth map in the past to a decent 3d effect[1]. The illusion when guided by your mouse as in the OP seems to be much stronger then in left-right strafing that I did.

One point that can be worked on, as pointed out by other people here is the occluded pixels (this is an issue when an image has sharp changes in distance from the camera). You simply can't show what wasn't seen by the camera, so you have to fake it. In the pre-baked animations I did, the occluded pixels were duplicated and then manually fixed by me in Photoshop.

Perhaps the solution here would be to capture and store images from several perspectives. These could then be used to generate the depthmap using what algorithms Lens Blur used, and also to interpolate the occluded pixels when viewing the photo from different perspectives.

[1] http://fooladder.tumblr.com/

Re: Parallax effect from Google Lens Blur photos

#18

Post by kordless »

ctdonath wrote: post #16
crazygringo wrote: post #7

Earlier quoted context omitted.

Indeed -- there are a bunch of spots where, if you look closely, you can see the foreground texture being "repeated" in the background texture, when you've moved your cursor to the edge. I wonder if a better kind of pixel-filling-in algorithm could make this more believable, or if the problem is with the inherent inaccuracies of the depth map. But still, as a whole it's remarkable effective. I'd like to be able to ap…

I'd like to be able to appreciate it without moving my mouse Oculus Rift? Not just different left/right angles for each eye, but also can rotate the angles by tilting head. That would be a spectacular way to view still photos in VR.

Given it only takes panning the image slightly and adjusting the parallax a bit, I think that's probably doable. It would be the modern day version of this: http://en.wikipedia.org/wiki/Stereoscopy
Post reply on HN