Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

3-Sweep: Extracting Editable Objects from a Single Photo [video]

youtube.com

Post reply on HN
Print view View original

11–20 of 75 posts

Re: 3-Sweep: Extracting Editable Objects from a Single Photo [video]

#11

Post by mwsherman »

The key here is really complementary use of ‘what humans are good at’ and ‘what machines are good at’.

In this case, it’s fair to say the machine, by analyzing pixels, can’t figure out perspective very well. The human can do that just fine, given an interface mechanism.

The machine is good at detecting edges and seeing similarity between pixels. Given hints from the human that ‘this point is within an object’ and here is the perspective, the machine can infer the limits of the object based on edges/colors and project it into 3 dimensions. Amazing.

Re: 3-Sweep: Extracting Editable Objects from a Single Photo [video]

#13

Post by jorgem »

jostmey wrote: post #12

I am skeptical, although I remain hopeful that my skepticism is misplaced. The "software" somehow seems to know what pattern of colors should exist on the other side of the object. Can someone explain to us how this aspect of the software works?

Looks like they just reverse the front.

Re: 3-Sweep: Extracting Editable Objects from a Single Photo [video]

#14

Post by prezjordan »

jostmey wrote: post #12

I am skeptical, although I remain hopeful that my skepticism is misplaced. The "software" somehow seems to know what pattern of colors should exist on the other side of the object. Can someone explain to us how this aspect of the software works?

Looks like the flip whatever is on the visible side. If you look at the underside of the telescope, it's just a repeated pattern of what was originally visible.

Re: 3-Sweep: Extracting Editable Objects from a Single Photo [video]

#16

Post by jack-r-abbit »

Also awesome is that it handles the background replacement so well. This could also be used to just remove an ugly lamp post, telephone pole, etc from an otherwise good photo. (assuming you can remove objects and resave the image)

Edit: I am aware that Photoshop has some of this available. I've not played with it so I don't know how they compare.

Re: 3-Sweep: Extracting Editable Objects from a Single Photo [video]

#19

Post by gohrt »

breckinloggins wrote: post #2

If you marked shadows and associated them with their source, could you then recover the light source(s) and be able to remove the baked shadows and recast them in real time? Also, with the shiny objects, could you specify the material properties and have it "back out" the reflection such that the reflection was recomputed as you moved the shape around?

Yes, there are other projects that do things like insert a synthetic object into a scene, with natural in-context lighting that is inferred from the light gradients on other objects in scene.

Re: 3-Sweep: Extracting Editable Objects from a Single Photo [video]

#20

Post by pwny »

jack-r-abbit wrote: post #16

Also awesome is that it handles the background replacement so well. This could also be used to just remove an ugly lamp post, telephone pole, etc from an otherwise good photo. (assuming you can remove objects and resave the image) Edit: I am aware that Photoshop has some of this available. I've not played with it so I don't know how they compare.

If you're just removing part of the image after cutting around it with a tool like this, having the object interpreted as 3D isn't really going to be of any benefit.

The impressive thing here, imho, is the seemingly effortless and seamless transition and replacement. The background is fixed and the surface texture is stretched in what seems like real time.

Post reply on HN