Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SHARP, an approach to photorealistic view synthesis from a single image

apple.github.io

Post reply on HN
Print view View original

61–70 of 114 posts

Re: SHARP, an approach to photorealistic view synthesis from a single image

#61

Post by drcongo »

Traubenfuchs wrote: post #52
superfish wrote: post #14

"Unsplash > Gen3C > The fly video" is nightmare fuel. View at your own risk: https://apple.github.io/ml-sharp/video_selections/Unsplash/g...

Early AI „everything turns into dog heads“ vibes. Beautiful.

I miss those. Anyone know if it's still possible to get the models etc. needed to generate them?

Re: SHARP, an approach to photorealistic view synthesis from a single image

#62

Post by Dumbledumb »

In Chapter D.7 they describe: "The complex reflection in water is interpreted by the network as a distant mountain, therefore the water surface is broken."

This is really interesting to me because the model would have to encode the reflection as both the depth of the reflecting surface (for texture, scattering etc) as well as the "real depth" of the reflected object. The examples in Figure 11 and 12 already look amazing.

Long tail problems indeed.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#63

Post by Traubenfuchs »

drcongo wrote: post #61
Traubenfuchs wrote: post #52

Earlier quoted context omitted.

Early AI „everything turns into dog heads“ vibes. Beautiful.

I miss those. Anyone know if it's still possible to get the models etc. needed to generate them?

I wish there was an archive of all those melty dreamscapes.

https://m.youtube.com/watch?v=DgPaCWJL7XI&t=1s&pp=2AEBkAIB0g...

https://www.youtube.com/watch?v=X0oSKFUnEXc

Re: SHARP, an approach to photorealistic view synthesis from a single image

#65

Post by necovek »

arjie wrote: post #8

This is incredibly cool. It's interesting how it fails in the section where you need to in-paint. SVC seems to do that better than all the rest, though not anywhere close to the photorealism of this model. Is there a similar flow but to transform either a video/photo/NeRF of a scene into a tighter, minimal polygon approximation of it. The reason I ask is that it would make some things really cool. To make my baby mon…

You'd still need one real measurement at least: this might get proportions right if background can be clearly separated, but the absolute size of an object can be worlds apart.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#68

Post by jijijijij »

accurrent wrote: post #7
calvinmorrison wrote: post #4

I understand AI for reasoning, knowledge, etc. I haven't figured out how anyone wants to spend money for this visual and video stuff. It just seems like a bad idea.

Simulation. It takes a lot of effort today to bring up simulations in various fields. 3 D programming is very nontrivial and asset development is extremely expensive. If I have a workspace I can take a photo of and just use it to generate a 3d scene I can then use it in simulations to test ideas out. This is particularly useful in robotics and industrial automation already.

I don't see any examples of a 3D scene information usable for simulation. If you want to simulate something hitting a table, you need the whole table (surface) in space, not just some spatial illusion effect extrapolated from an image of a table. I also think modelling the 3D objects for simulation is the least expensive part of an simulation... the simulation is the expensive thing.

I doubt this will be useful for robotics or industrial automation, where you need an actual spatial, or functional understanding of the object/environment.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#70

Post by zipy124 »

Leptonmaniac wrote: post #27

Can someone ELI5 what this does? I read the abstract and tried to find differences in the provided examples, but I don't understand (and don't see) what the "photorealistic" part is.

Basically depth estimation to split the scene into various planes, and then inpainting to work out the areas in the obscured parts of the planes, and then the free movement of them to allow for parallax. Think of 2D side scrolling games that have various different background depths to give illusion of motion and depth.
Post reply on HN