Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SHARP, an approach to photorealistic view synthesis from a single image

apple.github.io

Post reply on HN
Print view View original

31–40 of 114 posts

Re: SHARP, an approach to photorealistic view synthesis from a single image

#31

Post by ares623 »

Leptonmaniac wrote: post #27

Can someone ELI5 what this does? I read the abstract and tried to find differences in the provided examples, but I don't understand (and don't see) what the "photorealistic" part is.

Takes a 2D image and allows you to simulate moving the angle of the camera with correct-ish parallax effect and proper subject isolation (seems to be able to handle multiple subjects in the same scene as well)

I guess this is what they use for the portrait mode effects.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#33

Post by p-e-w »

Leptonmaniac wrote: post #27

Can someone ELI5 what this does? I read the abstract and tried to find differences in the provided examples, but I don't understand (and don't see) what the "photorealistic" part is.

Agreed, this is a terrible presentation. The paper abstract is bordering on word salad, the demo images are meaningless and don’t show any clear difference to the previous SotA, the introduction talks about “nearby” views while the images appear to show zooming in, etc.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#34

Post by emsign »

Leptonmaniac wrote: post #27

Can someone ELI5 what this does? I read the abstract and tried to find differences in the provided examples, but I don't understand (and don't see) what the "photorealistic" part is.

Imagine history documentaries where they take an old photo and free objects from the background and move them round giving the illusion of parallax movement. This software does that in less than a second, creating a 3D model that can be accurately moved (or the camera for that matter) in your video editor. It's not new, but this one is fast and "sharp".

Gaussian splashing is pretty awesome.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#35

Post by wfme »

harhargange wrote: post #19

TMPI looks just as good if not better.

Have a look through the rest of the images. TMPI has some pretty obvious shortcomings in a lot of them.

1. Sky looks jank 2. Blurry/warped behind the horse 3. The head seems to move a lot more than the body. You could argue that this one is desirable 4. Bit of warping and ghosting around the edges of the flowers. Particularly noticeable towards the top of the image. 5. Very minor but the flowers move as if they aren't attached to the wall

Re: SHARP, an approach to photorealistic view synthesis from a single image

#36

Post by avaer »

eloisius wrote: post #30
Leptonmaniac wrote: post #27

Can someone ELI5 what this does? I read the abstract and tried to find differences in the provided examples, but I don't understand (and don't see) what the "photorealistic" part is.

From a single picture it infers a hidden 3D representation, from which you can produce photorealistic images from slightly different vantage points (novel views).

There's nothing "hidden" about the 3d represenation. It's a point cloud (in meters) with colors, and a guess at the the "camera" that produced it.

(I am oversimplifying).

Re: SHARP, an approach to photorealistic view synthesis from a single image

#37

Post by eloisius »

avaer wrote: post #36
eloisius wrote: post #30

Earlier quoted context omitted.

From a single picture it infers a hidden 3D representation, from which you can produce photorealistic images from slightly different vantage points (novel views).

There's nothing "hidden" about the 3d represenation. It's a point cloud (in meters) with colors, and a guess at the the "camera" that produced it. (I am oversimplifying).

Hidden in the sense of neural net layers. I mean intermediary representation.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#38

Post by shwaj »

jjcm wrote: post #21
harhargange wrote: post #19

TMPI looks just as good if not better.

Disagree - look at the sky in the seaweed shot. It doesn't quite get the depth right in anything, and the edges of things look off.

Agreed. The head of the fly also seems to have weird depth.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#39

Post by avaer »

eloisius wrote: post #37
avaer wrote: post #36

Earlier quoted context omitted.

There's nothing "hidden" about the 3d represenation. It's a point cloud (in meters) with colors, and a guess at the the "camera" that produced it. (I am oversimplifying).

Hidden in the sense of neural net layers. I mean intermediary representation.

Right.

I just want to emphasize that this is not a NERF where the model magically produces an image from an angle and then you ask "ok but how did you get this?" and it throws up its hands and says "I dunno, I ran some math and I got this image" :D.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#40

Post by rv3392 »

calvinmorrison wrote: post #4

I understand AI for reasoning, knowledge, etc. I haven't figured out how anyone wants to spend money for this visual and video stuff. It just seems like a bad idea.

This specific paper is pretty different to the kind of photo/video generation that has been hyped up in recent years. In this case, I think this might be what they're using for the iOS spatial wallpaper feature, which is arguably useless but is definitely an aesthetic differentiator to Android devices. So, it's indirectly making money.
Post reply on HN