Live data from Hacker News

SHARP, an approach to photorealistic view synthesis from a single image

apple.github.io

51–60 of 114 posts

Re: SHARP, an approach to photorealistic view synthesis from a single image

#51

Earlier quoted context omitted.

What are free objects?

The "free" in this case is a verb. The objects are freed from the background.

Until your comment I didn't realise I'd also read it wrong (despite getting the gist of it). Attempted rephrase of the original sentence:

Imagine history documentaries where they take an old photo, free objects from the background, and then move them round to give the illusion of parallax.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#53
post #13

See also Spaitial[0] which announced today full 3D environment generation from a single image [0] https://www.spaitial.ai/

The best I've seen so far is Marble from World Labs, though that gives you a full 360 environment and takes several minutes to do so.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#54
post #36

Earlier quoted context omitted.

From a single picture it infers a hidden 3D representation, from which you can produce photorealistic images from slightly different vantage points (novel views).

There's nothing "hidden" about the 3d represenation. It's a point cloud (in meters) with colors, and a guess at the the "camera" that produced it. (I am oversimplifying).

"Hidden" or "latent" in a context like this just means variables that the algo is trying to infer because it doesn't have direct access to them.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#55

Apple's Spatial Scene in the Photos app shows similar behavior, turning a single photo into a small 3D scene that you can view by tilting the phone. Demo here: https://files.catbox.moe/93w7rw.mov

It‘s awful and often creates a blurry mess in the imaginated space behind the object.

Photoshop content aware fill could do equally or better many years ago.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#56

Can someone ELI5 what this does? I read the abstract and tried to find differences in the provided examples, but I don't understand (and don't see) what the "photorealistic" part is.

Black Mirror episode portraying what this could do: https://youtu.be/XJIq_Dy--VA?t=14. If Apple ran SHARP on this photo and compared it to the show, that would be incredible.

Or if you prefer Blade Runner: https://youtu.be/qHepKd38pr0?t=107

Re: SHARP, an approach to photorealistic view synthesis from a single image

#57
post #51

Earlier quoted context omitted.

The "free" in this case is a verb. The objects are freed from the background.

Until your comment I didn't realise I'd also read it wrong (despite getting the gist of it). Attempted rephrase of the original sentence: Imagine history documentaries where they take an old photo, free objects from the background, and then move them round to give the illusion of parallax.

I'd suggest a different verb like "detach" or "unlink".

Re: SHARP, an approach to photorealistic view synthesis from a single image

#58
post #51

Earlier quoted context omitted.

The "free" in this case is a verb. The objects are freed from the background.

Until your comment I didn't realise I'd also read it wrong (despite getting the gist of it). Attempted rephrase of the original sentence: Imagine history documentaries where they take an old photo, free objects from the background, and then move them round to give the illusion of parallax.

> Imagine history documentaries where they take an old photo, free objects from the background

Even using commas, if you leave the ambiguous “free” I suggest you prefix “objects” with “the” or “any”.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#60
post #26

cuda gpu only https://github.com/apple/ml-sharp#rendering-trajectories-cud...

No, model works without CUDA then you have .ply that you can drop into gaussian splatter viewer like https://sparkjs.dev/examples/#editor

CUDA is needed to render side scrolling video, but there is many ways to do other things with result.

Post reply on HN