Live data from Hacker News

SHARP, an approach to photorealistic view synthesis from a single image

apple.github.io

91–100 of 114 posts

Re: SHARP, an approach to photorealistic view synthesis from a single image

#91
post #83

Well, I got _something_ to work on Apple Silicon: https://github.com/rcarmo/ml-sharp (has a little demo GIF) I am looking at ways to approximate Gaussian splats without having to reinvent the wheel, but I'm a bit over my depth since I haven't been playing a lot of attention to those in general.

I'm quite delighted that the gif banding artefacts make it look life the photi of a fire is flickering, and also highly impressed that the AI was able to recognize the fire as a photo within a photo and keep it in 2d.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#94
post #83

Well, I got _something_ to work on Apple Silicon: https://github.com/rcarmo/ml-sharp (has a little demo GIF) I am looking at ways to approximate Gaussian splats without having to reinvent the wheel, but I'm a bit over my depth since I haven't been playing a lot of attention to those in general.

The example doesn't look particularly impressive to say the least. Look at the bottom 20 %

I just refactored the rendering and resampling approach. Took me a few tries to figure out how to remove the banding masks from the layers, but with more stacked layers and a bit of GPT-foo to figure out the API it sort of works now (updated the GIF)

Keep in mind that this is not Gaussian splat rendering but just a hacked approximation--on my NVIDIA machine that looks way smoother.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#96

Earlier quoted context omitted.

They're using their Depth Pro model for depth estimation, and that seems to do faces really well. https://github.com/apple/ml-depth-pro https://learnopencv.com/depth-pro-monocular-metric-depth/

Im not sure how the depth estimation alone translates into the view synthesis, but the current implementation on-device is definitely not convincing for literally any portrait photographs I have seen. True stereoscopic captures are convincing statically, but don't provide the parallax.

Good monocular depth estimation is crucial if you want to make a 3D representation from a single image. Ordinarily you have images from several camera poses and can create the gaussian splats using triangulation, with a single image you have to guess z position for them.

Re: SHARP, an approach to photorealistic view synthesis from a single image

#97
post #90
post #84

Earlier quoted context omitted.

Goading companies into improving image and video generation by showing them how terrible they are is only going to make them go faster, and personally I’d like to enjoy the few moments I have left thinking that maybe something I watch is real. It will evolve into people hooked into entertainment suits most of the day, where no one has actual relationships or does anything of consequence, like some sad mashup of Wall-…

It turns out the Great Filter is that any species with the technology to colonize space also has the technology to soma itself into annihilation. https://en.wikipedia.org/wiki/Great_Filter

There are way ways past this, from religion and Amish-style cultural approaches, to legal prohibition of making and selling and using it, to dictatorial control of the companies which could make it, to individuals being personally immune, to paying people money if they don't use it. Like there are people who avoid alcohol, opioids, heroin, all other wireheading-style drugs and experiences that exist already, and people who do exercise and stay thin in a world of fast food and cars.

A great filter needs to apply to every civilisation imaginable, no exceptions, nerfing billions of species before they get to a higher Kardashev scale, not just something that "could happen" or the latest “Dunning-Kruger” mic-drop in every thread. In 1960s "the great filter is nuclear war", in 1890 "the great filter is heroin", in 1918 "the great filter is world war, we are destined to destroy ourselves", in 2015 "the great filter is climate change our emissions will end us like bacteria in a petri dish", in antiquity "the great filter is the punishment for crossing the will of the Gods".

It's got to be something you cannot get around even if you try really really hard and get very very lucky, because there are ~200,000,000,000 stars in the Milky Way and with those numbers there will be some species which lucks its way past almost any candidate, spreads out and in a mere 100k years is all over this galaxy leaving rocket trails and explosion signatures and radio signals and terraforming signs and megastructures.

Maybe when NASA, ESA, SpaceX, RosCOSMOS, CNSA, IRSA all collapse because of this effect… look how many countries have a space agency! https://en.wikipedia.org/wiki/List_of_government_space_agenc...

Re: SHARP, an approach to photorealistic view synthesis from a single image

#99

Can someone ELI5 what this does? I read the abstract and tried to find differences in the provided examples, but I don't understand (and don't see) what the "photorealistic" part is.

Apple does something similar right now in their photos app, generating spatial views from 2d photos, where parallax is visible by moving your phone. This paper’s technique seems to produce them faster. They also use this same tech in their Vision Pro headset to generate unique views per eye, likewise on spatialized images from Photos.
Post reply on HN