Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image
worldsheet.github.io
Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image
1–10 of 34 posts
Re: Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image
#2I'm wondering when we'll get to easy Blade Runner tech, where taking a photo will let you browse it in a 3d sense, extrapolating to some level of certainty what certain aspects "behind" objects or out of view would like look, based off the analysis of the visible elements in the photo.
Re: Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image
#3I'm wondering when we'll get to easy Blade Runner tech, where taking a photo will let you browse it in a 3d sense, extrapolating to some level of certainty what certain aspects "behind" objects or out of view would like look, based off the analysis of the visible elements in the photo.
In lab settings, it seems very close ;)
Detecting Invisible People
Re: Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image
#4Amazing work, well done!
Re: Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image
#5I'm wondering when we'll get to easy Blade Runner tech, where taking a photo will let you browse it in a 3d sense, extrapolating to some level of certainty what certain aspects "behind" objects or out of view would like look, based off the analysis of the visible elements in the photo.
In lab settings, it seems very close ;) Detecting Invisible People http://www.cs.cmu.edu/~tkhurana/invisible.htm
That's not the same thing. This paper is talking about extrapolating objects that become occluded in the video stream until they become visible again, to provide a continuous sense of tracking.
Re: Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image
#6I wonder how long it takes to generate these images. It's not terribly hard to do by hand with rudimentary 3D modeling skills (i.e the most generous way to describe my own 3D modeling capability), if you know the height from the floor and shooting angle at which the camera took the picture. I've done it with 360 photos (therefore shooting angle didn't matter) in Blender with a VR headset to aid in positioning vertices. You can get results like this in about 15 minutes.
Complex scenes obviously take longer, but results can be extremely convincing for a limited range of view angles.
I learned the technique from this video: https://youtu.be/RSC40B8kHT8
Re: Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image
#7Could you combine many of these images together to make a 3D environment for a computer game, for example inside the bedroom take many photos and then interpolate between the 3D sheets as the player moves around?
Re: Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image
#8Could you combine many of these images together to make a 3D environment for a computer game, for example inside the bedroom take many photos and then interpolate between the 3D sheets as the player moves around?
That is basically what photogrammetry is all about. It's not easy, but it can work really well when done well.
Re: Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image
#9Could this be a camera app on the latest iPhones with the built in lidar like technology? You’d snap an image and a mesh, right?
Re: Worldsheet: Wrapping the World in a 3D Sheet, View Synthesis from a Single Image
#10In the future you could train a model to predict the correct mesh for a given image. Then things get interesting!