Is there anything that would prevent this approach working on 360 video?
i imagine a lot of unfortunate artefacts come out of stitching together the camera views that form a 360 or "spherical" image.
Atlas: End-to-End 3D Scene Reconstruction from Posed Images
11–20 of 26 posts
Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images
#12From the folks at Magic Leap. It looks remarkably good to me. The video at https://www.youtube.com/watch?v=9NOPcOGV6nU&feature=youtu.be is worth watching, especially the parts showing how the model gradually constructs and improves a labeled 3D mesh of a live room as it is fed more visual data by walking around the room. -- On a related note, Magic Leap has been trying to find a buyer for the business for several mon…
I have no experience in this field at all and they note on the video that the sequence shown was not realtime but I wonder how far we're from having something like this running in realtime or how "realtime" it could be given fancy hardware to be used in the wild?
Without much optimization, it can run at ~14fps on a NVidia TiTan RTX
Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images
#13I wonder how long it's going to be before we're able to run a significant portion of Youtube video (tourist videos, etc) through something like this, and generate a huge 3d mesh of the world. Combined with Street View data, you'd really have a ton of spaces covered.
Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images
#14Is there anything that would prevent this approach working on 360 video?
In theory this should work. I’ve been doing photogrammetry with spherical video and existing software packages often want to “dewarp” the image on to a plane, which works fine for narrow field of view but fails on spherical video. It would be interesting to see if atlas supports spherical input. Also 360 cameras have pretty low visual acuity. My 5.6k GoPro Fusion has to divide those pixels across the whole field of view, so images are less detailed. Still I think 360 video can be useful in photogrammetry with the right algorithms.
Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images
#15Is there anything that would prevent this approach working on 360 video?
i imagine a lot of unfortunate artefacts come out of stitching together the camera views that form a 360 or "spherical" image.
Well on a camera with dual fish eye lenses for 360 vision there’s some blurring at the edge where the images are merged together. But actually each camera separately just has normal fish eye effects, and if both images are used without blending them together you’d have minimal artifacts. Biggest issue is low visual acuity imo.
Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images
#16Is there anything that would prevent this approach working on 360 video?
Worst case, you can sample the 360 frames to get images with a smaller field of view. However, the app takes in camera intrinsics and positional data so it seems like it would work out of the box.
Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images
#17I wonder how long it's going to be before we're able to run a significant portion of Youtube video (tourist videos, etc) through something like this, and generate a huge 3d mesh of the world. Combined with Street View data, you'd really have a ton of spaces covered.
Google will do this, and then sell the data to security institutions. We will be told about it later, or consent to it during a Terms&Conditions update.
Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images
#18I wonder how long it's going to be before we're able to run a significant portion of Youtube video (tourist videos, etc) through something like this, and generate a huge 3d mesh of the world. Combined with Street View data, you'd really have a ton of spaces covered.
Cool idea, but how would you keep it maintained? It's tricky enough to keep maps up to date. A 3D Mesh would be even more complex to maintain.
Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images
#19Here's a challenge question to folks reading this and learned with the tools of the trade (my apologies in advance for somewhat hijacking the thread): consider this video of an endoscopy: https://www.youtube.com/watch?v=DUVDKoKSEkU -- say, from 3:00 to 5:00. And I have a bunch of movies (i.e., a series of images!) and I want to do a 3d reconstruction of this.
It seems super, super difficult... there are free-flowing liquids, and since this is an esophagus/upper lining of the stomach which is changing in form quite drastically so often. How would you guys approach this problem?
Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images
#20Looks awesome. Given it takes position data along with images, how accurate must the position data be? Could it handle something like sensor drift in the position data over time?