Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Atlas: End-to-End 3D Scene Reconstruction from Posed Images

github.com

Post reply on HN
Print view View original

21–26 of 26 posts

Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images

#21

Post by yorwba »

pen2l wrote: post #19

Here's a challenge question to folks reading this and learned with the tools of the trade (my apologies in advance for somewhat hijacking the thread): consider this video of an endoscopy: https://www.youtube.com/watch?v=DUVDKoKSEkU -- say, from 3:00 to 5:00. And I have a bunch of movies (i.e., a series of images!) and I want to do a 3d reconstruction of this. It seems super, super difficult... there are free-flowing…

You're not the first to come up with that challenge ;) https://endovis.grand-challenge.org/

Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images

#22

Post by pfranz »

tibbon wrote: post #13

I wonder how long it's going to be before we're able to run a significant portion of Youtube video (tourist videos, etc) through something like this, and generate a huge 3d mesh of the world. Combined with Street View data, you'd really have a ton of spaces covered.

I believe random videos are too low of a quality. Like this, most of the stuff I've seen uses constrained videos.

I have seen random still images used for this kind of thing: https://nerf-w.github.io/

I haven't heard of any equivalent of EXIF for video. That goes a long way when trying to make sense of random video both for camera settings as well as GPS location if you're trying to correlate multiple videos.

Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images

#23

Post by Theodores »

kanobo wrote: post #2

On a tangential thought, it's interesting to me that a company (magicleap) that has raised several billion dollars generates so little value compared to other companies its size that this is the most notable output from them in a year and I thought it was a phd project until I looked at the project owner. Anyways, it's a very interesting project and thanks for sharing.

It is easy to have a billion dollar company if you have borrowed two billion. And haven't spent all the loot on salaries.

The company has a cool name and the product area is divisive. Some say it is vapourware and nobody wants Oculus Rift style VR. Others are gung-ho. It's like Bitcoin all over again.

Although this tech is being done with AI, it was being done with non-AI approach two decades ago for movies/TV. But it wasn't as if people ported this tech to their smart phones from the SGI desktop monsters of yesteryear.

Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images

#24

Post by ghj »

pen2l wrote: post #19

Here's a challenge question to folks reading this and learned with the tools of the trade (my apologies in advance for somewhat hijacking the thread): consider this video of an endoscopy: https://www.youtube.com/watch?v=DUVDKoKSEkU -- say, from 3:00 to 5:00. And I have a bunch of movies (i.e., a series of images!) and I want to do a 3d reconstruction of this. It seems super, super difficult... there are free-flowing…

Even more hijacking, I remember thinking medical applications were going to be the killer apps for VR. I was blown away by these demos almost half a decade ago https://youtu.be/MWGBRsV9omw?t=251

Did they ever make it into real life practice?

Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images

#25

Post by Cactus2018 »

ghj wrote: post #24
pen2l wrote: post #19

Here's a challenge question to folks reading this and learned with the tools of the trade (my apologies in advance for somewhat hijacking the thread): consider this video of an endoscopy: https://www.youtube.com/watch?v=DUVDKoKSEkU -- say, from 3:00 to 5:00. And I have a bunch of movies (i.e., a series of images!) and I want to do a 3d reconstruction of this. It seems super, super difficult... there are free-flowing…

Even more hijacking, I remember thinking medical applications were going to be the killer apps for VR. I was blown away by these demos almost half a decade ago https://youtu.be/MWGBRsV9omw?t=251 Did they ever make it into real life practice?

Thanks for linking to Doc Ok's youtube channel!

https://www.youtube.com/c/okreylos/videos

5 years ago he was active in the Vive VR world

http://doc-ok.org/

Re: Atlas: End-to-End 3D Scene Reconstruction from Posed Images

#26

Post by tonyarkles »

pfranz wrote: post #22
tibbon wrote: post #13

I wonder how long it's going to be before we're able to run a significant portion of Youtube video (tourist videos, etc) through something like this, and generate a huge 3d mesh of the world. Combined with Street View data, you'd really have a ton of spaces covered.

I believe random videos are too low of a quality. Like this, most of the stuff I've seen uses constrained videos. I have seen random still images used for this kind of thing: https://nerf-w.github.io/ I haven't heard of any equivalent of EXIF for video. That goes a long way when trying to make sense of random video both for camera settings as well as GPS location if you're trying to correlate multiple videos.

GoPro has a proprietary format that stores live metadata in the videos if I recall. Maybe it’s called GPX? About 6 months ago I extracted GPS coordinates from a video using an open source tool.
Post reply on HN