Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

3-D Depth Reconstruction from a Single Still Image (2007) [pdf]

cs.cornell.edu

Post reply on HN
Print view View original

11–20 of 30 posts

Re: 3-D Depth Reconstruction from a Single Still Image (2007) [pdf]

#11

Post by pen2l »

jmcmahon443 wrote: post #8
pen2l wrote: post #6

I wish these groups would make their code available so others could play with it and test it out. Some do, but I wish more did. I would be curious to see the performance of this 3d depth reconstruction technique for non-rigid environments.

I love open source, but there is a very good reason why academics generally don't release their code. They are scientists: they are best at testing, explaining, and documenting nature. Code is an engineer's job: making it as fast and robust as possible for the given application or with the latest tools. They want to make you work for it :)

With all due respect, that is a stupid reason.

I don't have time to implement every last thing. And I don't require robust code all the time -- it's very understandable for the code to not be robust if it's being provided by a small group with not a lot of funding or resources. Hell, if they make it open source I will contribute to it expecting nothing in return to make it better in any way I can.

I really think that the reason the code isn't shared is journals don't encourage this type of thing (or make it easy). Lab websites live and die, so hosting it there is not a good idea. Very simply, the code should should be packed with the paper somehow. I hope some journal takes the lead soon in creating a system that encourages a bundled presentation of paper + code.

Re: 3-D Depth Reconstruction from a Single Still Image (2007) [pdf]

#12

Post by TheArcane »

GrantS wrote: post #4

This is some very cool research, though many may be surprised to learn that it is an IJCV journal article from 2007, based on a conference paper from NIPS 2005. Source: http://www.cs.cornell.edu/%7Easaxena/learningdepth/

Gah! I thought this was cutting edge. Is this research used in any recent works/research?

Re: 3-D Depth Reconstruction from a Single Still Image (2007) [pdf]

#13

Post by leeoniya »

pen2l wrote: post #7
jvhaarst wrote: post #5

Video is a nice watch, especially as this is from 2007, so the state of the art should be even better. https://www.youtube.com/watch?v=UZ7_ED9g4FY

Google Tango ( https://www.google.com/atap/project-tango/ ) is able to do pretty impressive 3d reconstruction from what I hear. However it has a number of cameras, whilst linked technique can work with just one image (with enough training).

reconstructing depth from 2+ offset images is vastly simpler than from a single frame.

Re: 3-D Depth Reconstruction from a Single Still Image (2007) [pdf]

#14

Post by rboyd »

jmcmahon443 wrote: post #10
rboyd wrote: post #9

Earlier quoted context omitted.

Yeah! Having spent much of the last year plowing through computer vision papers, it's pretty rare to encounter published code or even datasets. Why is this? Do people spend all their time polishing prose and neglecting the code? Or maybe they keep it closed for commercial intentions? Or is it just a cultural thing in all of academia?

Science = Nature. Code = Applying Nature.

This would be a lot easier to jibe with if the paper was based on proof alone. A lot of the work in this realm isn't that. Plenty of these researchers are also fine engineers and I'm sure have great code. It just seems crazy to develop a new technique so completely and stop short of releasing the code.

I think it's because the academic culture encourages this as if the code detracts from the purity of the research.

Re: 3-D Depth Reconstruction from a Single Still Image (2007) [pdf]

#15

Post by derf_ »

pen2l wrote: post #6

I wish these groups would make their code available so others could play with it and test it out. Some do, but I wish more did. I would be curious to see the performance of this 3d depth reconstruction technique for non-rigid environments.

I re-implemented the techniques described here for a former employer (which sadly means I also cannot release the code). I was able to reproduce the quoted performance figures (both accuracy according to the paper's metrics and execution time), but the actual quality of the output was not very good.

Something like > 90% (number hazy based on vague recollection) of images produced results that had gross, glaring errors that would be unacceptable to a human. Errors like, "We added a horizon line where there wasn't one in the actual image," etc. But a few images produced compellingly plausible results.

It's still very interesting stuff, and some of his later work on using this as an additional prior for more normal structure-from-motion pipelines looked pretty promising. In the form reported in this paper it wasn't something ready to be productized. Although it's also completely possible that I just screwed something up.

Re: 3-D Depth Reconstruction from a Single Still Image (2007) [pdf]

#16

Post by Xcelerate »

rboyd wrote: post #9
pen2l wrote: post #6

I wish these groups would make their code available so others could play with it and test it out. Some do, but I wish more did. I would be curious to see the performance of this 3d depth reconstruction technique for non-rigid environments.

Yeah! Having spent much of the last year plowing through computer vision papers, it's pretty rare to encounter published code or even datasets. Why is this? Do people spend all their time polishing prose and neglecting the code? Or maybe they keep it closed for commercial intentions? Or is it just a cultural thing in all of academia?

Speaking from experience, the code is normally functional but unsightly, thus it would be embarrassing to place it online. Most researchers are happy to email you the code if you ask for it though.

Most of my code is for scientific computing and consists of a bunch of one-off Jupyter notebooks and Julia scripts that are written with two objectives in mind: get-the-task-done-before-my-advisor-meeting and make-it-as-fast-as-possible. Good coding practices and code cleanliness are not priorities. And particularly for code that needs to run on HPC clusters or supercomputers, it's really hard to make it look nice — one could easily spend weeks designing a proper, modular system only to throw it all away in favor of using a better algorithm that was just published. Anecdotally, the vast majority of my code has produced results that were later discarded.

For example, I have a function named YPXt. What does such a woefully named subroutine do? It multiplies a matrix Y by a permutation matrix P, and then multiplies that by another (transposed) matrix X. The matrices are small, so the overhead of using BLAS for the task is wasteful. It's better to use Julia's macro system to generate the necessary for loops on the fly and let the compiler SIMD-ify the result. Also, permutation matrix multiplies can be done in a way that doesn't actually involve creating a full permutation matrix. And of course the subroutine has side effects — in order to conserve memory I need to overwrite the input matrices instead of allocating new ones. Immutable data structures may work great for functional programming, but the overhead of red-black trees and stateless programming is too high for numerical data crunching.

Add that to the fact that a lot of scientists in academia are unfamiliar with tools like Git, and so manually versioning the code and emailing it back and forth is unfortunately a necessary part of the collaboration process.

Re: 3-D Depth Reconstruction from a Single Still Image (2007) [pdf]

#17

Post by pen2l »

leeoniya wrote: post #13
pen2l wrote: post #7

Earlier quoted context omitted.

Google Tango ( https://www.google.com/atap/project-tango/ ) is able to do pretty impressive 3d reconstruction from what I hear. However it has a number of cameras, whilst linked technique can work with just one image (with enough training).

reconstructing depth from 2+ offset images is vastly simpler than from a single frame.

Yup, but I think there's a desire to get good 3d-reconstruction techniques for single images because in practice you often want to do reconstruction in extremely small environments where using multiple cameras is sometimes not feasible for various reasons.

Re: 3-D Depth Reconstruction from a Single Still Image (2007) [pdf]

#18

Post by Xcelerate »

TheArcane wrote: post #12
GrantS wrote: post #4

This is some very cool research, though many may be surprised to learn that it is an IJCV journal article from 2007, based on a conference paper from NIPS 2005. Source: http://www.cs.cornell.edu/%7Easaxena/learningdepth/

Gah! I thought this was cutting edge. Is this research used in any recent works/research?

Definitely. It's been cited 658 times since it was published:

https://scholar.google.com/scholar?oi=bibs&hl=en&cites=18064...

Re: 3-D Depth Reconstruction from a Single Still Image (2007) [pdf]

#19

Post by leeoniya »

pen2l wrote: post #17
leeoniya wrote: post #13

Earlier quoted context omitted.

reconstructing depth from 2+ offset images is vastly simpler than from a single frame.

Yup, but I think there's a desire to get good 3d-reconstruction techniques for single images because in practice you often want to do reconstruction in extremely small environments where using multiple cameras is sometimes not feasible for various reasons.

Google Camera can already do this from a single camera. It takes a bunch of photos as you move the phone around a bit and reconstructs the depth from multiple shots.

http://googleresearch.blogspot.com/2014/04/lens-blur-in-new-...

Re: 3-D Depth Reconstruction from a Single Still Image (2007) [pdf]

#20

Post by AndrewKemendo »

Always love seeing computer vision related stuff posted.

Incidentally, we have improved on these techniques (given that this came out almost a decade ago!) for scale reconstruction with our mobile monocular SLAM system.

The authors are now killing it worldwide (Obviously for Ng) working on applications.

Post reply on HN