An algorithm that recreates 3D objects from tiny 2D images • Hacker News Board

Re: An algorithm that recreates 3D objects from tiny 2D images

#31

Post by dahauns » Thu, Aug 24, 2017, 1:10 PM UTC

averagewall wrote: post #29

xg15 wrote: post #27

Earlier quoted context omitted.

I was stumped by that as well. It spends half the text explaining the well-known parts (surface models) and then mentions the actual contribution completely in passing: > So first his system renders a 3D reconstruction of the 2D image in very low resolution [...] Next, do a higher-resolution render of the area you kept. Where apparently the author seems to think producing a 3D reconstruction of a 2D object is trivial…

That's not what the paper is about. The abstract quoted by the GP shows that it has been done before. Probably only very recently so it's going to be a surprise to a lot of readers, but perhaps old news measured in machine learning years.

It was actually astonishing for me how old a lot of the machine learning methods and algorithms are - quite often stuff I worked with on university almost 20 years ago. The main difference being that you have more computation power and memory by orders of magnitude. You can throw datasets at the NN that would have been prohibitive in size back then. Stuff I had to reserve time for on the uni "super-"computer probably runs on your phone nowadays.

Oh my god, I sound like my father.

Re: An algorithm that recreates 3D objects from tiny 2D images

#32

Post by 31reasons » Thu, Aug 24, 2017, 1:43 PM UTC

pasta wrote: post #15

Well I'm sorry but I think more is going on than the article describes. For example the red pickup truck. There is no way the algorithm could create that model from that image only (depth of the trunk). So my guess is that they use the tiny picture to search a database for similar pictures and then create a model with all that data.

If you see a photo of an object that you have never seen in your life and have no analogy to draw from, it will indeed be very hard for you to accurately guess its volume. However as humans we come across millions of objects to learn from. Same way you can train a neural network to learn from millions of objects and that would enable them to make similar guesses.

If this algorithm manages to be as good as humans in estimating volumes of everyday objects, I can see it being helpful in Self Driving Cars.

Re: An algorithm that recreates 3D objects from tiny 2D images

#33

Post by jackhack » Thu, Aug 24, 2017, 2:37 PM UTC

khedoros1 wrote: post #19

13of40 wrote: post #12

I kind of got lost on the first sentence: "With a lifetime of observing the world informing our perceptions, we’re all pretty good at inferring the overall shape of something we only see from the side, or for a brief moment. Computers, however, are just plain bad at it." I'm good at inferring the shape of something from a side view well enough that I can use it in some kludgy mental models, but down the page they're…

The computer's a savant at precise drawing, but the visualization isn't the interesting part. The interesting part is having the computer look at a 2D picture and come up with a "kludgy mental model" of the 3D shape represented by the picture. From the example images, it infers information more accurately than I would've expected.

I can't help but wonder what would result from an Escher drawing.

Re: An algorithm that recreates 3D objects from tiny 2D images

#34

Post by jackhack » Thu, Aug 24, 2017, 2:44 PM UTC

figgis wrote: post #4

EamonnMR wrote: post #3

Could be interesting to see it run on old game sprites. The Starcraft Remastered team could have used this, for example.

How would copyright and licenses apply to something like this?

presumably covered under the common "derivative works" clause

Re: An algorithm that recreates 3D objects from tiny 2D images

#35

Post by ealloc » Thu, Aug 24, 2017, 2:49 PM UTC

"Single Particle Reconstruction" for Cryo-EM reconstruction algorithm, anyone?

SPR is an algorithm already used to reconstruct 3d objects from a set of 2d images produced by an electron microscope, typically of a protein on a flat surface.

I haven't read the paper here, but I suspect a key part of the algorithm is that it can only reconstruct objects with symmetry planes: The airplane, chair, car are all symmetric across an axis. This greatly constrains the possibilities the algorithm has to search through. In em-reconstruction the user often specifies what they think the symmetries are beforehand.

Re: An algorithm that recreates 3D objects from tiny 2D images

#36

Post by ricardobeat » Thu, Aug 24, 2017, 9:59 PM UTC

khedoros1 wrote: post #20

ricardobeat wrote: post #6

Very interesting tech, though the article makes a piss poor job of explaining anything about how it works. Rendering surfaces is how literally 99.9% of 3D graphics work, not a "breakthrough" of any sorts.

99.9% of 3D graphics: "Computer, here's a definition of some 3D surfaces and their properties. Produce a 2D projection." This: "Computer, here's a 2D projection. Infer the likely 3D data that would produce the projection."

The comparison the article makes is against rendering a full voxel volume, as if occlusion was a great insight.

Re: An algorithm that recreates 3D objects from tiny 2D images

#37

Post by Aron » Fri, Aug 25, 2017, 1:43 AM UTC

dharma1 wrote: post #13

Aron wrote: post #8

Earlier quoted context omitted.

Seems like Google Earth is doing this yeah?

for the 3D Google Earth views? It's photogrammetry from a 5 camera rig on a low flying plane https://youtu.be/suo_aUTUpps?t=2m53s

Alright pretty old school. As an aside, one of the first things I did with my Oculus was pull up Google Earth and lay down in Yosemite valley which is feature around 5 minutes in this vid. Cheers.