An algorithm that recreates 3D objects from tiny 2D images

I agree. I think that the AI has knowledge of those objects.

The fact that all the objects are symmetric (at least in the examples) helps a lot.

This article is confusingly written. Its explanations sound silly to anyone even a little familiar with 3d, since it spends the first half of the article explaining that this "breakthrough" is "computationally clever and forehead-slappingly simple", the breakthrough being that you can represent things as surface-models instead of voxels.

Well, surface rendering is how almost all 3d work is done already, that's definitely not a breakthrough. You can probably spend an entire career never dealing with voxels.

What this article never mentions (surprisingly) is that this paper is about neural networks. I'm not an expert, but as I understand the article, voxel representations have been the standard specifically when building neural networks to turn 2d images into 3d. The main idea is that you can build a network that only renders high-resolution voxels when close to the surface of the model, and renders very low-resolution voxels everywhere else (say, on the inside of the model). This means you can both represent much larger models memory-wise, but also that you're not having to run the NN computations on voxels that are not likely to change, since everything "inside" of the model you probably guessed correctly pretty quickly.

Here's the paper's abstract, much better at explaining itself than the article:

"Recently, Convolutional Neural Networks have shown promising results for 3D geometry prediction. They can make predictions from very little input data such as for example a single color image, depth map or a partial 3D volume. A major limitation of such approaches is that they only predict a coarse resolution voxel grid, which does not capture the surface of the objects well. We propose a general framework, called hierarchical surface prediction (HSP), which facilitates prediction of high resolution voxel grids. The main insight is that it is sufficient to predict high resolution voxels around the predicted surfaces. The exterior and interior of the objects can be represented with coarse resolution voxels. This allows us to predict significantly higher resolution voxel grids around the surface, from which triangle meshes can be extracted. Our approach is general and not dependent on a specific input type. In our experiments we show results for geometry prediction from color images, depth images and shape completion from partial voxel grids. Our analysis shows that the network is able to predict the surface more accurately than a low resolution prediction."

As lots of people already pointed out: there is more going on than just a smart upsampling technique.

See section 4 of the paper - they train on a set of 3d models from some 'ShapeNetCore' dataset, from which they generate sample inputs (renderings of the model with randomized viewpoint and lighting) and corresponding target outputs (voxelized model).

They train specialized networks for different classes of objects 'aeroplanes, chairs and car', so reconstructing on all classes at the same time probably still has some issues.

An interesting point about their use of this coarse-to-fine progression that the article omits: they do the same trick for training their net - first train to predict the coarse voxels, and when those work start predicting the next level.

Yeah, there's definitely some previous knowledge of the kinds of objects it's inferring, which is used to deduce the parts.

Look at the chairs' legs, where it's transforming the flat base of the rotating chair into something that looks as wheels, and where it's completely missing the bars connecting the legs in the second tall chair.