Obvious Engine: a vision-based augmented reality engine for indie games

What would you be able to do that you can't do now?

While it may be obvious when a virtual object is missing a shadow (as in the demonstration video), even implementing that would not be enough to fool the human brain completely, though it isn't always obvious what the problem is, only a subconscious nagging.

If you're interested in what precisely the effects that the real world has and augmented reality generally doesn't have - i'd say the biggest ones are shadows (including soft shadows [0]), depth of field [1], ambient occlusion [2], and indirect lighting.

You don't necessarily have to write a full-blown raytracer with global illuminasion, casting billions of rays to get a passable result. All of the above mentioned things can more or less be approximated in some way, and most modern game engines do so (I don't know of any phone games that do, mind, as the calculations are still non-trivial, but I do remember an interesting tech demo from Nvidia).

The biggest problem, though, is that we can do these things in a controlled setting, where you know the exact shape, texture and light reflective properties on every object in the scene. As you can imagine, that is not so in the case of augmented reality (as I quipped in my earlier post, you would have to infer surface reflectance from the image in some way). Compared to the problem of global illumination in a virtual scene - well, let's just say it's orders of magnitude harder. I don't recall anyone actually having scanned a scene with a single camera. And that's an AI breakthrough that indeed people would talk about.

[0] : http://en.wikipedia.org/wiki/Soft_shadows [1] : http://en.wikipedia.org/wiki/Depth_of_field [2] : http://en.wikipedia.org/wiki/Ambient_occlusion [*} : Also, google images

Now I do like the particle trick. You see this in some video games to highlight objects of interest (e.g. World of Warcraft). It makes it much easier for players to locate them and zero in on them against background clutter. Probably useful for AR if you have a good application case.

However, the potential is heavily constrained until you get displays capable of "non-interfering annotation" - which is to say, you need a good HMD and not a phone. Not that you can go out and buy "a good HMD" by this measure, but that's a big thorny mess out to the side of the software concerns.