Earlier quoted context omitted.
I think blind people / screen readers might have difficulty with this interface, too. It's important to keep accessibility in mind when designing futuristic interfaces.
If you press tab, it at least cycles through all of the input boxes on the front side in an appropriate manner, and you could probably design it to flip over on the next tab press. I'm fairly certain that would make it work with screen readers (but not positive).
Those of us familiar with data entry get really used to tabbing.
It doesn't tell you if you go wrong either.
I surprisingly liked the idea and execution though (putting the other issues aside.)