Wow, author here, was not expecting this to end up here! This project is very much in development, it's not close to a finished product and I hope I didn't give that impression. I created the site so there'd be a space other than my blog to share updates about progress and so on. If everything I wrote sounds like nonsense, check back in 6 months. I am hoping things will be much further along, and more concrete by the…
"The program is a UI, and UI interaction is programming. Unison panels take the idea of spreadsheets, which blur the distinction between UI interaction and programming, to its logical conclusion."
...this blurring between programming and interaction (which I've also thought of) probably offers a potentially viable path to much more powerful voice control of devices, since programming APIs serving as user interfaces can now also (besides graphical UI) implicitly define strict grammars for structured voice input. That's the one thing I thought of that I didn't see mentioned there.