Learning from Terminals to Design the Future of User Interfaces

Are you referring to Google Home being able to stack up a few commands or the routines feature?

eg Google, whats the weather and set a timer for 15min?

On Alexa you can enable a follow up mode which listens to the next command for a bit. It's a bit more natural, e.g. Alexa, whats the weather... you wait for her to complete, she then listens again and you can say, Set timer for 15min, she completes and listens, Read news, etc.

Sometimes I just feel like the machines are teaching us how they would like to be talked to vs us teaching them, as some commands require specific verbal markers in proper order to complete as otherwise they just give you a completely random answer or they simply don't process it at all.

The transition from DOS to Windows in the 90s was difficult for blind users as well as screen reader developers. For some blind techies who were comfortable with DOS, Linux was indeed a more attractive next step. I was heavily involved in the blind Linux community from 1999 through 2001, and helped several newbies get started with Linux.

But that's ancient history. Even as I was deeply involved in the blind Linux community, Windows screen readers were getting good, particularly for everyday tasks like web browsing. Today, there would be no reason for any blind person other than a programmer or sysadmin to use a command-line interface, let alone a screen-oriented terminal interface.

To see why a screen-oriented terminal interface isn't in fact blind-friendly, consider that Red Hat text-mode installer I talked about last time. On screen, you have an approximation of a GUI using line-drawing characters, some ASCII art (for check boxes), and colors to convey where the focus or selection is. Suppose you're in a list of check boxes with buttons below it. What does the screen reader read when you arrow through the list? When you toggle a check box with Space? When you tab to one of the buttons? With the Linux Speakup screen reader in particular, the output wasn't at all intuitive, and one often had to use screen review commands to be sure of what was happening. I wish I still had a copy of a tutorial I recorded in late 2000 where I walked through the installation of Debian with Speakup. (The Debian and Red Hat installers were and are very similar in this regard.)

Contrast that with the Fedora or Ubuntu graphical installer running under GNOME with the Orca screen reader. Like other major GUI platforms, GNOME has an accessibility API. Screen readers and other assistive technologies can get a tree of UI elements, and receive events about those elements. Assuming the application implements its side of the API (and often the UI toolkit takes care of this), a screen reader has easy access to high-level information about the widgets on the screen, what's happening to them, which one has the keyboard focus, etc. So when you arrow through that list of check boxes, the screen reader can say things like, "Web server, not checked". Then when you hit Space, it can just say, "checked". Finally, when you press Tab, it can say something like, "Next, button". It's clearly a much better experience.

I'm happy to answer any questions if anyone is curious.

This is amazing - thank you!

I think that one of our biggest problems with UI is that it is immutable.

Users can't really change most UI. Developers have to make one ultimate one-size-fits-all UI for all of their users.

Or in my case: spend a bunch of time messing with and tweaking my shell config. Sometimes the tyranny of default options is helpful to getting things done.