We Have Reached Peak Screen. Now Revolution Is in the Air

I think the suggestion is more along the Ready Player One lines where the virtual world feels more real than the real world and most choose to spend most of their lives there.

Visual signals and interfaces have information density and permanence, which gives gleanability. You can put lots of things into your visual field (e.g. multiple application windows, multiple computing devices, multiple inanimate objects), where they'll stay put without further input. Inherent cognitive ability even allows us to track multiple objects in motion throughout a scene and keep them coherent -- which is the skill that enables driving. Visual interfaces are very well suited to how humans best absorb information, and how they context switch given a large number of potential tasks.

A world where we begin to move off of visual interfaces will be awkward. While humans are good at absorbing conversational audio, they mentally filter most of it out to distill it down to its essential elements, and knowing what's essential may not even be known ahead of time. We'll direct voice-outputting interfaces to repeat things often, and they must be smart enough to accurately determine the context of our inquiry.

Voice output is often paired with voice input, but voice propagates well in public, leaking information to everything in range. Devices that capture speech-like input in a private way are not yet widespread. Meanwhile, structured command input through voice is awkward, and natural language processing doesn't sound natural yet. It's complex to implement and the computer frequently encounters a situation it doesn't yet understand, which is the most discouraging kind of interaction one can have with a computing platform. Factors like these highlight that audio-based interfaces are rarely programmed to be discoverable, and even if they were, exchanging that information over audio is less efficient than doing so visually.

New research into interface design is needed to address many of the shortcomings of current attempts to de-emphasize screens.

Great message, but this is mostly an opinion piece with few technical details. I am also not a fan of how the answer is to wait for Apple (Big Tech) to save us.

Bret Victor wrote a piece [0] lamenting the convergence on screens as the interaction design paradigm almost 7 years ago. Bret explains why screens are limiting interaction design through examples centered around the human body.

Ironically, this NYT piece gives the impression that a human being is a floating head and fingers i.e. an AR/VR avatar that they seem to loathe. I hope the future of computing isn't just the ability to check my calendar without a screen while walking. I want to use my body in tandem with computation. I don't have a Killer App for this interaction paradigm, but I found this paper by Scott Klemmer, Björn Hartmann, and Leila Takayama useful for thinking about it [1].

[0] http://worrydream.com/ABriefRantOnTheFutureOfInteractionDesi... [1] https://hci.stanford.edu/publications/2006/HowBodiesMatter-D...

So, a feature phone?

Think of it this way: we spend 16 hours a day looking at things in general. Over 50% of our brain's cortex is dedicated to processing visual information. It makes sense that we're going to gravitate towards devices that provide us with an easy means of giving us visual information. We're just structured that way. If over 50% of our brain's cortex was dedicated to processing aural information, or physical touch, or smell, then the dominant devices nowadays would be related to that.

The main problem with screens (and the reason why I try not to do it as much as I used to) are 1) If you're interacting with a screen all the time, you're probably not interacting with someone in person, and everyone's ability to read body language and subtle cues probably goes way down when they do interact in person. 2) The current screens emit light that at the very least probably harm our biorhythms, but may harm us in other ways as well. Meanwhile the Kindle isn't much different than looking at a book or a sign. and 3) Screens provide a limited window into another world, and at least right now, that window has zero depth (and doesn't engage our other senses either while we're at it). We don't get to take advantage of having two eyes and seeing depth while we're on a screen. This may change once VR becomes more and more viable and realistic.

I do try to do things more analog myself now though. Writing or designing on paper while outside in good weather is much preferable to inside on a screen. It's also why I've gravitated to more offscreen hobbies, such as board games and board game design, as opposed to staying on screens and programming games and apps in my off hours. Screens can let you get those things done faster, though (i.e. it's much faster and less straining on my hands to type than to write all of my thoughts).