That's the classic example of using a new technology to emulate an old technology, without taking advantage of the unique advantages of the new technology, before the grammar and language of film had been invented.
https://en.wikipedia.org/wiki/Film_grammar
https://en.wikipedia.org/wiki/History_of_film
>The first decade of motion picture saw film moving from a novelty to an established mass entertainment industry. The earliest films were in black and white, under a minute long, without recorded sound and consisted of a single shot from a steady camera.
>Conventions toward a general cinematic language developed over the years with editing, camera movements and other cinematic techniques contributing specific roles in the narrative of films. [...]
>In the 1890s, films were seen mostly via temporary storefront spaces and traveling exhibitors or as acts in vaudeville programs. A film could be under a minute long and would usually present a single scene, authentic or staged, of everyday life, a public event, a sporting event or slapstick. There was little to no cinematic technique, the film was usually black and white and it was without sound. [...]
>Within eleven years of motion pictures, the films moved from a novelty show to an established large-scale entertainment industry. Films moved from a single shot, completely made by one person with a few assistants, towards films several minutes long consisting of several shots, which were made by large companies in something like industrial conditions.
>By 1900, the first motion pictures that can be considered "films" – emerged, and film-makers began to introduce basic editing techniques and film narrative.
Simply projecting desktop user interfaces designed for flat 2D screens and mice into VR is still in the "novelty show" age, like filming staged plays written for a theater, without any editing, shots, or film grammar.
VR window managers are just a stop-gap backwards-compatibility bridge, while people work on inventing a grammar and language of interactive VR and AR user interfaces, and re-implement all the desktop and mobile applications from the ground up so they're not merely usable but actually enjoyable and aesthetically pleasing to use in VR.
The current definition of "window manager," especially as it applies to X-Windows desktops, tightly constrains how we think and what we expect of user interface and application design. We need something much more flexible and extensible. Unfortunately X-Windows decades ago rejected the crucially important ideas behind NeWS and AJAX, that the window manager should be open-ended and dynamically extensible with downloadable code, which is the key to making efficient, deeply integrated user interfaces.
For example, the "Dragon Naturally Speaking" speech synthesis and recognition system has "dragonfly", a Python-based "speech manager" that is capable of hooking into existing unmodified desktop applications, and scripting custom speech based user interfaces.
https://github.com/t4ngo/dragonfly
Another more ambitious example is Morgan Dixon's work on Prefab, that screen-scrapes the pixels of desktop apps, and uses pattern recognition and composition to remix and modify them. This is like cinematographers finally discovering they can edit films, cut and splice shots together, overlay text and graphics and pictures-in-pictures and adjacent frames. But Prefab isn't built around a scripting language like dragonfly, NeWS or AJAX.
Here's some stuff I've written about the direction that user interfaces should take to move beyond the antique notion of "window managers", and enables much deeper integration and accessibility and alternative input and output methods.
https://news.ycombinator.com/item?id=14182061
>Glad to see people are still making better window managers! [...] I think extensibility and accessibility are extremely important for window managers. [...] I'd like to take that idea a lot further, so I wrote up some ideas about programming window management, accessibility, screen scraping, pattern recognition and automation in JavaScript. [...] Check out Morgan Dixon's and James Fogarty's amazing work on user interface customization with Prefab, about which they've published several excellent CHI papers: [...]
>Imagine if every interface was open source. Any of us could modify the software we use every day. Unfortunately, we don't have the source.
>Prefab realizes this vision using only the pixels of everyday interfaces. This video shows the use of Prefab to add new functionality to Adobe Photoshop, Apple iTunes, and Microsoft Windows Media Player. Prefab represents a new approach to deploying HCI research in everyday software, and is also the first step toward a future where anybody can modify any interface.
https://news.ycombinator.com/item?id=18797818
>Here are some other interesting things related to scriptable window management and accessibility to check out: aQuery -- Like jQuery for Accessibility
https://donhopkins.com/mediawiki/index.php/AQuery
>It would also be great to flesh out the accessibility and speech recognition APIs, and make it possible to write all kinds of intelligent application automation and integration scripts, bots, with nice HTML user interfaces in JavaScript. Take a look at what Dragon Naturally Speaking has done with Python:
https://github.com/t4ngo/dragonfly
>Morgan Dixon's work with Prefab is brilliant.
>I would like to discuss how we could integrate Prefab with a Javascriptable, extensible API like aQuery, so you could write "selectors" that used prefab's pattern recognition techniques, bind those to JavaScript event handlers, and write high level widgets on top of that in JavaScript, and implement the graphical overlays and gui enhancements in HTML/Canvas/etc like I've done with Slate and the WebView overlay.