How X Window Managers Work, and How to Write One (2014)

Why does QT needs to be smarter ? Just asking.

There are always bugs in complex software, so sure. I can also consider myself lucky that I'm not forced into using a Wayland compositor as it'd force me to change my workflows in all kinds of ways.

> If you only care about bugs that affect you then it's not even worth having a discussion about this, you can't expect everyone to wait to make changes until it's personally convenient for you

I'm not expecting anything other than to keep using the software which works for me. I'm not the one arguing for people to change setups that works for us. Use what works for you. If that's Wayland that's your choice. For me it's not - it involves putting effort into something that doesn't give me amy benefits. Nobody is arguing for you to do anything or not anything.

> I would urge you to look into how much work it would take to actually fix those bugs and push out incompatible server/libraries/etc to people and then come back and see if you'd like to revise this statement.

No, I wouldn't like to revise that statement, because none of the bugs are relevant for an updated protocol unless you choose to keep the things that are broken, since the "worst case" is to throw away and start over the same way Wayland did.

> Remember what you are asking here is for every toolkit and program to do a "if x12 then dostuff elseif x11 then do otherstuff" and weigh that versus "if wayland then dostuff elseif x11 then do otherstuff".

And it's obvious that the number of if x12 then dostuff would have been far smaller and/or amortized over longer time because you'd only need them in cases where you actually introduced a breaking change that couldn't be fixed by using functionality that would keep working the same on both.

Most modern X11 apps already don't use most of the functionality of X that is worth deprecating, and for many of those who do the solution would be to change to a request that'd be worth keeping, not to add an if x12. E.g. as I mentioned there are a number of different ways of rendering text with X. The most popular terminals are split roughly in two broad groups: Those who use the old server side rendering, and those who use XRender (a couple of outliers use OpenGL). Most of the ones I've looked at use Xlib etc. directly rather than toolkits, though some of the font stuff in Xlib is wrapping the server side functionality in ways that could be retargeted to use Freetype.

If you remove the former, then ~half of the terminals would need to get updated to use the latter method or forced to use a proxy (ditching most of them isn't really an option - many of the most feature-complete ones use X directly, and all of the ones I have looked at have users who swear by them and will keep using them until there is a feature complete replacement; many of them are decades old and their users still don't agree there's one worth switching to) or client-side reimplementation of the font rendering. But assuming you were to retain the XRender glyph rendering you wouldn't need to special case for this hypothetical x12, just upgrade apps to use XRender, and you could yank out all the font support in the server. Moving to Wayland on the other hand requires all of them to change

To date most of the terminals I've looked at haven't been updated with Wayland support, and many probably never will, making XWayland and so most of Xorg still hang around as long as they have users and someone cares. Given how long many of them have survived, I'm guessing you'll have to wait for their user bases to literally die out.

The X12 approach would have allowed starting to actually deploy the improvements stepwise more than a decade ago. Or more. E.g. the XFIXES extension happened in 2003 and was the first real attempt at some somewhat potentially breaking changes hidden behind an extension flag rather than a version change (and didn't allow the server to actually ditch the old code). Nothing other than politics stopped a more drastic approach.

This notion that there were technical barriers stopping an iteration of X is pure fiction.

I don't get what you point is. What problem or confusion do you think I have with the difference between the terms "client" and "server", and are you blaming me that a remote X-Windows client is called a "client" and a remote web server is called a "server"? Or is it my fault that web "servers" and web browser "clients" are called the other way around than X-Windows "clients" and X-Windows "servers"? Or are you just offended that I pointed out that fact?

The fact that the web finally ended up the way it did, with a so-called "AJAXian" architecture using JavaScript in the web browser (and often the web server too) (not to mention the PostScript Stencil/Paint imaging model in canvas), instead of a fixed protocol like X-Windows (stuck with its terrible obsolete imaging model), certainly proves my point that NeWS's architecture of sending code instead of a fixed protocol like X-Windows is by far the superior, more efficient, future-proof, open-ended architecture.

The term "AJAX" was coined in 1999, but NeWS (aka SunDew) was doing it in 1985, 14 years before the term "AJAX" was coined in 1999. And X-Windows still isn't doing it, 37 years after James Gosling originally proposed that architecture in his paper describing SunDew.

https://en.wikipedia.org/wiki/NeWS

>NeWS was architecturally similar to what is now called AJAX, except that NeWS coherently:

>used PostScript code instead of JavaScript for programming.

>used PostScript graphics instead of DHTML and CSS for rendering.

>used PostScript data instead of XML and JSON for data representation.

I worked on the display driver for the NeWS version of Gosling/UniPress Emacs, and also on the NeWS display driver for Gnu Emacs, and thanks to NeWS's ability to download code that implements application specific protocols, both versions of Emacs ran quite smoothly, efficiently, and responsively over slow dial-up modems, ISDN, and the slow internet connections of the late 80's - early 90's.

Here's an email I sent to James Gosling in 1988 describing how the Emacs display driver I implemented for UniPress Emacs 2.20 works, by providing smooth live text selection feedback without requiring client/server round trips every time the mouse moves, as well as interactive pie menus and tabbed windows, all implemented in PostScript in the NeWS server:

https://www.donhopkins.com/home/archive/emacs/to.jag.txt

>Hi! I'm working on Emacs this summer at Unipress. It now supports menus, and text selection, with rubber banding in the server. It works, and is much fun to use, but I have a few questions about getting it RIGHT!

>When you initiate a selection by clicking with the mouse, emacs replies with an array of the line widths in the current window, and the row of the window's top line. Some code I stole from nterm rubber bands an outline around the text as you drag around the window. I modified it to show that a line's trailing newlines was selected by drawing the right edge with an arc or a slant.

>[...] Other stuff we're working on include a menu compiler, which compiles nested menus described as linked info nodes into PostScript code and MLisp key bindings. (Menu keys are assigned keystroke sequences, with keytop names that identify them, which are bound to MLisp functions. Menu keys can invoke submenus or do anything else you want, too.) So you can make menus without touching PS or MLisp! The menus all run in the server, so you can type into the emacs underneath while you're in the middle of completing a menu selection. I changed getmenuaction to execute executable keywords, to support indirect submenu references, and magic functions that decide the submenu or menu action.

>[...] I made a class of window that has thin borders, and a little handle sticking out the top right with its FrameLabel -- you can have a bunch of them overlapping, and their names all stick out to the right, where you can click on them to bring them to the top or get a frame menu. They work really well with multiple emacs frames!

>Thanks for all the neat toys!

Here's a screen snapshot of that code running, from the wikipedia page about tabbed windows:

https://en.wikipedia.org/wiki/Tab_(interface)#/media/File:Hy...

And here's the PostScript source code of the emacs client side display driver:

https://www.donhopkins.com/home/archive/emacs/emacs.ps

Also, in case you weren't aware, a single X-Windows "client" can have simultaneous connections to multiple X-Windows "servers", in the same way that a single web "server" can have simultaneous connections to multiple web "browsers".

A good example is SimCity (aka multi player SimCityNet and open source Micropolis), which is a commercial product that I developed for NeWS in PostScript, for X-Windows in TCL/TK (later releasing a single player version as Micropolis for the OLPC), and a multi player open source version for web servers in Python (whose distributed client/server "AJAXian" architecture was most similar to the original NeWS version, no matter how you label the client and server).

The TCL/Tk/X11 version of multi player "SimCityNet" supported simultaneous connections to several players on different (or the same) X-Windows servers at once, so they could play together in the same city.

Product Announcement: Multi Player SimCity for X11 is now available from DUX Software! (1993):

https://www.donhopkins.com/home/catalog/simcity/simcity-anno...

SimCityNet: a Cooperative Multi User City Simulation (1993 INTERCHI Amsterdam demo)

https://www.donhopkins.com/home/catalog/simcity/simcitynet.h...

Multi Player SimCityNet for X11 on Linux. Demo of the latest optimized Linux version of Multi Player SimCity for X11. Ported to Unix, optimized for Linux and demonstrated by Don Hopkins:

https://www.youtube.com/watch?v=_fVl4dGwUrA

Micropolis Online (multi player SimCity running on a web server in Python) web demo. A demo of the open source Micropolis Online game (based on the original SimCity Classic source code from Maxis), running on a web server, written in C++ and Python, and displaying in a web browser, written in OpenLaszlo and JavaScript, running in the Flash player. Developed by Don Hopkins.

https://www.youtube.com/watch?v=8snnqQSI0GE

Unfortunately the support for multi-player applications in X-Windows toolkits like TCL/Tk was buggy and incomplete, so I had to fix it for SimCityNet, and the support for security and authentication in X-Windows is STILL abysmally terrible, so I had to drop the multi player feature from the OLPC version of Micropolis, because I wouldn't want to require children to type "xhost +" or enter hexadecimal magic cookies and ip addresses to play a game.

But it was much safer and easier to implement a secure multi player game in Python on a web server over https to multiple simultaneous web browsers.

Here's the Python source code of the web server, which uses the C++ reimplementation of SimCity called MicropolisCore via SWIG wrappers:

https://github.com/SimHacker/micropolis/blob/master/turbogea...

And to demonstrate what I mean by downloading procedural code implementing custom compression protocols between client and server instead of using a fixed protocol, and communicating using efficient application specific protocols instead of rigid fixed protocols like X-Windows, here is a concrete example you can see for yourself.

This is the server side of the tile compression protocol: GetTileData returns a buffer of compressed tile differences (of only the tiles the user can currently see) to send to the browser, which is further optimized to support SimCity's notion of "animated tiles", so it doesn't even have to send deltas for tiles that are automatically animate in the client, or outside of the user's scrolling view:

https://github.com/SimHacker/micropolis/blob/master/Micropol...

The Python code running in the web server gets the compressed tile data and sends it to the client:

https://github.com/SimHacker/micropolis/blob/master/Micropol...

The JavaScript code running in the web browser receives the compressed tiles, decompresses them, and updates the tile view:

https://github.com/SimHacker/micropolis/blob/master/laszlo/m...

The resulting distributed real time animated user interface is quite snappy and responsive over the network (especially the pie menu, which track input, interact, and display feedback locally, and only send messages to the server upon completion), as you can see in the demo video above.

Are you following along so far? This "AJAXian" architecture is precisely what James Gosling was writing about 36 years ago in 1985, when he described the client interaction of "SunDew", which was later renamed "NeWS":

http://www.chilton-computing.org.uk/inf/literature/books/wm/...

>5.3.4 Client Interaction

>Client interaction with the window system can be broken down into three layers: messages that get passed between the window system and the client, the programs that are contained in the messages, and a procedural interface to program construction. This is one of the key unique points about this window system: clients send programs to the window system whose elaboration causes graphic operations to appear. The client program does not send graphic operations directly. There is, of course, a procedural interface to program construction that blurs the distinction.

>This approach allows the client to redefine the protocol to compress messages passed to the window system. For example, if the client wishes to draw a grid it can download a procedure which iteratively draws the lines, generating their coordinates as a function of the loop index. This is a substantial compression over transmitting a large set of lines, even if the grid is drawn only once. There are other performance advantages: message-passing interactions are relatively slow and downloading a procedure that will respond locally eliminates this overhead. For example, if a client program needs rubber band lines, then rather than have the window system send a message to it each time the mouse moves, it can download a procedure that will track the mouse locally.

>There are a number of features of PostScript that make it attractive. It is very simple, and it is complete. Its simplicity makes it fast to translate and interpret - this can almost be done as quickly as packets can be disassembled in a more traditional IPC environment. Adobe did a very good job of designing a set of primitives and data structures that fit together well. Its chief drawback is that it can be hard for people to understand; the postfix notation is well-suited to consumption and generation by programs, but humans find it obscure.

>It is important to think about the client programmer's model of what the window system does. We expect there to be two levels of model. The first completely hides the existence of PostScript with a veneer of procedure calls that construct and transmit fragments of PostScript programs. The second exposes PostScript. Beyond a certain level of functionality, learning PostScript is inevitable: it can be pushed off by making the veneer more comprehensive, but this just makes the eventual leap harder.

Thanks for asking! Hold my bong. ;)

I've never had any reason to use Wayland or desire to learn much about it, so I can't tell you anything technical or from first hand experience.

But I think we have X-Windows to thank for the fact that most Unix programmers use Macs now.

And it's way too late for Wayland to change that fact. Especially with the release of the M1 Max. That ship has sailed.

It didn't matter how much better NeWS was than X-Windows -- it still lost despite all its merits.

And I don't see Wayland as being any more better than X-Windows, than NeWS was better, decades ago.

So simply "being better than X" is not sufficient to displace it, and Wayland isn't even that much better than X, and is even worse in some ways.

The fact that it didn't occur to the Wayland designers to make it extensible with an embedded scripting language like PostScript in NeWS or Lisp in Emacs or Python in Blender or Lua in Redis or JavaScript in web browsers means that it was obsolete from the start, and its designers should have considered the design and architecture of NeWS and Emacs and Blender and Redis and web browsers before designing Yet-Another-X-Windows-Clone. It's not like those ideas were secret, or patented, or hard to find, or wrong.

The world moved up the stack a layer to the web browser, and that's where all the excitement's happening these days, not in the window system layer.

Why have X-Windows or Wayland at all, when you could just run the web browser itself directly on the hardware, as efficiently and flexibly as possible, and implement all your user interface, window management and desktop stuff with modern open standard JavaScript / WebAssembly / JSON / XML / HTML / SVG / CSS / Canvas / WebGL / WebGPU / HTTPS / WebRTC?

I've written about this numerous times before, but I'll transclude and reorganize some of it with checked and updated archive urls to save you the pointing and clicking:

https://news.ycombinator.com/item?id=5844345

>colanderman>And this is my primary issue with Wayland. I cannot fathom why anyone would think it's a sound design decision to bundle a hardware-independent component (the window manager) with a hardware-dependent component (the compositor). This hearkens back to the days of DOS video games – what fun it was to implement support for everyone's sound card! Instead now we'll get to support KMS, Quartz, whatever-the-heck BSD uses, etc.

>Just put a JavaScript (or whatever) interpreter in the window server, and program the window manager locally in that. Then you aren't fucked by synchronization issues. James Gosling did something like that with PostScript many years ago, an alternative to X11, which was then merged with X11, and it was called NeWS (and later X11/NeWS or OpenWindows):

http://en.wikipedia.org/wiki/NeWS

>I've written several window managers / user interface toolkits / tabbed window frames / menu system in PostScript for NeWS. We even wrote an X11 window manager in PostScript, complete with rooms, scrolling virtual desktop, tabbed windows, pie menus, and seamless integration of X11 and NeWS windows.

https://www.youtube.com/watch?v=tMcmQk-q0k4

https://news.ycombinator.com/item?id=5845119

>Having the window manager running as an outboard process, communicating via an asynchronous protocol, is a terrible idea, but is baked into X11 at a very low level -- with the ICCCM thrown in as a kludgy afterthought. The X11 protocol has hacks and band-aids like "mouse button grabs" and "server grabs" to mitigate the problems, but they cause a lot of other problems, complexity and inefficiencies on their own.

https://news.ycombinator.com/item?id=20182294

>That's exactly the point of using HyperLook as a window manager, which we were discussing at Sun in 1991 before they canceled NeWS.

>HyperLook, which was inspired by HyperCard, but written in PostScript for the NeWS window system, let you build specialized task-oriented user interface by assembling and customizing and scripting together components and applets into their own "stacks".

https://news.ycombinator.com/item?id=19007885

>HyperLook (which ran on NeWS, and was like a networked version of Hypercard based on PostScript) was kind of like looking outwards through windows, in the way Rob Pike described the Blit (although it was architecturally quite different).

>I think his point was that window frames should be like dynamically typed polymorphic collections possibly containing multiple differently typed components that can change over time (like JavaScript arrays), not statically typed single-element containers that are permanently bound to the same component type which can't ever change (like C variables).

>With X11, the client comes first, and the window manager simply slaps a generic window frame around it, subject to some pre-defined client-specified customizations like ICCCM properties to control the window dressing, but not much more, and not under the control of the user.

>With HyperLook, the "stack/background/card" or window frame came first, and you (the user at runtime, not just the developer at design time) could compose any other components and applications together in your own stacks, by copying and pasting them out of other stacks or warehouses of pre-configured components.

https://donhopkins.medium.com/hyperlook-nee-hypernews-nee-go...

>For example, I implemented SimCity for HyperLook as a client that ran in a separate process than the window server, and sent messages over the local network and drew into shared memory bitmaps. SimCity had its own custom frames ("screw-up windows") that looked more "mayoral" than normal windows.

https://cdn-images-1.medium.com/max/2560/0*ZM8s95LNxemc5Enz....

>There were multiple map and editor views that you could copy and paste into other stacks while they were running, or you could copy widgets like buttons or drawing editors into the SimCity stack while it was running. So you could copy and paste the "RCI" gauge into your graphics editor window to keep an eye on it while you worked, or paste a clock into your SimCity window to see when it was time to stop playing and get some sleep. And you could even hit the "props" key on the clock to bring up a PostScript graphics editor that let you totally customize how it looked!

https://cdn-images-1.medium.com/max/600/0*oHtC0F5qK83ADw1H.g...

>It had "warehouse" stacks containing collections of pre-configured component prototypes (including widgets, applets, and window management controls like close buttons, resize corners, navigation buttons, clocks, graphics editors, etc) that you could copy and paste into your own stacks, and which would automatically be listed in the "New Object" menu you get in edit mode, to create them in place without opening up the warehouse:

https://cdn-images-1.medium.com/max/600/0*sHClGU8ALljuRQKb.g...

https://cdn-images-1.medium.com/max/600/0*QwIQ_GLxQl1v968F.g...

https://cdn-images-1.medium.com/max/600/0*aWbuo6k_eJuZnUmV.g...

https://cdn-images-1.medium.com/max/600/0*zya4vNBP3libpNSA.g...

https://cdn-images-1.medium.com/max/600/1*G0LWky2iejYm4IGBsU...

>Normal X11 window managers put the cart before the donkey. The window frames are generic, stupid and unscriptable, and can't even be customized by the user, or saved and restored later. The client comes first, with just one client per generic frame. You can't move or copy a widget or panel from one frame to another, to use them together in the same window. That's terribly limited and primitive.

>We implemented a more-or-less traditional (but better than normal) ICCCM X11 window manager in PostScript for X11/NeWS, which had tabbed windows, pie menus, rooms, scrolling virtual desktop, etc, uniformly for all NeWS apps and X11 clients. But the next step (not NeXT Step) was to use HyperLook to break out of the limited one-client-per-frame paradigm.

http://www.art.net/~hopkins/Don/unix-haters/x-windows/i39l.h...

>(It's interesting to note that David Rosenthal, the author of the ICCCM specification, was also one of the architects of NeWS, along with James Gosling.)

>Our plan was to use HyperLook to implement a totally different kind of integrated scriptable X11 (and NeWS) window manager, so you could compose multiple X11 clients into the same frame along with HyperLook widgets and scripts. You could integrate multiple X11 clients and NeWS components into fully customizable seamless task oriented user interfaces, instead of switching between windows and copying and pasting between a bunch of separate monolithic clients that don't know about each other. But unfortunately Sun canceled NeWS before we could do that.

>Here's some more stuff I've written about that stuff, and how window managers should be implemented today in JavaScript:

https://news.ycombinator.com/item?id=18837730

>That's how I implemented tabbed windows in 1988 for the NeWS window system and UniPress Emacs (aka Gosling Emacs aka Evil Software Hoarder Emacs), which supported multiple windows on NeWS and X11 long before Gnu Emacs did. That seemed to me like the most obvious way to do it at the time, since tabs along the top of bottom edges were extremely wasteful of screen space. (That was on a big hires Sun workstation screen, not a tiny little lores Mac display, so you could open a lot more windows, especially with Emacs.)

>It makes them more like a vertical linear menu of opened windows, so you can real all their titles, fit many of them on the screen at once, and you instantly access any one and can pop up pie menus on the tabs to perform window management commands even if the windows themselves are not visible.

https://news.ycombinator.com/item?id=18865242

>But now that there's another Web Browser layer slapped on top of X-Windows (and the terms "client" and "server" switched places), you're stuck with two competing mutually incompatible window managers nested inside of each other, one for the outside frames around apps, and one for the inside tabs around web pages, that don't cooperate with each other, and operate uncannily differently.

>So none of your desktop apps benefit from any of the features of the web browser (unless they include their own web browser like Electron).

>And you can't write lightweight apps in a couple pages of script like "Big Brother Eyes" that run in the shared window server, without requiring their own separate external app with a huge runtime like Electron.

https://www.donhopkins.com/home/archive/news-tape/fun/eye/ey...

>Nor can you easily integrate desktop applications and widgets and web pages together with scripts, the way HyperCard, OpenDoc, CyberDog and HyperLook enabled.

https://medium.com/@donhopkins/hyperlook-nee-hypernews-nee-g...

>Here's how the web browser and window manager should work together seamlessly:

https://news.ycombinator.com/item?id=18837730

>>Now Firefox and Chrome still need decent built-in universally supported and user customizable pie menus, but unfortunately the popup window extension API is inadequate to support them, because there's no way to make them pop up centered on the cursor, or control the popup window shape and transparency. Unfortunately they were only thinking of drop-down linear menus when they designed the API. (Stop thinking inside that damn rectangular box, people!)

>>But I remain hopeful that somebody will eventually rediscover pie menus in combination with tabbed window for the web browser, and implement them properly (not constrained to pop up inside the browser window and be clipped by the window frame, and not just one gimmicky hard coded menu that user's can't change and developers can't use in their own applications). But the poorly designed browser extension APIs still have a hell of a lot of catching up to do with what it was trivial to do in NeWS for all windows 30 years ago.

>And this is why a modern window manager should be written in JavaScript, leverage HTML, Canvas and WebGL, and support Accessibility APIs, as well as screen scraping, pattern recognition, screen casting, virtual desktops, overlays, drawing and image composition, input event synthesis, journaling, macros, runtime user customization and scripting, programming by demonstration, tabbed windows, pie menus, etc:

https://news.ycombinator.com/item?id=18797587

>[...] About 5 years ago I opened this issue, describing an experiment I did making the web browser in a topmost window with a transparent background to implement user interface overlays scripted in HTML.

>WebView window for HTML user interfaces like pie menus to Slate. #322:

https://github.com/jigish/slate/issues/322

>Slate used a hidden WebView for its scripting engine. So I made it un-hidden and float on top of all the other windows, and was easily able to use it to draw any kind of user interface stuff on top of all the other Mac windows. And I could track the position of windows and draw a little clickable tab next to or on top of the window title bar, that you could click on to pop up a pie menu.

>It actually worked! But I didn't take it much further, because I never got any feedback on the issue I opened, so gave up on using Slate itself, and never got around to starting my own JavaScript window manager myself (like you did!). I opened my issue in June 2013, but the last commit was Feb 2013, so development must have stopped by then.

>[...] Think of it like augmented reality for virtualizing desktop user interfaces and web pages.

https://news.ycombinator.com/item?id=5861229

>[...] So I adapted the "uwm" window manager to support pie menus, so you could define your own pie menus and linear menus in your .uwmrc file. Because a window manager could really benefit from pie menus: lots of the commands are spatially oriented and can be arranged in appropriate mnemonic directions, and they have a fixed set of common window management commands that can be thoughtfully arranged into a set of efficient pie menus, as well as a menu definition language to enable users to create custom menus for running apps, connecting with remote hosts, etc.

>[...] I had been using Mitch Bradley's "Forthmacs", which was a very nice Forth system for the 68k. (It eventually evolved into the Sun 4 Forth boot ROMS, OpenFirmware, and the OLPC boot ROMs). It was capable of dynamically linking in C code (well not dll's or shared libraries, but it would actually call the linker to relocate the code to the appropriate place in Forth's address space, read in the relocated code, and write Forth wrappers, so you could call C code from Forth, pass parameters, etc -- SunOS 4.2 didn't have relocatable shared libraries or light weight threads back then, so Forth had to do a lot of the heavy lifting to plug in C code. But Forth was a really great way to integrate a library into an interactive extension language, call it from the Forth command line, build on top of C code in Forth, call back and forth between C and Forth, play around with it from the Forth command line, etc).

https://news.ycombinator.com/item?id=16839825

>Inspired by HyperCard, we (old Sun NeWS hands) also pleaded until we were blue in the face to make HyperLook (a NeWS/PostScript/network based reinterpretation of HyperCard) the window manager / editable scriptable desktop environment!

https://news.ycombinator.com/item?id=18314265

https://medium.com/@donhopkins/hyperlook-nee-hypernews-nee-g...

>Alan Kay on NeWS:

>“I thought NeWS was ‘the right way to go’ (except it missed the live system underneath). It was also very early in commercial personal computing to be able to do a UI using Postscript, so it was impressive that the implementation worked at all.” -Alan Kay

>What’s the Big Deal About HyperCard?

>"I thought HyperCard was quite brilliant in the end-user problems it solved. (It would have been wonderfully better with a deep dynamic language underneath, but I think part of the success of the design is that they didn’t have all the degrees of freedom to worry about, and were just able to concentrate on their end-user’s direct needs."

>"HyperCard is an especially good example of a system that was “finished and smoothed and documented” beautifully. It deserved to be successful. And Apple blew it by not making the design framework the basis of a web browser (as old Parc hands advised in the early 90s …)" -Alan Kay

https://news.ycombinator.com/item?id=8546507

>HyperLook was so far ahead of its time in 1989, that there still isn't anything quite like it for modern technology. Since we developed HyperLook and SimCity at the same time, that forced us to eat our own dog food, and ensure that HyperLook supported everything you needed to develop real world applications. (Not to imply that SimCity is a real world! ;)

http://www.art.net/~hopkins/Don/unix-haters/x-windows/i39l.h...

>Who Should Manage the Windows, X11 or NeWS?

>This is a discussion of ICCCM Window Management for X11/NeWS. One of the horrible problems of X11/NeWS was window management. The X people wanted to wrap NeWS windows up in X frames (that is, OLWM). The NeWS people wanted to do it the other way around, and prototyped an ICCCM window manager in NeWS (mostly object oriented PostScript, and a tiny bit of C), that wrapped X windows up in NeWS window frames.

>Why wrap X windows in NeWS frames? Because NeWS is much better at window management than X. On the surface, it was easy to implement lots of cool features. But deeper, NeWS is capable of synchronizing input events much more reliably than X11, so it can manage the input focus perfectly, where asynchronous X11 window managers fall flat on their face by definition.

>Our next step (if you'll pardon the allusion) was to use HyperNeWS (renamed HyperLook, a graphical user interface system like HyperCard with PostScript) to implemented a totally customizable X window manager!

https://news.ycombinator.com/item?id=13785384

>>You can't do opengl efficiently over the network. At least xorg can't. Most applications uses opengl these days.

>Sure you can, it's just called WebGL! ;)

>They just added another layer, flipped the words "server" and "client" around, and added more hardware.

>Now you run the web browser client on top of the local window system server, through shared memory, without using the network. And both the browser and GPU are locally programmable!

>And then the local web browser client accesses remote web servers over the network, instead of ever using X11's networking ability.

>One way of looking at it in the X11 sense is that a remote app running in the web server acts as a client of the local window server's display and GPU hardware, by downloading JavaScript code to run in the web browser (acting as a programmable middleman near the display), and also shader code to run in the window server's GPU.

>Trying to pigeonhole practices like distributed network and GPU programming into simplistic dichotomies like "client/server," or partition user interface programming into holy trinities like "model/view/controller," just oversimplifies reality and unnecessarily limits designs.

https://news.ycombinator.com/item?id=20183838

>Imglorp> Hi Don! Tabs aside, NeWS got so many things right, and not just at the UI but the underlying idea of code instead of bitmaps. I wonder if we were going to start afresh with Wayland/X11 now, bare hardware, what place do you think the NeWS lessons would have? Would a modern NeWS be on top of PS or is there something better now?

>Hi Imglorp!

>Simply use a standard JavaScript / WebAssembly / WebGL / Canvas / HTML based web browser as the window system itself! And use WebSocket/SocketIO/RTP/HTTP instead of the X-Windows or VNC protocols.

>Microsoft kinda-sorta did a half-assed inside-out version of that with Active Desktop, but Internet Explorer wasn't powerful or stable enough to do it right, and it didn't eliminate and replace the whole Win32 / MFC layer, which misses the main point.

https://en.wikipedia.org/wiki/Active_Desktop

>There was recently this discussion about a browser based Mac window manager project (now offline), and there have been others like it (Slate), but the ideal goal is to completely eliminate the underlying window system and just use pure open web technologies directly on the metal:

>Show HN: Autumn – A macOS window manager for (Type|Java)Script hackers (sephware.com):

https://news.ycombinator.com/item?id=18794928

>Site archive:

https://web.archive.org/web/20190101121003/https://sephware....

Comments:

https://news.ycombinator.com/item?id=18797587

>I was also quite inspired by Slate. Unfortunately there hasn't been any activity with it for about 5 years or so. It's great you're picking up the mantel and running with it, because the essential idea is great!

https://news.ycombinator.com/item?id=18797818

>Here are some other interesting things related to scriptable window management and accessibility to check out:

>aQuery -- Like jQuery for Accessibility

https://web.archive.org/web/20180317054320/https://donhopkin...

>It would also be great to flesh out the accessibility and speech recognition APIs, and make it possible to write all kinds of intelligent application automation and integration scripts, bots, with nice HTML user interfaces in JavaScript. Take a look at what Dragon Naturally Speaking has done with Python:

https://github.com/t4ngo/dragonfly

>Morgan Dixon's work with Prefab is brilliant.

>I would like to discuss how we could integrate Prefab with a Javascriptable, extensible API like aQuery, so you could write "selectors" that used prefab's pattern recognition techniques, bind those to JavaScript event handlers, and write high level widgets on top of that in JavaScript, and implement the graphical overlays and gui enhancements in HTML/Canvas/etc like I've done with Slate and the WebView overlay.

>Web Site: Morgan Dixon's Home Page.

http://morgandixon.net/

Web Site: Prefab: The Pixel-Based Reverse Engineering Toolkit.

https://web.archive.org/web/20130104165553/http://homes.cs.w...

>Video: Prefab: What if We Could Modify Any Interface? Target aware pointing techniques, bubble cursor, sticky icons, adding advanced behaviors to existing interfaces, independent of the tools used to implement those interfaces, platform agnostic enhancements, same Prefab code works on Windows and Mac, and across remote desktops, widget state awareness, widget transition tracking, side views, parameter preview spectrums for multi-parameter space exploration, prefab implements parameter spectrum preview interfaces for both unmodified Gimp and Photoshop:

http://www.youtube.com/watch?v=lju6IIteg9Q

>PDF: A General-Purpose Target-Aware Pointing Enhancement Using Pixel-Level Analysis of Graphical Interfaces. Morgan Dixon, James Fogarty, and Jacob O. Wobbrock. (2012). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '12. ACM, New York, NY, 3167-3176. 23%.

https://web.archive.org/web/20150714010941/http://homes.cs.w...

>Video: Content and Hierarchy in Prefab: What if anybody could modify any interface? Reverse engineering guis from their pixels, addresses hierarchy and content, identifying hierarchical tree structure, recognizing text, stencil based tutorials, adaptive gui visualization, ephemeral adaptation technique for arbitrary desktop interfaces, dynamic interface language translation, UI customization, re-rendering widgets, Skype favorite widgets tab:

http://www.youtube.com/watch?v=w4S5ZtnaUKE

>PDF: Content and Hierarchy in Pixel-Based Methods for Reverse-Engineering Interface Structure. Morgan Dixon, Daniel Leventhal, and James Fogarty. (2011). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '11. ACM, New York, NY, 969-978. 26%.

https://web.archive.org/web/20150714010931/http://homes.cs.w...

>Video: Sliding Widgets, States, and Styles in Prefab. Adapting desktop interfaces for touch screen use, with sliding widgets, slow fine tuned pointing with magnification, simulating rollover to reveal tooltips:

https://www.youtube.com/watch?v=8LMSYI4i7wk

>Video: A General-Purpose Bubble Cursor. A general purpose target aware pointing enhancement, target editor:

http://www.youtube.com/watch?v=46EopD_2K_4

>PDF: Prefab: Implementing Advanced Behaviors Using Pixel-Based Reverse Engineering of Interface Structure. Morgan Dixon and James Fogarty. (2010). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '10. ACM, New York, NY, 1525-1534. 22%

https://web.archive.org/web/20150714010936/http://homes.cs.w...

>PDF: Prefab: What if Every GUI Were Open-Source? Morgan Dixon and James Fogarty. (2010). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '10. ACM, New York, NY, 851-854.

https://web.archive.org/web/20150714010936/http://homes.cs.w...

Morgan Dixon's Research Statement:

https://web.archive.org/web/20160322221523/http://morgandixo...

>Community-Driven Interface Tools

>Today, most interfaces are designed by teams of people who are collocated and highly skilled. Moreover, any changes to an interface are implemented by the original developers and designers who own the source code. In contrast, I envision a future where distributed online communities rapidly construct and improve interfaces. Similar to the Wikipedia editing process, I hope to explore new interface design tools that fully democratize the design of interfaces. Wikipedia provides static content, and so people can collectively author articles using a very basic Wiki editor. However, community-driven interface tools will require a combination of sophisticated programming-by-demonstration techniques, crowdsourcing and social systems, interaction design, software engineering strategies, and interactive machine learning.

Screensharing on wlroots compositors has been possible since May 2020 (the first release of xdpw).