Could ImGUI Be the Future of GUIs?

I've written real-time game UIs before so I think I have some relevant experience here.

1. It is very possible to write a retained-mode GUI in a graphics API like DirectX or OpenGL. In fact, a retained GUI would typically wipe immediate GUIs in terms of performance in this context. In immediate mode, the GUI's vertex buffers need to be completely reconstructed from scratch every single frame, which is slow, CPU bound, and cannot be (easily) parallelized. It's like reconstructing the game world every frame -- that would be ludicrous for any non-trivial game.

2. I don't think there would be that much of a difference between the two UI models, since data updates can be dispatched from the event loop. It would be faster, too, because only UI components that need updating could be redrawn. This is far faster than updating the entire UI every single frame.

3. As mentioned earlier, immediate mode GUIs are going to be a lot slower than retained mode, when implemented properly. Immediate mode GUIs put most of the work on the CPU instead of offloading most of the work to the GPU like in the retained model.

I think developers that are using immediate mode GUIs are doing so because of their ease of use. I think retained mode is typically harder for a game developer to conceptualize because immediate mode is conceptually similar to a game loop. Also, I don't know of any free & open source retained mode GUIs for DirectX and OpenGL and the like.

Also, DirectX at least (and probably OpenGL) encourages a retained-like model for general rendering. The only way to get decent performance is to re-use vertex buffers between frames and only update them when something changes.

I didn't say that it's not possible to solve these problems in a retained GUI, just that existing ones, QT, Win32, WPF have these problems.

And since writing a full retained GUI is not exactly trivial, people just wrote mini-GUIs using immediate mode.

When I talked about retained mode APIs, I was thinking about scene graphs, where you say "addMesh" or "addSphere" and then just call "renderFrame". I'm aware that most game engines implement their own scene graph anyway, but it's game specific, not some generic one provided by the OpenGL/DX API.

It will. In fact, even a simple one will require this. But it's not much of a problem - 99% of the time, the UI is the same from one frame to the next, and it isn't much work to detect this. So even if every change requires a complete rebuild of everything, it's not much of a problem.

(I've written systems like this for Cocoa and Win32, and it never turned out to be necessary to do anything other than just regenerate the entire UI any time anything changed. The update runs at 30Hz or 60Hz, and when anything is changing, the UI gets regenerated a lot! - but so what? Most of the time, the UI doesn't get regenerated at all. Then something happens, and the code spends 2 seconds getting absolutely hammered continuously, malloc malloc malloc malloc malloc, god help us all... and then, once again, nothing. The operator puts their fingers to one side and stares at the result with their eyes. Repeat.)

Every game I've worked on didn't actually use an IMGUI for anything you'd ever show a player; (unless it's a style of game that really doesn't need much UI), it's just custom gui implementations.

A lot of modern games actually put the UI in with the scene graph of the game itself (IE: Unity's new GUI system)

I feel that the future lies in combining retained and immediate interfaces, preferably with the granularity to allow deeply nested retained interfaces that are fast for complex ui (or for a realtime system with memory to spare), whilst still allowing one to go the other direction, such that a simple ui can be written cleanly and logically for low-memory systems (such as embedded or boot guis). It would need a very well designed api for this, but I feel the benefits are worth the effort (and I'll probably look into this next time I have the freedom to choose ui apis).

A balance may be letting people define it either way, so that manually written ui still can have auto-layout yet intuitive code (following control flow primitives), whilst allowing generated retained uis to be manually editable -- perhaps even allowing one to then embed one within the other, a boon for scripted interfaces that perhaps have people of various levels of experience producing ui elements, such as a musician with little experience being able to add a simple visualiser in an immediate manner to a deeply retained daw gui.

Of course, there's a lot here that is implementation, and some criticism either way can be optimised out. Immediate mode can still cache its rendering, we've had optimised blitting since the early days, and is only usually a problem with complex ui. Retained would get fewer cache misses if we weren't allocating madly across the heap and took a more disciplined approach allocating to contiguous memory -- which is almost entirely a language/api problem (in my experience) that can also happen with immediate but we typically don't see since it's often done in a more procedural style that is allocating to some pool.

Other api elements, such as handling lists etc aren't really a differentiation between retained and immediate, those can be made in either.

For me, I often find that the ability to write out prototype ui code in an immediate style in very quick and satisfying (exactly what I want in prototyping), however once I start to expand upon a ui, I find it best to over time refactor towards a retained style, since by then I will typically have some templates for what ui elements look like, and so I just have to pass a string and function pointer to fill in the template.

Can't see why we can't have nice things and let both coexist...