Earlier quoted context omitted.
You build custom retained-mode APIs on top of the immediate mode APIs—they're called game engines. What happens if you try to present an immediate mode API for UIs is the status quo with APIs like Skia-GL. You frequently end up switching shaders and issuing a new draw call every time you draw a rectangle, and you draw strictly in back to front order so you completely lose your Z-buffer . Imagine if games worked like…
Ehh, game engines are not really retained-mode in the way you mean. There isn't usually a cordoned-off piece of state that represents visuals only. Rather, much of that state is produced each frame from the mixture of state that serves all purposes (collision detection, game event logic, etc). "What happens if you try to present an immediate mode API for UIs is the status quo with APIs like Skia-GL." I don't know wha…
pcwalton seems to be presuming that part of the contract of an "immediate mode API" is like old-school ones it actually immediately draws to the frame buffer by the end of the call.
Whereas you are talking about modern "immediate mode API"s where the calls just add things to an internal data structure that is all drawn at once, avoiding unnecessary shader switches etc. IIRC this is how Conrod (Rust's imgui library) and https://github.com/ocornut/imgui work, although with varying levels of caching.
One point to make about retained mode GUIs is I remember reading an argument that immediate mode is great for visually simple UIs, such as those in video games, but isn't as good for larger scale graphical applications and custom widgets. For example when rendering a large text box, list or table you don't want to have to recalculate the layout every frame so you need some data structure that sticks around between frames specific to the widget type, so that's what retained mode APIs like Qt do for their widgets.
Sure you can do the calculations yourself for exactly which rows of a table are currently in view and render those and the scrollbar with an immediate mode API, but the promise of toolkits like Qt is that you don't have to write calculations and data structures for every table.