The bottleneck of UI is not the rendering. A measly 60 fps is plenty fast for UI that feels immediate. We had this in the 90's with software rendering, you don't need a GPU for that today. What causes user interfaces to hick up is that it's too easy to do stuff in the main UI thread. First it doesn't matter but stuff does accumulate, and eventually the UI begins to freeze briefly for example, after you press a button…
I think the bottleneck comes from updating each UI element instead updating them in batches and updating elements that don't need to be updated.
If you want to do this, where can you start? What are some patterns for making code that's not too spaghetti when you have to handle tabbing, focus, layout, speech of element contents, the actual hierarchy of the elements etc? Are there standardized OS accessibility API hooks or something?
I’ve heard it’s hard to even work this out as all of the screen reader tools are expensive, proprietary, and there are no standards. The typical way is to just make your program, and if it gets popular, the screen reader companies will find a way to make their product work.
That's partially true, but fortunately not completely. There are widely use open-source screen readers for Windows and, of course, there's no proprietary screen reader on Linux.
And, definitely, there are standard APIs which are used to communicate the accessibility tree between an app and a screen reader. Yes, they are specific for each platform, and Windows has multiple of these, but they are standardized at least for each platform.
If you want to do this, where can you start? What are some patterns for making code that's not too spaghetti when you have to handle tabbing, focus, layout, speech of element contents, the actual hierarchy of the elements etc? Are there standardized OS accessibility API hooks or something?
I’ve heard it’s hard to even work this out as all of the screen reader tools are expensive, proprietary, and there are no standards. The typical way is to just make your program, and if it gets popular, the screen reader companies will find a way to make their product work.
> work this out as all of the screen reader tools are expensive, proprietary, and there are no standards.
ARIA is a good start, and screen readers built into the OS are a good start.
Moreover, major OSes have accessibility APIs that screen readers will use:
"Inspired by the gaming world, we realized that the only way to achieve the performance we needed was to build our own UI framework" I'm surprised you did not look at "Dear ImGui", "Noesis", and "JUCE". All three of them are heavily used in gaming, are rather clean C++, use full GPU acceleration, and have visual editors available. Especially JUCE is used for A LOT of hard-realtime professional audio applications. "Wh…
Maybe you need to add an "(in Rust)" to all these sentences? Sure there are C++ frameworks, but they probably wanted a pure Rust UI framework? My 2 cents: it's nice to have smooth rendering in your editor, but I'm currently mostly using a Java-based IDE (IDEA family), and it's responsive enough for my taste. If I were to use the current prototype of their editor, I'm afraid the usage of fixed-width fonts all over the…
"Inspired by the gaming world, we realized that the only way to achieve the performance we needed was to build our own UI framework" I'm surprised you did not look at "Dear ImGui", "Noesis", and "JUCE". All three of them are heavily used in gaming, are rather clean C++, use full GPU acceleration, and have visual editors available. Especially JUCE is used for A LOT of hard-realtime professional audio applications. "Wh…
ImGui is at least only used for debug rendering, not something that makes it to end user. At least in the small subset of companies I worked at.
Maybe you need to add an "(in Rust)" to all these sentences? Sure there are C++ frameworks, but they probably wanted a pure Rust UI framework? My 2 cents: it's nice to have smooth rendering in your editor, but I'm currently mostly using a Java-based IDE (IDEA family), and it's responsive enough for my taste. If I were to use the current prototype of their editor, I'm afraid the usage of fixed-width fonts all over the…
Not true, even if you add Rust. egui was released earlier: https://github.com/emilk/egui And egui isn't tied to Apple proprietary frameworks.
Afaik egui isn't doing any kind of the fancy GPU based 2D graphics this blog post is about though.
While I do enjoy a nice and smooth gpu-accelerated ui, I never use a gpu-ui framework for my own project for one simple reason: Almost none of them properly support accessibility. Electron (and in general the web), despite its sluggishness has a very good support for accessibility. Most "traditional" native ui toolkit also do. That would be my advice to anyone making a gpu-accelerated ui library in 2023: Try to suppo…
The Rust GUI ecosystem looks particularly promising in that regard, because there is a foundational accessibility library called Accesskit that's being incorporated in several UI frameworks (egui being the first one to have it already but works is underway do add it in several other places)
"Inspired by the gaming world, we realized that the only way to achieve the performance we needed was to build our own UI framework" I'm surprised you did not look at "Dear ImGui", "Noesis", and "JUCE". All three of them are heavily used in gaming, are rather clean C++, use full GPU acceleration, and have visual editors available. Especially JUCE is used for A LOT of hard-realtime professional audio applications. "Wh…
ImGui is at least only used for debug rendering, not something that makes it to end user. At least in the small subset of companies I worked at.
You are correct. I've also encountered it sometimes in internal business gui wrappers.