Thoughts on Rust GUIs

libs
1

This thread is a snapshot of some recent conversations about Rust GUIs. This is still a few steps away from a plan, but hopefully we can generate some good discussion around these ideas and what we might explore next.

Rust needs a strong solution for GUI programming. You can work with various existing GUI toolkits, but these are generally fairly thin wrappers over existing systems that have their own learning curve. This prevents you from easily moving your app to another platform, or even reusing the knowledge of one platform on another.

Finding the right approach for Rust

Flora Cat (@FloraCatz on twitter) has been exploring what it might mean to try to map Rust on OOP GUI styles. Unsurprisingly, it appears this is a bit tricky. Rust’s ownership model makes it difficult to model the circular references that some GUI frameworks allow.

The question then is “what’s a more natural way to model a GUI in Rust?” There’s some recent work here that builds on reactive programming. React and Elm give a way of working with events and state that is closer to a functional approach. Elm’s approach takes in the state of the world, and rather than manipulating it directly, you work with the Elm system to handle each update to state, side effect, or even multi-step process. While there’s a lot to be learned working in this model, it too isn’t an exact fit with the more pragmatic Rust approach.

Still another approach could use the channel mechanism that comes with Rust. Rather than GUI components owning each other, or having a runtime reactive system, the channel mechanism would allow components to message back to the programming logic and vice versa. Some research would need to explore if the overhead created by channels pays for itself.

Native vs One-size-fits-all

There’s a fork in the road early in the design of a GUI: should it work directly with the GUI elements that are native to the platform, or should it elect to have a more uniform, one-size-fits-all approach where the app works and looks the same regardless of the platform? There are pluses and minuses to each approach.

Native GUIs

Native GUIs retain the look-and-feel, and will instantly be familiar to users of the platform. Additionally, relatively tricky nuance like how fonts render, how text input works (including multi-language input), keyboard shortcuts, and more will match what the user expects, reducing friction.

This is approach is partly served by the existing native GUI wrappers. That said, there’s an opportunity for a library that works across these native wrappers and offers a Rust-like, single interface to GUI development. The resulting code could then more easily port between platforms.

Creating a universal wrapper gives the programmer a universal API, but is not without its drawbacks as the resulting API is the intersection of the supported platforms. To get the full capability of the platform would need extensions (see below).

One-size-fits-all

Java, wxWidgets, qt, and more have created a more uniform platform that may look similar to the native platform but is built from a more central design.

This approach is less prone to breakage of the underlying platforms, as it only wraps them where necessary. On the other hand, the maintenance cost likely moves to keeping up-to-date with the changes in the native GUIs.

Extensions and crates.io

Rust has a strong friend in cargo and crates.io when it comes to extending existing applications with more functionality. Similarly, the GUI toolkit has a need for optional capabilities. If someone codes up a new widget, it should be easy to share with others on crates.io. Likewise, if the GUI works a bit more universally, having platform specific features and additions will be welcome, and if they’re easily found on crates.io, all the better.

Having cake and eating it, too

I mentioned earlier that there’s a fork in the road between creating native widgets and a more one-size-fits-all approach. Perhaps this GUI library could take a similar approach to some of the thoughts around the compiler and new platforms. Maybe the one-size-fits-all GUI is just another platform you can target with the library: eg) gtk, windows, macOS or rust-ui.

Rough sketches but no solid answers

I’ve cobbled together the notes from some of the recent twitter discussions, but I’m curious to hear other folks thoughts. What approaches are better? What platforms should be included first (ie, should we include web-based UIs or focus on native platforms)? Are there some key application areas it should work well in first (this might help us answer the previous question)? Do you have your own GUI experiments you want to share the results of?

31 Likes
3

I think this and @FloraCatz's gist are a great summary of the problem space and the potential solutions. The main thing I'd like to add is that I think there will be, and should be, multiple ways to do things here. Trying too hard, especially early on, to make a single API that supports native wrappers and a custom solution will lead to too many compromises.

As far as what approach we should try first, I think the one with the most impact would be a new platform designed around idiomatic Rust.

Native GUIs all tend to be designed around their native language, e.g. C# for WPF, C#/COM for UWP, Swift/Objective-C for Cocoa, C for GTK, C++ for Qt/KDE, Java for Android, Dart for Flutter. For those of the school of thought that GUIs should be redone for each platform, Rust is well-positioned to implement shared business logic. For those who prefer native GUI wrappers, Rust still needs a lot of work around things like bindgen to be able to talk to C++, C#/COM, etc, and that would be a distraction from a good Rust GUI API.

While making a good native-feeling look and feel for a new platform is a lot of ongoing work, the underlying API design can happen without it. We need to figure out idioms for state and event handling, for custom controls, for themeing, and for integrating with OS accessibility/automation APIs. That's necessary regardless of whether the end product tries to match the host OS or not, so it serves the most diverse userbase and can even help determine how a native GUI wrapper should work once Rust is in a position to have one of its own.

Another reason for a new platform: many native GUIs are based on software rendering, and those that do have hardware acceleration don't make very good use of it. It would be ideal for these new idioms to take into account the research done on Servo and Quantum. Rust should have a way to use those techniques in native apps, without pulling in an entire web engine saddled with Javascript as its data model.

(Small nit: wxWidgets is a native GUI wrapper, unlike Java's Swing or JavaFX, or Qt.)

16 Likes
4

I think these posts summarize the issue at hand very well. Given that Rust’s type system is closer to Elm’s than it is to Java/C/C++/etc., I think it could be really interesting to do some exploration around the Elm Architecture. One project that is already doing that and has gained quite a bit of notoriety is yew.

One thing I really want to stress is accessibility. Having it in the design from the beginning will make it so that creating accessible applications is natural and most importantly by default. Implementing accessibility is not always trivial on every platform, but thinking about how it would be integrated is very important, even if it won’t be a feature right away.

Here’s some info about that for various platforms:

Accessibility should be something that the GUI toolkit provides by default with the ability to optionally enrich the accessible experience with more metadata.

24 Likes
5

How about we design a system of traits that define the capabilities of a GUI in a platform-independent manner. Then, platform-specific supports can be added by providing impls for these traits.

This allows many independently developed implementations to coexist as separate libraries on crates.io. One can also create a simple one-fits-all software rendered proof-of-concept.

5 Likes
6

I’m not sure it makes sense to design the traits without the (full, production-quality) implementation.

I’m also not sure a fully trait-based API is the way to go, for that matter. There’s rarely a reason to have multiple implementations of a GUI in the same program.

1 Like
7

What about just using HTML and having a framework that can either deploy to the web by compiling the Rust code to WebAssembly and using web APIs for 3D/sound/etc., or produce a desktop application by bundling Servo and using the native APIs directly, plus mobile support either with embedded Servo, with a WebView or something like react-native (or just react-native itself)?

The advantage over any other approach is that this allows to have a single codebase and support the web and mobile (that wouldn’t be possible using GTK/wxWindows/etc., although Qt supports mobile but not the web) and web and mobile technologies have the most developer mindshare, and thus would attract the most developers to Rust and also allow to reuse a bunch of JavaScript UI code if desired.

Regarding the “circular references”/etc. approach I think a React-like framework is the best way to solve it.

Event handling can be performed by having each level in the stack provide a closure that merely wraps the event in an enum (and thus doesn’t need to own any reference except an immutable one to the event-handling closure in the passed by the parent component, which can be owned or Rc depending on whether there are multiple child closures or not) and then all event handling can happen at the top level.

For instance, if you press the “done” checkbox in a TodoMVC-like app, it would start as a Click sent to the closure on the checkbox, which would turn it into a TaskEvent::TaskDone, and then into a TaskListEvent::TaskEvent(task_num, TaskEvent::TaskDone), then AppEvent::TaskListEvent(TaskListEvent::TaskEvent(task_num, TaskEvent::TaskDone)) which would then be handled at the app level by passing it back down to a component owning the state and then passing it down through the model’s structs/enums.

1 Like
8

One thing I’ve noticed is if you want to create a non-trivial GUI program it helps to have a full blown visual designer and IDE to do it in. A good example would be Visual Studio (C#) and Qt Creator (C++). I’ve had a decent amount of experience with the applications we create at work and it’d be extremely painful to manage a 100+ kloc enterprise app without an IDE that is deeply integrated with the language and has a visual designer.

We don’t really have a mammoth IDE which is the environment for writing Rust code in, instead most people prefer to use their favourite text editor (e.g. vim, VS Code, emacs). This is one of the things I really like about Rust because I’m a massive vim fan, but it does tend to make things more difficult in the GUI space.

In general I’d agree with the previous comments about creating an idiomatic GUI framework which fits in with Rust’s ownership model. As people have already mentioned, React-style “DOM” approach would be a good fit. I feel like you’ll have a bad time trying the “standard” GUI approach where components get back-references to their parent widgets and there’s loads of shared references. Your application probably won’t compile without loads of Arc's, but reference-counting is just a poor man’s GC and it leaves you open to memory leaks via cycles.

3 Likes
9

I don't personaly use QML, only Qt Widgets, and for Qt Widgets I don't see big difference between standalone Qt Designer tool and similar tool inside Qt Creator.

If return back to Rust GUI framework, it should of have something similar to Qt/.ui or Qt/.qml, or C#/xaml. But in compare to xml/javascript/xml it should be DSL based on Rust macroses, with strong typing. Thus, standalone tool like Qt Designer and for example VS Code can share connection to RLS as backend to give functionality comparable with IDE all in one.

10

A interesting alternative is to have live-realoding for GUI code. I belive ClojureScript, Elm and Flutter can do this. This seems to require some sort of runtime for GUI though :frowning:

A craziest idea would be to create a low-level rendering library in Rust (webrender + path finder + something for layout & styling?), a WASM run-time with bindings to it (cretonne?) and finally create a pragmatic FP (or perhaps just reuse Elm?) langauge to write GUIs in :slight_smile:

8 Likes
11

I agree that having the ability to use a WYSIWYG for at least some of the work is an important requirement. Maybe not initially, but keeping the goal in mind that some of the UI design can come from external sources other then just pure rust code is important.

We should also probably have something that functions like CSS. Having a common place where you can change colors and font choice, padding ect w/o rebuilding the entire project is exceedingly helpful. It also allows designers to easily tweak things w/o having a developer at hand.

Since we are already aiming for cross platform, it makes sense that we could probably even do fancy things like embed the framework in the browser. Not to dream too big, but being able to place DOM elements using the same code that uses native elements on a desktop would be pretty helpful for a number of niches.

That being said, I would probably stick to building out the UI for a single platform that people are comfortable with before trying to untangle how to make it easily cross-platform after we felt we had something good. Even if that means being nothing more then a GTK or UIKit wrapper for the first year of the project’s incubation period. Less complexity makes the important bits of the API easier to experiment with.

Yaw looks really cool, I’m actually thinking that could be a much easier route to go down. Since it has an API that is more or less working today! Having the UI draw top down like that gets ride of all the issues with cycles that I discusses in my rant.

5 Likes
12

That's a very good point. For each of these where we connect to it, we'll always have to work to make a Rust-like API to connect to their more native approach, which may or may not be a perfect fit.

Still, my hunch is that this is at least worth trying, as the possibility of having native GUIs is attractive.

Or, at the very least, standing up the traits along with an implementation (like Futures and Tokio). I agree, though, that you can't build the traits without building the implementation and knowing how, for example, events and the like will work.

I definitely agree, though I'm not skilled in this area. We should pull in people who do have this expertise early in the process.

I see a set of tradeoffs when I think about native GUIs vs using the web browser.

The browser gives us a few things we definitely want: a lot of developers who are already familiar with how to write web apps can use that knowledge to write web apps. Design tools already understand HTML. There are some disadvantages, too. Web browsers will make up a substantial part of the application in terms of memory usage, load times, and also require a fairly extensive knowledge to know how to use well.

A native, Rust-focused GUI can be more streamlined, with a focus that helps put people onto a happy path without having to know about HTML, JS, CSS, and their respective quirks. As @Flora points out, you'd likely still want some of the capabilities common to the web. Being able to style the page quickly lets you improve the UI without waiting through full edit-compile-view cycles.

I don't necessarily see it as either-or. Like others mentioned, reusing pieces of web tech like WebRender, Pathfinder, and others let us leverage code that's already being used in other projects.

That said, I suspect going fully in the web direction would look like wasm+web engine+electron (or electron-like library), which is a bit different than some of the motivating ideas around making something that works well on platforms directly. Not that one is bad or good, just a different direction.

You know... you joke about this later in your message, but there are ways we can do this. I've experimented with one called Rhai. It lets you expose functions and values to a tiny runtime, and these symbols would be visible from inside the scripting language. A similar approach could potentially work with the GUI library, which would let you have some of the flexibility that @Flora mentions.

It's not quite a live reload, but it's a step in that direction.

Oooh, interesting. I'd seen this before but hadn't made the connection. Is this because you can see more of the connections at one time? Not sure I understand all the implications, but I do think it's a very cool looking direction.

6 Likes
13

If you are interested in the Elm Architecture or React, you may want to check out this page on the Flux design pattern. I especially recommend watching the video which shows how having a unidirectional data flow helps deal with some of the pitfalls of traditional MVC. Lots of really popular ways to write UI like React and Redux have been based on ideas like the ones in that video. The elm architecture is really easy to understand if you already understand Flux because it is (in a way) an implementation of that design pattern. Redux is another take on Flux and Elm which has its own benefits and drawbacks.

I’ve worked with all of these technologies over the past decade and I’d say that it would be great for anyone interested in any of these to start experimenting with different approaches and then see what works. There is definitely a lot we can draw from mature platforms like the web, but there is still no silver bullet. My gut says that something like the Elm Architecture would be best, but we may discover something else that works even better.

So please, if you have any ideas at all, try to start building it and get other people on board to help out. Let’s make these ideas into reality. :blush:

3 Likes
14

I’m also not sure a fully trait-based API is the way to go, for that matter. There’s rarely a reason to have multiple implementations of a GUI in the same program.

For the main program, no, although there might be cases where you want to call a stub implementation from unit tests.

15

As I haven’t seen it mentioned in this thread, I’d like to point at Relm, a nice GUI library based on GTK+ that implements the Elm Architecture for desktop apps and looks like it’s already very usable.

Relm’s github webpage has a list of applications already using the library and an API overview.

6 Likes
16

I would prefer unit tests work via common types and data structures rather than common traits. It would be unfortunate to force everything into traits and the limitations they bring just for mocking.

17

I would much prefer the one-size-fits-all approach. I’ve tried s.th. like that some months ago:

It’s called Rune (Really Unique iNtErface or Rust User iNtErface) and unfortunately I didn’t have much time to continue to work on it.

Currently only a window and a simple push button is supported. There is even a branch where I have implemented a message passing approach to avoid the inheritance / ownership problems that Rust has.

It’s based on SDL so it should be easy to port to other platforms (*BSD, Haiku, Redox, mobile, …), no OpenGL yet but that should be possible. In the end we just need to be able to put pixels on the screen.

I think conrod has a similar approach but is much further advanced.

Using native GUIs has the problem that not all platforms support all features / widgets so you have to decide on a mimimal set of features. And native GUIs are a moving target so maintainance is much higher than with the same custom widgets on all platforms. Just my thoughts…

18

Random not-very-insightful thoughts:

  • The solution needs a good accessibility story. While this is “citation needed” territory, I recall some project rejecting Gtk+ as a cross-platform toolkit, because the accessibility support wasn’t cross-platform where painting and mouse events were.

  • It might be worthwhile to examine what Eclipse SWT and LibreOffice VCL do.

  • There are likely very different requirements within one app: e.g. a preference pane that needs a bunch of native widgets and some visualization window where the primary concern is getting at OpenGL/Vulkan with minimal overhead.

  • Swing has “pluggable look and feel” (PLAF). In practice as far as platform-specific things go, feel isn’t really pluggable and plugging look is hard, so Swing isn’t great. Still, if the solution doesn’t go with native widgets per se, it’s worthwhile to look into painting the custom widgets with the native theme engine. Otherwise, there are user cohorts who’ll reject the new toolkit. The behaviors are likely to require platform-specific code for Windows/Mac/Gnome/KDE/Android/iOS.

  • Please also accommodate platform-specific widget spacing.

  • Inevitably someone wants to do a custom widget, which will look bad on some platforms. Hard problem. Custom tab bars seem to be a particular problem.

6 Likes
19

Apologies for the slow responses, though I want to see if we can keep the discussion going.

Maybe a quick question for those interested:

  • Should we start a repo for our experiments?
  • Would gitter be a better place to chat (like the working groups use)? Is there another place folks would feel comfortable brainstorming?

Sounds like there are some existing projects we’ll want to try working with ourselves and make notes on what works.

20

This would be equivalent to either Java AWT, or SWT depending on how you approached it. SWT has been pretty successful with Eclipse, but, never really took off outside that ecosystem. Looking at Eclipse SWT (which is really JNI native wrappers around native API's for the most part, or at least began life that way). It generally results in some kind of "Lowest Common Denominator" that isn't all that useful.

This alone argues for not trying to wrap the underlying native OS toolkit because that just creates huge amounts of dependencies that buy you nothing from a maintenance standpoint. Begin able to do:

cargo install some-rust-gui-program

and have it just download the GUI core library and any specialized components it uses would be better than anything currently in existence. Hands down!

Forget about native GUI toolkits. Trying to create a universal wrapper has been proven a fools errand time and again. The only where "native" really become relevant is things like very "High Level" dialogs like "File Selector Dialog". These could be provided in the "One Size Fits All" world by a widget that is "Wrap a Native Widget" type of thing. So, you could have a File Selector Dialog widget in the "One Size Fits All" that would try to be super awesome on all platforms, but, you could provide alternative "File Selection Dialogs" that wrap native "File Selection Dialog" for those who wanted applications that used the native one (for whatever reason). Even this, to me, seems counter-productive, as just making the "Rust Awesome GUI" file selection dialog be "Awesome" and as good, or better, than anything on any native platform would be more useful.

21

Also, have you taken into account the work on OrbTK and Orbital in Redox?

7 Likes