Thoughts on Rust GUIs

libs
23

What do you think about React Native approach? Basically, they provide React as API/engine which works the same on all the platforms, and a set of bridges to native components (there is no guarantee that all the components will present on all the platforms). Their mantra is "learn once, write anywhere", which IMO works great for mobile platfroms (and they show competing performance with the pure native applications).

Flutter uses another approach; they went through a hassle of reimplementing all the components from scratch in Dart (AFAIK Qt went the same way). It is definitely not a short-term project, but it gives a great portability since they control all the drawing and, thus, can have a better portability to exotic platforms (e.g. with such an approach, we don't need to implement a whole set of bridges to all the components for every single platform, instead, we only need to adapt the drawing engine and we can run e.g. Mac OS look application on Redox), but implementing a native feel theming and accessibility will be a tedious task.

There is a project called Toga (written in Python), which follows React Native ideas targetting desktop OSes first (though they have experiments with mobile and web platforms) and it seems that they are doing great (though, they don't have enough contributors to polish the project to the level of React Native in terms of feature-completeness and documentation).

I like React Native / Toga approach the best, and I believe it is an order of magnitude easier to implement a single API with bridges to native components than reimplement the whole drawing from scratch.

P.S. Another project I want to bring to this discussion is Yoga, which is a cross-platform layout engine which implements Flexbox. It may or may not help someone with an implementation of layout system.

1 Like
24

First, i just wanted to say that my last post was quite negative. I don't intend to devalue projects that follow a paradigm i don't like, so i'll answer this post and try to expand a little bit.

I haven't worked with react native specifically yet (but have worked with toolkits that are comparable in this regard), so take this with a grain of salt: The problem is that this prohibits a lot of the stuff some applications need. Many applications tweak native components (often that happens through subclassing, and that does not fit into this model). Native components on different platforms may be not equivalent for all the usecases of a specific application.

If an application fits into the common denominator of all standard UI-components and no further control is needed, this can work out great. But i cannot imagine writing a complex, UI-heavy program like a digital audio workstation or an image manipulation program that way.

Side note: maybe this is in indicator that the problem space of this discussion is underspecified. What are the programs that our hypothetical GUI-framework should enable?

I've worked with QT for a few years for a job, and i've a ton of appreciation for the project. But it just never looks quite right... a little bit like the uncanny valley. That can be jarring or even confusing, when a component that mimics a native one behaves differently in subtle ways. Also, like you said, it's an unholy amount of work. If a project like QT, with all their resources and all their expertise cannot pull it off properly, i'd wouldn't want to try it...

On a general note: Maybe i should backtrack a little bit. One of the assumptions i made, but didn't specify, is that rust niche in the desktop space is more in the "complex" territory. I don't see the "wrapper around common native components" approach work for that, but there is certainly some areas where thats the right solution.

On an even more general note: "We" hackers and computer scientist love abstraction and slicing up the problem space into neatly separated sections where elegant solutions can be found. But (in my experience) UIs, information design, layout in general and similar problems don't let you do that. Content cannot be separated from the form it's presented in, form easily (mis-)communicates as much as the content itself and delightful UI-solutions may require unreasonable amounts of customisation. Imo applications should be able to do that.

8 Likes
25

I too, on reflection, found my original response more negative than I intended. I agree with everything you’ve said above pretty much for the same reasons you’ve articulated.

In my experience, uniformity of application GUI (within reason) is nothing more than marketing ploy (one that every few years advocates some new “paradigm” that is supposedly the be-all, end-all only to be abandoned as “legacy” the next cycle). A GUI library is useful if the following is true:

  • Customizable (styling/theming/user-overridable)
  • Extensible (Application can extend/inherit components and create new components easily)
  • GUI implements widely accepted standards, but, does permit diverging from those standards as needed for the specific application
  • Looks the same (or nearly so) on all platforms (given using the same theme/style/overrides)
  • Permits access to the lowest-levels of the rendering and events stack from application code if necessary (but doesn’t require it)
  • Ability to be themed/styled/customized such that it looks pleasing on and behaves compatibly (withing reason) to the “Native” GUI

What is NOT important:

  • Looking exactly like the “Native” (whatever that is even supposed to mean these days)
  • Behaving exactly like the “Native” GUI
  • Being the actual “Native” GUI (this one is particularly counter-productive in my opinion)
2 Likes
26

My requirements are:

  • Accessible by default. Especially on Windows where all the tooling is there and applications have no excuse to not implement the necessary interfaces.
  • Use native bits wherever possible. Especially for things like file selection dialogs where often the entire thing is pre-made and ready to use with minimal effort, such as IFileOpenDialog on Windows.
  • High DPI support. Everything should be designed in a way that the user has no choice but to write their application in a manner that supports high DPI.
  • Pure Rust. Should not depend on any external libraries other than system libraries.
17 Likes
27

Couldn't agree more. I must admit that i'm out of my depth on that one. All GUI work i did was in industry automation, where assistive technologies where not a big concern, or for the web.

Do you know if it's possible to abstract over the OS capabilities? Of course Windows is a priority here, but the Mac is quite important too, since afaik a lot of people with visual impairments use Apple stuff.

It may be good to have folks who use assistive technologies themselves on this hypothetical GUI project. Also, missing accessibility should be an automatic veto for a feature.

I'd say, use native bits wherever sensible. The file dialog would be the prime example. Also standard key combinations. In the end, it'll be a value judgment on a case by case basis.

My first thought was: "Yeah, that goes without saying". But one has to be very careful to not bake in any assumptions between pixels and actual size.

28

Just to dig in a bit more, what cases would it make sense to be intentionally not the native experience?

29

The cases where one would have to sacrifice portability or flexibility. Take text fields for example. They are commonly extended with syntax highlighting and other live feedback about the inputted text and they are often styled quite heavily. Providing a unified interface for that will prohibit a lot of stuff or will leak platform details. Also, on most platforms they own the content, which could clash with a more functional data-flow based approach and they handle unicode in a way that could inject platform dependent quirks into the whole framework.

30

Using native file selectors and other controls like that is fine. Trying to use exclusively native controls has a lot of downsides- least-common-denominator support, lack of control over the rendering implementation, etc.

Native buttons/textboxes/etc is probably not worth it. Windows doesn't even have a single native version of these at this point- there's Win32 and WPF and UWP, with no clear "winner" in terms of the way forward.

So reimplementing the primary set of controls in a way that can be composed with whatever idioms we come up with for custom controls and state management, usefully hardware-accelerated (i.e. like WebRender or QtQuick, not like WPF or pre-WebRender browsers), and themed using the same APIs across platforms, seems far more desirable to me.

1 Like
31

That might be true for some widgets, where it's just as easy to draw your own. I mentioned textboxes a bit earlier in the thread, and how I found these to be extra tricky to do. Input sources can be different types, plus dealing with multi-byte characters, different kinds of unicode, selection/copy/paste like native, etc. It was just a lot of details to get right. I think @madmalik's comment says something similar (though correct me if I'm wrong). Maybe there's a way to have the native functionality without going all the way to being the native widget?

This is a good point, and why it's even hard to say "native widgets" for a give platform, since there might not even be just one set of native widgets. Same issue on Linux, which doesn't have native widgets so you end up having to pick gtk, qt, or both.

Events are another thing that got brought up early in the thread. Each UI system seems to have its own event system and as you pointed out earlier in the thread, each system is paired up with a language that can work pretty well with it. There are some experiments we need to do here, but perhaps there is a system that feels native to Rust (like the reactive ones that folks have pointed out). And this system would need to work well with the native widgets we do decide to support.

4 Likes
32

The question of text input is a bit more interesting when you consider internationalization- I’m not as familiar with text input APIs as I am with accessibility APIs. I suspect they’re at least as (if not more) applicable to non-native controls as the accessibility APIs are, but it would be worth investigating.

4 Likes
33

To summarise the discussion:

  • there seems to be somewhat of a consensus (?) to not rely on native component (whatever they may be, since that is not always clear), but use them where it makes sense.
  • Accessibility is a major concern. More expertise is needed.
  • There seems to be a push for Flux inspired data model, tailored to rust strengths and weaknesses and not a lot of appetite for object based solution in the QT/Swing/every other GUI toolkit from that era sense

Imo its time to shift the discussion into another avenue? It’s getting confusing here I’ve made a gitter room since that seems to be the preferred medium for working groups. Maybe thats a start?

2 Likes
34

Hi @all. Currently we working with a group of three people from my work on an update for OrbTk. We are one user experience designer and two ui developers (one Qt/Qml developer and one Qt/Qml + WPF developer). Our plan is to create a fast and easy to use pure Rust ui framework. We are strongly oriented towards https://reactjs.org/ and https://flutter.io/, because we think such an approach is good to implement with / for Rust. We’ve started with the planning and implementation last week.

11 Likes
35

Excellent, I'll hop over there as well. And thanks for the summary, I think that's a good starting place.

36

The key, IMHO, is decouple some tasks (kind of separate “back-end” UI from “front-end” UI).

We can do “partial/almost” cross-platform UI, if we think that some stuff can actually cross cleanly:

  • Layout (the big one, IMHO) with something like https://yogalayout.com.

  • A well defined way to separate the back from the front UIs. The ELM architecture is a good contender. The above point solve how communicate the back from the front, yet, is trivial to port and easier to do UI testing without actually touch the graphics.

  • Colors, Fonts, Gradients, Borders, and maybe the specification of common animations, can also cross similar to CSS (where is just text and is responsability of the browser/svg/canvas to actually do stuff).

  • Theming and styling definition, agains, as CSS (but could be better as with LESS)

  • Abstract controls: “Label”, “Button”, “Window” are common terms that no need to actually be a specific control

A big chunk of the logic is totally cross-platform and “only” need to adapt the render of controls. This way of working allow to work with pure objects for the model and the view without actually commit to exactly what is the view UI. Instead, is delegated to the “update” side. It can totally be just in-memory, testeable object.

So, this actually could look similar to HTML/CSS :slight_smile: However, is more abstract, where the actual implementation of the UI is “left to the reader”.

Then finally, the rest can be fully native OR custom-UI OR lowest-common-denominator:

  • Controls/UI Toolkit

  • Drawing

  • Catch and dispatch native events or the UI loop…

  • Animations

  • Call to native libs

This mean that we build components instead of a class hierarchy and is possible to swap what is a “control”, like, from HTML to iOS Views without moving the rest of the logic.

The big question is what to use for coding this. I’m using .net but wish to have something more low-level. Now, I’m using a layout engine in C (https://github.com/xamarin/flex) because Yogas is harder to interface… I think Rust here could become the alternative to electron and provide 80% of the work… let complete the implementation to other projects…

Wish to have a coding partner to do this :slight_smile: but I don’t know Rust… however this and a database engine is a dream of mine…

2 Likes
37

I am a developer currently trying to develop a desktop app in pure Rust, but I just discovered this thread. I am currently working on a GUI solution called azul, which was inspired by the existing framework limn. So far my experience has been:

  • Ditch OOP. It simply doesn’t work in Rust. An HTML-like scene builder works way better because Rust has already crates around HTML / CSS due to servo. It’s also way more flexible when it comes to custom UI elements.
  • Use webrender for rendering, if you’re using non-native UIs. webrender has the prospect of being the most supported way to create GUIs, because of servo.
  • Use CSS for styling. It doesn’t make sense to create a XML or custom-meta-language approach, because people already know CSS and can use it. Pretty much every GUI toolkit has begun as button.with_color(RED); but then switched over to a more flexible / stringified model of CSS a la button { color: red; }. CSS can be hot-reloaded, which is important for fast development.
  • Seperate the UI itself from a canvas / SVG-like drawing element. For drawing custom charts / images, I’d not try to shoehorn it into the UI framework, rather I’d build a seperate API for drawing lines, polygons, SVG elements, etc.
  • Whatever you’re going for, test that it works early. I’ve often seen experimental UI frameworks that didn’t work beyond a Hello-World example. You could call “println!(“Hello World”);” from a UI - but you couldn’t update the UI / data model or do anything else. Such UI frameworks are useless.
  • Even though it is important, I’d think about accessibility last. The first goal is to get something on the screen and make it extensible enough so that accessibility features can be added by just upgrading the library.

azul features a simple CSS parser, similar to what LCUI does. This was because existing CSS parsers were too complex for my taste. In my approach, I used cassowary for layout solving (which is what Apple uses for AutoLayout). I have experimented with yoga, but I think that cassowary-layout is better to maintain and debug in the long run. Also it’s a pure-Rust solution, which was important to me.

7 Likes
38

That general direction sounds reasonable but I’m concerned about unnecessary complexity and prematurely limited performance.

Full CSS support is rather complex, and requires some heroics to make fast, but probably isn’t needed for a new library that’s incompatible with the web. Some kind of separate style language, without cascading or such a wide variety of selectors, could be much faster and easier to maintain while still providing the same benefits.

Cassowary also feels like an awful lot of work for very little gain. I realize Apple uses it, but constraint solvers are a lot of indirection between what the developer specifies and what shows up on screen, leaving a lot of room for subtle unexpected behavior. I would prefer they exist as an extra layer outside a much simpler core, which should be sufficient for the vast majority of UIs.

2 Likes
39

Just to get some numbers, I’ve ran performance tests using azul … A full parse-css-style-render cycle takes 1 - 5 milliseconds, with ~ 1000 DOM nodes, on a single thread, on my 2012 laptop. It’s too slow for a game, but it’s definitely fast enough for a desktop app - Rust is faster than you might think. A custom language without cascading will probably be pretty awful to use. So I think that this is a premature optimization and CSS is fine, maybe with a few modifications regarding centering :wink:.

Yes, cassowary is an indirection, but it’s a very scalable and efficient system that doesn’t break for more complex UIs. I doubt that home-rolled layout systems are more bug-free and maintainable than cassowary. And if you use a few wrapper constraints, it can be quite expressive, too - it’s not too hard to read what AlignBelow or MatchWidth mean, is it? cassowary is a seperate library, so it’s already an “extra layer” - the library is already done, tested and independent of any rendering frontend.

I’m not saying azul and cassowary are the answer to everything, but I think the performance concerns are not quite valid.

2 Likes
40

I agree. Is better to think in “executable DSL” than in a external text format as html/css. The ability to use something like that is something that could be left for another project (and certainly is better to delay it until is mature enough, so we don’t get stuck as with old HTML/CSS assumptions and discover later that people want to use it for do apps ).

In my mind, this could be alike sqlite, where rust is used to build the skeleton and a surface API so can be reused thousands of times.

41

I’m not talking about making cassowary a separate library in that sense, or about the question of “executable DSL” vs “external text format.” I just don’t want to tie the API’s users to standard CSS or to cassowary- an extra layer for them would be fine but ideally it would be usable without either.

I’m also not convinced by those performance numbers. The web has far more extensive experience working with CSS-based styling+layout and it has a lot of performance problems- one benchmark of a WIP library doesn’t really shift my opinion of the approach as a whole, at least not without some more concrete reasoning around why it’s different this time.

1 Like
42

I agree in principle; but CSS has a lot going for it in terms of developer familiarity -- we shouldn't underestimate that.

AFAIK (correct me if I'm wrong) WebRender isn't reliant on CSS, but you can reuse some other surface style language and talk to WebRender. What we could do to eat our cake and keep it is to provide that other style language and CSS -- then we can also provide a translation of CSS -> other style language as a const fn (or macros) and so we gain most of the benefits of familiarity and performance.

1 Like