Terminal platform abstraction

libs
1

A while ago I looked at making a project for terminal abstraction. I dived into it because that’s what my naive self thought was a good idea. Turns out terminal control is really hard, because there is such variety in how to access various bits of functionality, from over a serial cable right through to powershell, including the possibility of escape sequences over stdout (don’t mix this with system calls).

I lost heart and the project stalled. Because of the talk around the roadmap about ergonomics and ergonomic libraries, I thought this was a good time to talk about this again.

The community is working on creating some ergonomic libraries. A terminal library was mentioned. Whilst I don’t think the base libraries exist for an ergonomic term lib, maybe this is the time to talk about creating those base libs!

Below I cross-post a proposal about how to move forward. It’s not really about the technical stuff, that can come later if there is sufficient interest to make progress.

TL;DR I’m just a community member and whilst I have the enthusiasm, I know I don’t have all the knowledge required to complete this project on my own. Therefore I’m looking for others that consider this a priority and would be willing to lend their expertise to help develop this project.

We should make the first library totally about platform abstraction, with the lowest cost possible. We should focus on providing all the functionality we can, with ideally 0-cost over the system calls. We should always provide the fastest way to do things, even if there is some API overlap. This can be handled in a higher-level ergonomic crate on top that knows how to delegate the the right methods.

  1. Get more stakeholders. There are people in the community that have better knowledge of this domain than me, and they should be involved at least in an advisory capacity, or this project will struggle to achieve its goals.
  2. Make an (incremental) plan. What functionality should we have at the end? Can we split this functionality into more deliverable units? I would think using one of the existing term crates as a starting point and pulling out a platform-agnostic API would be the first step. Then implementing this API for lots of backends (platforms). Then increasing the surface area whilst maintaining support for all backends.
  3. Implement the first round of this functionality for all platforms.
  4. Advertise that we have done this. Think of a cool pun name for the library (optional). Stress that this is an inclusive project where we need lots of contributors with specialist knowledge of the different platforms. Be open about where we are unsure we have the best approach, and encourage people to contribute if they have a better idea.
  5. In parallel
    1. Make (possibly breaking) changes to the API to make it “better” where necessary.
    2. Add more API surface area as specified in the plan.
  6. When we have all the APIs we want, plan for 1.0 and stabilization. Let the libs settle without adding new features to see what problems come up. Encourage people to start experimenting writing ergonomic libs on top and see what issues arise.
  7. Release 1.0.
  8. Market the library and get other people to make ergonomic libs on top for different use cases (TUI (a-la vim), just colored output, status line (a-la pacman).

It would be good if we could make using a terminal as easy as clap/structopt makes handling command line args. But it’s a harder and more risky goal than CLI parsing because there is more variation between platforms and more platform-specific code is required.

–

Anyone interested?? :slight_smile:

4 Likes
2

I don’t have a ton of thoughts in this space, but I have a lot of domain knowledge with respect to making colors work in a variety of environments. The very first thing you need to decide on that point is whether you want to facilitate the use of the Windows console API to do coloring in multithreaded command line tools. Whether you do or don’t will have a dramatic impact on the cross platform API you design.

For example, you could answer, “no,” in which case, colors could only be made to work in a Windows console (cmd.exe or PowerShell) for Windows 10 users. Users of older Windows systems will never get color since the only way to get color is through the console APIs. Windows 10 and newer supports an opt-in toggle to enable VT100 support in Windows terminals, which, when enabled, permits (mostly) standard ANSI escape sequences.

There are other design points, like, “yes, we’ll support color in a Windows console but only in single-threaded CLI tools,” but that will lead to its own challenges.

3 Likes
3

But the VT100 was black-and-white! The VT340 added color! :stuck_out_tongue_winking_eye:

7 Likes
4

Could be good to also check out the colorama Python package that solves similar problems.

5

It’s worth linking in the CLI apps domain working group in relation to this. I agree that since this is a focus area for Rust 2018 it is bound to get a lot of traction.

A terminal platform abstraction seems fundamental to this goal. @killercup might be interested in pitching in here?

2 Likes
6

Thanks for the ping! I’ll kick off the working group next week. This is definitely a topic I want us to tackle!

1 Like
7

I might be better able to contribute as part of a group like that rather than trying to do something like this on my own. From @burntsushi’s comment, it seems that there needs to be some considerable design discussions about terminal interaction. I’ll link PDcurses here as I think their code may help with things like capturing mouse events.

2 Likes
8

Thanks for linking to this. colorama’s approach is different from termcolor's and may very well be superior. It is also compatible with a design that accommodates multithreaded command line tools. The reason why I didn’t go that route is that you need to write an ANSI escape sequence interpreter. Not that hard, but more effort than I wanted to spend at that particular moment in time. :slight_smile:

2 Likes
9

That looks really cool!

Maybe using this approach makes it possible to do terminal manipulation as ansi escape, and just make OS calls for handling mouse/keyboard input and resize, whilst still supporting all targets.

10

I’ve thought about this some more, and I really like the idea of polyfilling ansi escape sequences. Maybe the terminal support task could be split into 2 different parts: input and output. Output would be ansi sequences, so only input would need to have an API. Output could also be done first and independently of input.

I’m going to do some experimenting to see if this is a valid strategy.

11

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.