A vision for portability in Rust

[retep998]

Runtime trait dispatch brings back traumatic memories of the old IO where everything was done through dynamic dispatch to support both libgreen and libnative. Please make sure the solution is done using static dispatch.

[rpjohnst]

Really traits feel like the wrong solution for cross-platform dispatch- they not only suggest that dynamic dispatch might be available, but they also suggest that multiple implementations may be available in a single compilation and force you to program as if they are. They also make more work for the compiler, which in this case is completely redundant because there will only ever be one implementation at a time.

Ideally this would not use traits at all- a clearly defined stable interface for porting std is great, but defining it in terms of #[cfg] is probably more straightforward, easier to work on top of, and faster to compile.

[comex]

They also make more work for the compiler, which in this case is completely redundant because there will only ever be one implementation at a time.

But in theory, that work could be guaranteeing (ish) that your code will successfully compile on any platform satisfying your requirements, not only the current one, without actually having to test it on every Rust-supported platform. In other words, the difference between Rust’s type-safe generics and C++ templates, but applied to global parameters, such as the current platform.

This would require a bit of new language functionality, and it wouldn’t be perfect, but IMO it would be a real ‘holy grail’ of a feature.

[Alfriadox]

I have to agree with @comex here, this sort of feature sounds awesome.

[rpjohnst]

Doesn’t the portability lint do basically the same thing, though?

[bossmc]

Obviously this is unstructured speculation, but you can model the various sub-modules of std as APIs (think std::thread which has spawn(), current() etc.) with associated types (Thread, JoinHandle etc.).

A trait based model would have a std::thread “trait” that specifies that API with some number of impls for various platforms. So far, so obvious, I think this is what everyone is picturing in the last few comments.

If we allow some std-only magic (or a new language feature) we could implement the std::thread module’s API through the trait statically (so the code says std::thread::spawn() but that is compiled down to <std::linux::x86_64::gnu::thread::Impl as std::thread::ModTrait>::spawn() (i.e. the existing API stays usable, but is backed by a statically-dispatched, selected-at-compile-time implementation, that depends on platform and/or, as @matthieum suggests mocking or scope attributes).

Shouldn’t slow down compilation speeds, since all the translations are fixed for a single compile, and won’t slow down the generated code, since the function calls are all static-dispatch.

Wild speculation begins here Expose all (not just std) module APIs through a trait rather than unbound functions so any module can be swapped out for specialisation or testing purposes. End wild speculation

[japaric]

and becoming part of a Portability Working Group.

I’d love to participate in the discussions. (Though that may be overloading myself a bit too much)

[pierzchalski]

Huh, you might even be able to do that in stable Rust with #[cfg] and default type args:

// `std`
#[cfg(os = "linux", ...)]
pub use std::linux::...::Impl;

#[cfg(os = "osx", ...)]
pub use std::osx::...::Impl;

pub mod thread {
    pub trait ModTrait {
        fn spawn();
    }
    pub fn spawn<ThreadImpl=std::Impl>() where ThreadImpl: ModTrait {
        ThreadImpl::spawn();
    }
}

Not sure if there’s a way to make the trait bound ‘invisible’ to the caller until they want to provide their own impl type, or if that matters.

I’m torn between how this would let you use two different versions of the standard library in the same codebase (“let me just quickly use my little DebugStdImpl here…”), but it would also let you use two different versions of the standard library in the same codebase (“Ah no see that file got opened by our NFS overlay on top of std::fs, but this file…”).

[matthieum]

Runtime trait dispatch brings back traumatic memories of the old IO where everything was done through dynamic dispatch to support both libgreen and libnative. Please make sure the solution is done using static dispatch.

I would never suggest run-time dispatch

@pierzchalski: I was more thinking of something akin:

#[cfg(os = "linux", ...)]
use std::linux::thread::Impl;

pub mod thread {
    pub fn spawn() { Impl::spawn(); }
}

Without the trait impl nor the trait usage being visible.

Also, I would prefer “magic” to bring the trait implementation in, so that the trait can be implemented by a crate outside of std. This would make supporting alternate platforms much easier.

Want to play on an embedded ARM target? In the target description file, specify your own crate as providing the low-level std trait implementations, and hop you’re up and running without patching the official std crate, all statically dispatched.

[aturon]

Thanks, all who have responded! There are already some very interesting issues being raised in this thread, and next week I’ll try to summarize them in another post.

In the meantime, those who want to be part of the working group, can you please fill out a scheduling poll and make sure to include your email address? We’ll have a video meeting at a time that works for all, and go from there.

[ratmice]

One of the questions i suppose that i had about the portability linter, as described in the RFC uses a SAT solver, and whether this should aim to leverage the solver built into chalk in some fashion, or not.

[FenrirWolf]

Another thing that might be good to address in the portability story is the situation with path prefixes. Apparently Redox has its own style of prefixes that can’t be accounted for without breaking backwards compatibility.

[aturon]

I’ve sent meeting invites for those of you who responded to the doodle poll. I’ll try to write up some summary notes before then.

See you all next week!

[MaikKlein]

I am currently working on porting rustc to the GPU. The project is called rlsl. See the latest blog post for more infos.

The portability initiative seems amazing. Although so far my experience with no_std has been pretty good, I only re-export the things that I need and the unsupported functions never get collected because they are not exposed.

I am not sure how reasonable the cfg approach would be. For me, I think there would have to be a cfg attribute on almost every item.

[newpavlov]

I think we need an ergonomic way to define crate-level target configuration (os, feature, arch, etc.) which will be translated to all items inside it and perhaps module-level as well. From user perspective Cargo.toml is ideal place for it, but some raised concerns that such coupling could be undesirable.

[aturon]

Here are some pre-meeting notes which we’ll fill in as we go today. The main focus of the initial meeting will be on deciding which problems to try to solve, and in what order – not getting bogged down into solution specifics.

If you think a problem is missing from the list linked above, please add it!

The meeting today is at 1pm Pacific, on hangouts. Feel free to join even if you didn’t participate on the doodle poll.

[ratmice]

FWIW, I was a bit skeptical of the notion of using Traits with #[cfg(…)]'d out members, in the sense that i wasn’t sure it would allow

1. mixing different cfg’s between different dependencies.
2. if so would that allow one to violate object safety in some fashion when coercing to a trait object.

which seems to resolve (pleasantly) to

1. yes,
2. not within a compilation at least.

here was the repository i attempted this in…

matt rice / trait-experiment

[aturon]

Follow-up: we had a fantastic kickoff meeting yesterday, with about 12 people in attendance! The Rust community never ceases to amaze.

The main take-away from the kickoff meeting is that the most important priority is refactoring std to make writing and maintaining ports to new platforms easier. In particular, because std today intertwines platform-specific code throughout, keeping an out-of-tree port up to date requires very painful rebasing. We found that most of the goals we were interested in (including having a “pluggable” std) are ultimately blocked on factoring out a cleaner “platform abstraction layer”.

@jethrogb has agreed to lead a new Portability Working Group! He will be working to organize the group over the coming weeks, and I imagine will have more to say about this soon.

I’m excited to treat portability as a first-class concern in Rust, and to start making these long-term investments in taking Rust to a wider variety of targets! Thanks y’all.

[jethrogb]

I created a space for the Portability WG. I already added some issues tracking things we discussed in this and other topics and during the meeting. Feel free to add more issues yourself!

We’re also on IRC: #rust-portability channel (server: irc.mozilla.org). Most of us use our GH usernames as our IRC nicknames.

Official WG announcement: Rust needs to run, and be easy to use, on every platform

[aturon]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.