This is intended to be a follow-up to @aturon’s previous pre-RFC for platform-specific APIs in the standard library. The libs team discussed this topic during triage today and we started out by outlining some problems with today’s methodology.
Current approach
First, I’d like to recap what I mean by “today’s methodology”. Today in the standard library we largely expose a platform-agnostic API surface area in the standard library. This isn’t always sufficient, however, so platform-specific APIs can be found in the std::os module. This module is organized along the lines of std::os::$platform::$module such as std::os::unix::fs or std::os::windows::process. Put another way, if we want to expose a platform-specific API, we allocate a new module inside of std::os and figure out how to put it there (with an extension trait giving access to the functionality).
This works out great for users who don’t want to worry about a Windows/Unix distinction because you just avoid the std::os module and otherwise the rest of the standard library is generally cross-platform enough to get by.
Problems with current approach
This has worked pretty well for just giving access to this platform specific functionality, but unfortunately there are a number of problems with this approach to APIs in the standard library:
- It’s very difficult for libraries other than the standard library to follow this convention. In practice it seems to be very rarely followed. That being said, this sort of compatibility problem certainly comes up in practice!
- Right now the design requires a strict hierarchy of features to be added. Unfortunately features are not always hierarchical, however, such as CPU features.
- APIs don’t live in their “natural” location. For example the
CommandExt::execfunction on Unix is pretty far away from the relevantCommandtype.
Proposed solution
Continuing the concept of “scenarios” from before, I’ve been thinking that we can solve these problems with an addition of a new attribute and system to the compiler. Specifically:
- The standard library will define scenarios, such as
unixandwindows. - APIs in the standard library can be tagged with a scenario.
- Consumers of the standard library can enable particular scenarios (or none)
- The standard library will have a set of “default on” scenarios, but these scenarios may not always exist for all platforms.
Some strawman syntax for defining scenario-gated APIs would perhaps look like:
// src/libstd/fs.rs
impl File {
#[cfg(unix)]
#[scenario(unix)]
pub fn as_raw_fd(&self) -> c_int {
self.fd
}
#[cfg(windows)]
#[scenario(windows)]
pub fn as_raw_handle(&self) -> *mut HANDLE {
self.handle
}
}
Here we’re working with std::fs::File, and we implement the as_raw_fd and as_raw_handle methods directly on the File type. This solves the problem above where APIs don’t appear in their “natural locations”. Additionally, I’m thinking that #[cfg] is entirely orthogonal to #[scenario]. We can see here that the methods are tagged with both, but you could imagine #[scenario] implying #[cfg] perhaps eventually.
This is just an example of defining a scenario-gated API in the standard library. Consumers would perhaps look like:
#![scenario(unix, windows)]
#[cfg(unix)]
fn cross_platform_method(f: &File) {
let fd = f.as_raw_fd();
// ...
}
#[cfg(windows)]
fn cross_platform_method(f: &File) {
let handle = f.as_raw_handle();
// ...
}
Here we see a crate that activates the unix and windows scenarios, meaning that it will now have access to those methods in the standard library. There’s still a separate implementation for Windows and Unix, however. When they’re implemented they use the methods as if they were defined on the type inherently (which in a sense they are).
If the #![scenario(unix, windows)] were omitted then the above code would be a compile error. For example this code would not compile:
fn main() {
let f = File::open("/dev/null").unwrap();
println!("{}", f.as_raw_fd());
}
Here we’re calling the as_raw_fd method (and maybe even compiling for Unix), but the unix scenario was not activated. The compiler could perhaps even provide a tailored error message indicating that this method does exist but the gated scenario isn’t defined.
Usage in external libraries
I’d like for this system to be extended to all crates, not just the standard library. For example net2 crate provides Unix and Windows-specific APIs, or the openssl crate provides functionality depending on what version of OpenSSL you’re linked against.
As given, though, I think it’s possible for all of this to extend naturally to other crates. Crates would simply tag their methods with #[scenario(foo)] which would then require that scenario to get activated in a downstream crate before use (similarly to the standard library). This should be relatively lightweight to add as well!
I would also imagine that there’s sort of a global namespace for scenarios. That way if any other crate decides to put APIs behind the unix scenario (like the standard library) then you’ll only need to activate that scenario once.
Default scenarios
Ok, so this may plausible provide a solution to platform-specific APIs. There’s platforms like emscripten, however, which don’t have features like threads that are present in the “major platforms” of Unix and Windows. To handle this I’d propose something like the following:
- Basically the entire standard library is split up into scenarios. Every API (maybe entire modules) would be tagged as such.
- The standard library then defines a default set of these scenarios to be activated by default. That is, code does not have to opt-in to the APIs.
- Crates could, however, disable this default scenario and re-enable them.
With a system like this, the standard library would simply be missing APIs on platforms where they’re not supported. For example on emscripten the thread module would be simply missing. If you’d like your crate to explicitly support emscripten then you’ve got one of two possibilities:
- You simply avoid
std::thread. Code then naturally compiles for emscripten as you never accessed what doesn’t exist. - You explicitly state in your crate
#. Basically you whitelist a few scenarios in the standard library where you can use APIs. This disallow access tostd::threadif you accidentally leak it in.
This way crates can explicitly declare what scenarios they support and get a guarantee into the future about continuing to support that scenario. Furthermore new platforms can simply omit swaths of the standard library. Support for these platforms would be done by declaring a scenario and avoiding it or by using #[cfg] appropriately to use the features on Unix/Windows and otherwise avoid it on the relevant platform.
Ok, so that’s what I’ve been thinking. What are your thoughts on this? Does your platform perhaps not fit into this sort of model? Is this perhaps too permissive? Can you see a killer hole in this we should close?
I think an important sentiment brought up by @aturon at the triage meeting today is that if you step back it’s actually pretty hard to be worse than today’s system. To that end we can probably get a lot of mileage without trying to be perfect for something like this. Not to say we shouldn’t try to refine it, but just something to keep in mind!