Pre-Pre-RFC: making `std`-dependent Cargo features a first-class concept

tools and infrastructure cargo
1

For background, this is somewhat related to some ongoing bugs in Cargo and how it unifies build-dependencies and dev-dependencies, and this recent comment on that in particular:

Rust API guidelines state that Cargo features are both additive and that when gating std support, it should be under an eponymous std Cargo feature:

https://rust-lang-nursery.github.io/api-guidelines/naming.html#feature-names-are-free-of-placeholder-words-c-feature

Many crates to do this to facilitate no_std use, gating all std-dependent functionality under a default-enabled std feature.

For no_std users, this means they wind up with Cargo.toml dependency sections that look like this:

[dependencies]
aaa = { version = "...", default-features = false }
bbb = { version = "...", default-features = false }
ccc = { version = "...", default-features = false }
ddd = { version = "...", default-features = false }
eee = { version = "...", default-features = false }
fff = { version = "...", default-features = false }
ggg = { version = "...", default-features = false }
hhh = { version = "...", default-features = false }

(This is exacerbated by diagnostics for which crate is linking std being bad, but I digress...)

Failure to track down a single one of those unintentionally activated std features is enough to prevent linking, and this is often in a hierarchy of transitive dependencies.

I feel like the tension here is between these two things:

  1. Crate authors who want to expose a subset of functionality to no_std users don't want that to come at the cost of additional complexity to std users, and when push comes to shove will typically optimize for the latter
  2. no_std users must track down every last crate failing to #![no_std] or, after doing so, doing an extern crate std

I am wondering if making std features more of a first-class concept, rather than a "guideline", could help find a better balance between these to.

The goal: make it easy for crates which link against std to activate the std feature in all of their dependent crates without those dependent crates having to do default = ["std"]

Here are a couple ideas along those lines. Now granted, these may be the sorts of things that only make sense when bumping editions, but if that's the case, better we start thinking about them sooner than later:

Declarative activation of std for all dependencies

I'm imagining something like this in Cargo.toml:

[package]
name = "..."
edition = "202X"
std-features = true

Now, I'm not sure the [package] section is the right place for this (I chose it because [features] seems quite overloaded), or if std-features is a good name, or if a boolean value is the right way to express this, but let me just get to the core idea: a declarative way to activate the std features of all dependencies as an alternative to each of those dependencies doing default = ["std"].

Declarative deactivation of std-dependent features for no_std users

Alternatively, perhaps instead asking the whole ecosystem to flip its defaults, Cargo could provide a more declarative way for no_std crates to shut the default-on std features off. Here's the same idea, in reverse:

[package]
name = "..."
edition = "202X"
std-features = false

This could potentially allow embedded/WASM or other no_std users to declaratively say "I don't want std features in my dependencies or any default-on features that depend on std", eliminating all of that default-features = false boilerplate.

Other options?

I'm sure there are many other mechanisms this could be tied to: a whitelist of targets a particular crate is intended to be built for perhaps? If a crate only targets no_std platforms, regardless of how this is implemented, it sure would be nice if there were an easier way to request that dependencies to not link with any std-dependent features.

I would be very curious to hear any other idea about how the end goal of "make it easy to deactivate std-dependent features of (transitive) dependencies" can be accomplished.

2 Likes
2

Of course, while this would be nice, it wouldn't really help with #1796.

The issue there is that the test dependencies really do want std. Criterion really does use and need (well, want, since it could reimplement all of it) the std-dependent features of itertools.

1 Like
3

Yes, there's a lot of different things going on with #1796, which perhaps I shouldn't have even mentioned because the solutions to it are ultimately orthogonal to this particular issue. I have definitely worked around that particular issue for criterion specifically many many times with something I'm not a huge fan of but does the job: creating a benches crate in the same workspace which imports the "real" crate as a dependency.

I brought up that particular comment on #1796 because I have recently seen several people make variations of this point:

My 2cts: std is not a feature in the sense that you can take a union of it, rather its opposite no_std would make sense in a union. But none of this makes sense if the union is taken across all build targets.

Here's another:

My proposed solution in that case: take std out of default features. But it's not that easy: removing std from the default features of crates that primarily target users who want to use std-dependent features is bad ergonomically.

If there's a central theme of this Pre-Pre-RFC, #1796, and the linked discussion immediately above, I think it's that handling of std/no_std as a "feature" is special in a way no other cargo feature is or ever will be.

Normally spurious feature activation is harmless, but std-as-a-cargo-feature is ubiquitous, often transitively activated, and failure to shut off every last activation thereof breaks linking.

4

I'd like to see a fix for default-features that works in general. Disabling of any default features is always super fragile, so it's not only a no/std problem, but a problem for every crate that uses default-features (such as disabling bindgen, backtrace support, static linking, unwanted database drivers, or just reliably disabling unwanted features that bloat the executable).

2 Likes
5

I feel like std presents a dimension to default-features that isn't adequately captured. It's almost like crates need two sets of default features: one for std users, and one for no_std users.

Something like this, maybe?

[dependencies]
default = ["no_std_thing"]
default-std = ["std_thing"]
no_std_thing = []
std = []
std_thing = ["std"]

Or here's a crazy idea: what about an entire new toplevel no_std section to Cargo.toml that can be specialized to the no_std use case, which is unified with the existing configuration for std users?

Something like:

[dependencies]
default = ["std_thing"]
std = []
std_thing = ["std"]

[no_std.dependencies]
default = ["no_std_thing"]
no_std_thing = []

Perhaps a more general issue: it would be nice if Cargo knew it were building a no_std project and could apply customizations to the build accordingly.

6

This really would need to deal with three variants now; core-only, core + alloc and std.

One thing I’ve considered being useful is a way to opt-in to default-features = false for all dependencies to reduce the repetition (in the context of feature-rich std using crates, but seems like it could be useful here as well).

1 Like
7

I have experience as a Gentoo developer, where we have the concept of USE flags for our package ebuild scripts. I think there's a lot Cargo could learn from such systems to evolve the functionality of features, which honestly feels very MVP now.

I think this is all kind of skating around the issue that the additivity property is unnecessarily restrictive. Instead we could allow negative dependencies (strawman syntax) like !std, which would request a build that doesn't have that property or errors out if another dependency in the tree requires std.

Gentoo also maintains a limited list of "global" USE flags in addition to the package-specific flags. These basically get extracted as we go when we encounter a flag that's used across a bunch of packages with the same meaning, and then you can set a global value for it. If we bake in a short list of global features like std, alloc and maybe things like serde, I think that would go a long way towards helping with this.

(I've also occasionally wanted something like Gentoo's REQUIRED_USE, which allows more complex restrictions of what flags can be used together. For example, this would allow enforcing mutual exclusion of some feature flags, which could help for example with keeping dependency trees for those features from conflicting -- I guess this would require extensions to the Cargo.lock format.)

7 Likes
8

Great idea! This seems like a simple solution which would help a lot.

I guess it'd still need to be done in a new edition, but I have so many crates where every dependency has default-features = false for almost every dependency it'd be really nice to simplify that.

9

I was thinking about this proposal again in the context of a 2021 edition and how it could help eliminate some lingering usages of extern crate.

Namely, we can eliminate all of this boilerplate by replacing it with first-class std and alloc features:

#![no_std]

#[cfg(feature = "alloc")]
extern crate alloc;

#[cfg(feature = "std")]
extern crate std;

Here are some prospective rules for "paving the cowpaths" using first-class features:

  • If a hypothetical 2021-edition-with-this-pre-rfc crate does not have either std or alloc features, it automatically links against libstd and liballoc just like any crate ordinarily does today. This hopefully makes such changes relatively low impact.
  • If such a crate has an std feature, it would have the same effect as declaring #![no_std], but enabling it could automatically provide the equivalent of extern crate std, rather than the boilerplate above. Since I believe it's generally desired to eventually eliminate extern crate, this could be a step towards that goal.
  • This could similarly be applied to liballoc via an alloc feature, although nailing down the semantics is a bit trickier than a first-class std feature, but it could also get rid of the need to extern crate alloc.

Perhaps the transition could be eased by allowing the extern crate boilerplate to continue to exist (relegated to what's effectively a no-op), but having such boilerplate would be considered non-idiomatic, removed via a cargo fix --edition-idioms, and linted via hypothetical rust_2021_idioms.

I also think #![no_std] can stay: some crates don't want to have a std feature ever! But interestingly, under this regime, #![no_std] could potentially allow a crate to declare: "I never want to link to std" in a way that's not possible now.

3 Likes
10

In the today's context there will be, IMHO, an even better option:

  • add an analog of #![no_std] that is declared in Cargo.toml and makes std unavailable to dependencies (i.e. it would be "unknown crate" during compilation instead of link error);
  • use #[cfg(accessible = "::std")] for no_std support instead of #[cfg(feature = "std")].
5 Likes