Pre-RFC: `cargo test --default-features-except` and co

This would be my first ever rust-lang RFC, and first time hacking on cargo, so any early feedback is welcome! Especially if the feedback is "no, absolutely not, never" :slight_smile:

Markdown source.


Summary

This proposal introduces two new command line parameters to cargo, allowing people to directly request the default or --all-features sets minus specified features.

Motivation

The goal is to improve the CLI around selecting features when running (eg) cargo test. Commonly I want to run tests with all features except one (because it implies a slow- or hard-to-build dependency), or default features except one for similar reasons.

Currently this requires memorising the complete sets of features, then typing them out except the one you want to exclude. That is quite brittle if the set of features change. [1] is a concrete example, this would be cargo test --all-targets --all-features-except ring,fips under this proposal. [2] and [3] are other motivating examples.

Guide-level explanation

Say your crate [features] section looks like:

[features]
default = ["a", "b", "c", "d", "e"]
a = []
b = []
c = []
d = []
e = []
f = []

If you want to test your crate with the default features minus the e feature, today you would manually copy the default feature set and write:

$ cargo test --no-default-features --features a,b,c,d

If you save that command into a script somewhere, then later add f to the default features, your script changes in meaning from "default features minus e" to "default features minus e and f". This is brittle.

This proposal introduces:

$ cargo test --default-features-except e

Which achieves the same.

It also introduces the similar:

$ cargo test --all-features-except e

Which (in the above example) enables features a, b, c, d, and f.

These new options:

  • are less typing,
  • allow the intention of the command to be clear, and
  • are robust when the set of features change over time.

These options are available for subcommands that currently support --all-features. Note that does not include cargo add, and there is no equivalent for specifying features for crate dependencies in your Cargo.toml.

Reference-level explanation

The action of --default-features-except is to filter out the specified features when expanding the default feature set.

It is not validated that the specified features are part of the default set, because the user's intention is already met. That follows the lead of --features foo,foo not being an error.

The action of --all-features-except is to filter out the specified features when adding the extant features (as --all-features currently does).

It is validated that the set of features exist in the crate/workspace (as currently happens for --features).

--default-features-without is mutually exclusive with --no-default-features.

--all-features-without is mutually exclusive with --all-features.

There is a prototype-quality implementation of this proposal: feel free to play with it, but do not give its behaviour primacy over this spec.

Drawbacks

Adding any additional CLI options has a marginal cost on all users in terms of complexity and readability of --help output.

Rationale and alternatives

There are other potential CLI designs that compose existing options with a notation that inverts the meaning of a feature, for example:

# "--all-features-except foo" equiv.
$ cargo test --all-features --features !foo
$ cargo test --all-features --features ~foo
$ cargo test --all-features --features -foo

# "--default-features-except foo" equiv.
$ cargo test --features !foo
$ cargo test --features ~foo
$ cargo test --features -foo

Unfortunately the three characters that commonly mean negation (!, ~, -) collide with shell functionality or look like a command-line flag. It also is unclear without prior knowledge whether --features !foo,bar means "without foo and bar" or "without foo, with bar".

Another option is to add a general composable "remove features" option, such as:

# "--all-features-except foo" equiv.
$ cargo test --all-features --except-features foo

# "--default-features-except foo" equiv.
$ cargo test --except-features foo

That option is rejected because it suggests the order of command line arguments are significant. The design here only activates features (having performed a computation on the set of features to activate), so the order of arguments does not matter.

Another alternative is not to build this into cargo itself, but instead write an external tool called like:

$ cargo test $(cargo compute-features --default-except foo)

That tool would resolve the default feature set by reading Cargo.toml, resolve the workspace features, and then output --no-default-features --features <...> for consumption by cargo. That is doable, but is not discoverable by someone running cargo test --help, and is duplicative of work cargo already does. [3:1] is an example of this approach.

Prior art

#3126 contains discussion about a related but wider problem about taking a crate dependency while only taking a subset of its default features. This comment covers just the command-line aspects, and is the proposed interface there is covered in alternatives, above.

Unresolved questions

  • Does adding this interface make it harder to address #3126 in the future?

Future possibilities

Theoretically a notation for doing this for crate dependencies in Cargo.toml as discussed in #3126 would make it possible to support these options for cargo add.


Thanks!


  1. rustls/.github/workflows/build.yml at 06dc1d540cc0a28941790430a37788fd04db956e · rustls/rustls · GitHub ↩︎

  2. Change the CI `--all-features` to an explicit list. · chronotope/chrono@849932b · GitHub ↩︎

  3. cargo-edit: avoid vendored libgit2 · carlocab/homebrew-core@902e516 · GitHub ↩︎ ↩︎

1 Like

Overall, I feel like there are a lot of weird cases people might want to cover and I'd rather not be doing one-off solutions.

Instead, I think the "right" ways of handling this are

  • cargo hack
  • #3126
  • Build orchestrators on top of cargo (e.g. clap uses a Makefile to select between different feature sets)

I think the continuously expanding set of options in cargo-hack shows that it's not an easy problem, and doesn't have an obvious solution yet.

Cargo hasn't got any solutions yet for either disabling default features, nor mutually-exclusive features. I think these two should be designed first, with proper integration in Cargo.toml, before adding special command-line arguments, because it could end up being confusing if Cargo.toml and CLI settled on different implementations.

Use at the CLI is a bit simpler than the Cargo.toml dependency case, because the CLI is only for testing a single (and local root) package. But also a bit more involved, since dependencies don't have an "all-features" cargo-hack is also a lot more involved because it's not only concerned with a single configuration but also creating a feature matrix to test different configurations.

It's certainly the case that it'd be better if both dependency and CLI specification use the same language to disable features. But where the CLI can be simpler is that it can just bail in unclear cases, e.g. when trying to disable a feature where another enabled feature is dependent on. Also, exclusively local usage has more limited interaction with the resolver

But if you want to see progress here, cargo#3126 is the thing to look at. A full design for what exactly it means to disable a feature that acknowledges the different nuances mentioned in that issue discussion, maybe along with a reference implementation of the desired semantics.

Um, workspaces? If I run a plain cargo test today it certainly tests all crates in my current workspace.

If you specify any feature flags other than --all-features or --no-default-features it doesn't do as well when there's more than one default workspace member.

But more what I'm getting at is that there aren't any nonlocal transitive dependency edges that could be enabling a feature which we're trying to disable.

Thanks for the comments so far. Sounds like there's not an appetite for an incremental change here before #3126 happens.

Build orchestrators on top of cargo (e.g. clap uses a Makefile to select between different feature sets)

I've also been pointed at actix-net/justfile at master · actix/actix-net · GitHub which actix-net use to write (what would be) --all-features-except tokio-uring,io-uring