Feature Request: "unstable"/"opt-in"/"non-transitive" crate features

[NfNitLoop]

Hello! The GitHub feature request template directed me here.

I don't keep up with the forums here (yet!) so this may already be a feature that's being discussed or proposed elsewhere -- if that's the case, please point me in the right direction.

Context:

I'm a new user of Tokio. It's got this great new JoinSet that I'd like to use. However, it's still a bit experimental, so it's part of their unstable API, which must be enabled with --cfg tokio_unstable.

I've been using Rust for a few years, but so far I've never had to directly apply compiler flags, so I had to go and figure out how to do that. I ran into a couple issues along the way:

• Despite seeing some examples in the wild, rustflags does not seem to actually work from Cargo.toml.
  (To be fair, Cargo did warn about an "unused manifest key". But of course, I'm doing this while I'm in the middle of hacking on a project so there were so many other warnings that I didn't notice until much later.)
• If you specify a cfg key that doesn't exist, there's no error or warning. (And can be no warning. Because that --cfg foo is applied to all rustc invocations.) It just seems to not work and the user is left scratching their head.

Why Not a Crate Feature?

So, frustrated, I head over to the Tokio Discord for help, and immediately figure out my own problem.

But we also had a chat about why this isn't exposed as a feature:

In Cargo, features are transitive. If foo depends on bar, and bar enables unstable_tokio, foo will get that feature enabled silently. Foo doesn't don't know that bar is likely to break in the future. And foo may also accidentally depend on unstable features.

To avoid that behavior, the Tokio developers used this cfg flag which must be specified as part of the top-level build, so that end users always opt in explicitly.

In searching for rustflags in Cargo.toml above, I found out that the uuid-rs crate uses the same pattern for unstable features. Maybe this is a commonly known and used pattern?

Finally, The Feature Request:

It seems like folks are using unstable --cfg flags to work around a shortcoming in Cargo. It would be nice if Cargo could handle this, so we could just use feature flags!

Benefits:

• Useable from Cargo.toml
• Cargo can check if you reference a feature that's not (or no longer) present, or a typo.
• Makes it easier to follow semver rules by testing out unstable features before making them part of a stable feature set.

My initial (TBH not very well thought-out) brainstorm:

A new feature type. It could be called "opt-in features", or "non-transitive features", but "unstable" seems to describe this particular use case, so I'll use that here.

Within a crate, unstable features work just like stable ones. ex:

# file: foo/Cargo.toml

[features]
default = ["foo", "unstable_bar"]

# Defining an unstable feature:
[unstable_features]
unstable_bar = ["baz"]

# Using unstable features in direct dependencies:
[dependencies.third_party]
version = "*"
features = [ "unstable_whatever" ]

The new "unstable feature" functionality become apparent when transitive dependencies come into play:

# file: my-proj/Cargo.toml (errors)

[dependencies.foo]
version = "*"

This would yield some error like:

Crate 'foo' has been configured to use the following unstable features:

• crate: foo, features = [ "unstable_bar" ]
• crate: third_party, features = [ "unstable_whatever" ]

Unstable features may break semver guarantees, so require explicit opt-in. See: [docs].

And to fix it, the end user just explicitly opts in to the unstable features in their Cargo.toml:

# file: my-proj/Cargo.toml (fixed)

[dependencies.foo]
version = "*"
# Opt in to direct dependency's unstable feature. 
features = [ "unstable_bar" ]

# Must also opt into all transitive unstable features:
[dependencies.third_party]
version = "*"
features = [ "unstable_whatever" ]

Other possibilities:

• If we're worried about a scourge of unstable features showing up in crates, cargo publish could deny crates that have unstable features enabled in their default configuration.
• Might want to force a prefix (unstable_?) for these types of feature flags.
• Maybe add some metadata to unstable features, like a link to docs about why they're unstable?

[NfNitLoop]

Alternatively, maybe "unstable" is just a piece of metadata you can add to a feature. We could enable a slightly different feature declaration syntax:

[features]
default = ["foo"]
foo = []

[features.bar]
unstable = true
enables = [ "foo" ]

# This would even allow additional metadata, like a description:
# Would be handy to compile these into rustdoc!
description = "enables barring foos, which is unstable until we figure out X."

[mathstuf]

Non-transitive has a problem with impl Trait for N. Let's say there are crates A, B, C, and D. A provides some type K with an impl SomeTrait behind a feature flag. B requests the feature of A, C does not, and D uses A (with the feature), B, and C. If there is a function f (in any crate) that has a T: SomeTrait bound.

Questions:

• If f is in crate C, can D pass a K to the function?
• What if D gives f to B to then call with K?
• If it can be called, is this passed-in K the same TypeId as C's lookup of K?
• If it cannot be called, what kinds of error messages would make any sense when this happens?

I think hiding names (anything you can name with use) behind "you didn't ask for this feature" is fine, but acting as if the feature is completely gone is not feasible. You cannot hide an impl behind a use statement (i.e., it is global state), so it cannot be hidden by such feature flag logic.

[Aloso]

Marking some parts of the API as "unstable" can be useful. For example, clap 3.1.1 does it with cargo features that start with unstable-, which is okay because clap is usually only used in binaries. For crates that are used in libraries, we have the problem you described: You don't just get the features you enabled, but the union of the features enabled by you and your dependencies. This is why features are called additive.

This can cause problems, because a crate can use features of a dependency which it didn't explicitly enable. Let's say, the feature in question is called feat, which is enabled by a crate called dep. If dep publishes a new version that no longer enables feat, upgrading to that version might break your build. The solution is to explicitly enable all the features you use. Unfortunately, Rust can't assist in detecting this issue.

With unstable features, we have the same problem: When a crate enables a feature unstable-feat, another crate in the same dependency graph can use the feature even if it didn't enable it.

What you propose is an interesting solution: Make it illegal to enable an unstable feature by default. This means that the root crate has to opt into unstable features of all its transitive dependencies. This works, because dependencies can forward features of their dependencies:

[unstable-features]
unstable-feat = ["dep1/unstable-feat", "dep2/unstable-feat"]

This has the advantage that we always know when there are unstable features enabled somewhere in the dependency graph. This is significant, because unstable features can be removed or changed in a minor version. If we use an unstable feature, upgrading the dependencies might break our build, so we have to track our lockfile.

However, this doesn't completely fix the underlying issue: If we want to use a crate dep1 that enables an unstable feature in dep2, we have to enable it explicitly. But if we also depend on dep2, the unstable feature is enabled implicitly.