Pre-RFC: Testing a dependency features

language design
1

Summary

Allow a crate to test whether a feature of an upstream dependency is activated, or not:

#[cfg(feature = "rustls/default-crypto-provider")]

Motivation

At the moment a crate can define its own feature, and should the user enable it, enable an upstream dependency feature in turn, for example:

[features]

default-crypto-provider = ["rustls/default-crypto-provider"]

It is not, however, possible to simple query whether an upstream crate feature is enabled.

And therefore, instead of library authors simply querying for whether an upstream crate feature is enabled, define hundreds of "clone" features in their libraries, which users must then discover, and enable for each independent dependency tree in their codebase.

Guide-level explanation

If you wish to know whether an upstream crate feature is enabled, simple prefix the feature by the crate name.

#[cfg(feature = "<upstream-crate>/<feature>")]

The usual warnings will be emitted if the upstream crate has no such feature, they can be silenced as usual.

Reference-level explanation

As above.

Drawbacks

A slightly more complicated cargo & rustc implementation, but really not by much.

Rationale and alternatives

Pre-declare in Cargo

At a very slight loss of ergonomics, due to added boilerplate, requiring that features to be queried be pre-declared in Cargo.toml would avoid having to pass 100s of useless features to a crate which has 100s of upstream crates.

Cargo-level Querying

An alternative would be to condition the enabling of a local feature based on the presence of an upstream feature, something like:

[features]

my-default-crypto-provider = []

[features.rustls]

default-crypto-provider = ["my-default-crypto-provider"]

This opens the door to fixpoint resolution of features, however, as enabling my-default-crypto-provider could then in turn enable other crate features, which in turn could be queried, etc...

It may be useful, one day... but since it's overkill here, it's perhaps best avoiding.

Prior art

cargo-metadata in build script

There are existing work-around, today, using cargo-metadata to inspect a dependency's enabled features in a build.rs.

This is, quite obviously, an abomination. And of course, it's not a fixpoint abomination, so it has bizarre shortcomings.

Features across workspaces

Managing the propagation of features across a large workspace is infeasible without tooling support · Issue #9094 · rust-lang/cargo · GitHub

The ability to query the feature of upstream dependencies is more generic -- it works across workspaces -- and would solve the above at the cost of creating a root tikv-features crate to define the common workspace features, then use those everywhere.

Alternatively, a special workspace/ prefix (or cargo-workspace, or something) could be used to query workspace features, so as to not to have to create a crate just for it.

Unresolved questions

  • Is pre-declaration in Cargo necessary?

Future possibilities

Enhanced workspace support

Introduce a special workspace/ or cargo-workspace/ prefix to allow a crate which is part of a workspace to query whether a feature is enabled at workspace level.

Besides the possible name collision, this would require extra work at publication time, as the workspace features would have to be replicated within each crate... which using a separate crate within the workspace to define the features does not suffer from.

2 Likes
2

Related cargo issues:

In particular, we should keep in mind Managing the propagation of features across a large workspace is infeasible without tooling support · Issue #9094 · rust-lang/cargo · GitHub

I think more is needed to understanding the motivating use cases. In some situations, this is about cross-cutting functionality which might be better served by Pre-RFC: Mutually-excusive, global features.

3

Somewhat related, but features are not used solely for enabling dependencies:

  • It would be impossible to detect whether a feature which only enables more APIs is enabled, say default-crypto-provider which would make CryptoProvider implement Default.
  • It would allow detecting whether aws_lc_rs is available somewhere, but not whether rustls itself is configured to use it by default.

It would allow rustls to automatically enable its aws_lc_rs related APIs, but that's a slippery slope. For example, you may not want a crate with 1000s of types to suddenly enable serde serialization/deserialization just because it detected that serde was in the dependency graph.

This issue could be solved by:

  1. Using a tikv-features crate in the workspace, which controls which features are enabled/disabled at the workspace level.
  2. Then simply depend on that tikv-features in all other crates, and query which of its features are enabled.

I'll edit it in prior art.

Mutually-exclusive, global features, are a very different beast.

Here, the motivation is rather simple. Imagine that rustls has a default-crypto-provider feature, which:

  1. Enables the aws_lc_rs crate.
  2. Enable certain APIs, such as ClientConfig::builder(), which will automatically use aws_lc_rs as the crypto-provider since it's the default.

As the author of a library built above rustls, say, hyper or tungstenite, you'd like to yourself offer this high-level shortcut APIs if available.

You absolutely do not, however, want to enable these APIs if nobody else enables them, as otherwise your users will pay the cost of compiling (and bundling) the beast that is aws_lc_rs even if they don't use it.

At the moment, the two work-arounds I know of are:

  1. Create a feature of your own, which enables the feature in your upstream dependency. Users will only be able to user your shortcut API if they enable your feature.
  2. Use the hack that is running cargo-metadata in build.rs to detect whether rustls enables the feature.

And neither is great.

4

I work in the world of CMake quite a bit and these kinds of "implicit enabling of behaviors" based on "ambient" environment properties (versus "intrinsic properties" of the environment) are hellish to deal with. Projects changing behaviors just because I installed a system package have led to quite…"fun" debugging sessions. If it is something that is cared about, I think it needs to be explicitly mentioned at some level in the Cargo.toml.

To clarify the difference between "ambient" and "intrinsic", the former are things that can change about a platform while the latter are pretty invariant. For examples of "intrinsic" properties, you can safely ignore Metal on non-Apple platforms. Likewise, searching for winsock or Win32 libraries is pretty useless on non-Windows platforms. But "does libX exist?" is a completely different beast and should have a user-level control knob on whether to even look for it (an auto setting existing is fine and can even be the default, but please let be express "no, I do not want XYZ support in this build" as well as "this build must have XYZ support" (with appropriate error conditions if it cannot be satisfied).

IIRC, there have been RFCs about "does an item exist at path X::Y::Z" that may be able to more precise detection rather than feature flag-based detection.

3 Likes
5

+1 on having some way to make code conditional on other features.

In the past, I regularly had issues caused by serde's error traits requiring std::error::Error if the std feature was enabled, when I'd have a dependent crate with an optional std feature. My crate would have something like this:

#[cfg(feature = "std")]
extern crate std;

struct DeserError { .. }

#[cfg(feature = "std")]
impl std::error::Error for DeserError { .. }

impl serde::de::Error for DeserError { .. }

This works well so long as my crate's std feature is active whenever serde's std feature is active, but there's no way to ensure that direction of feature dependency. So I'd end up cases where I have a dependency tree like this (where this crate is D):

A --> B --> serde/std
  \
   -> C --> D (w/o std) --> serde (w/o std)

But then feature unification means I have serde with std and D without std and that causes it to fail because D::DeserError: std::error::Error is required but never implemented.

Luckily, I've been able to avoid this exact case recently because std::error::Error is now in core, so I can just do impl core::error::Error for DeserError { .. }, but these same conditions exist for any other std-only trait (e.g. io::{Read, Write} are two traits I could easily see having these issues).

Alternatively, some way to specify that serde/std requires D/std, even as the crate-level dependency goes the other way could also have resolved this.

1 Like
6

It's not quite clear to me what you're suggesting here, so I'll answer multiple potential interpretations.

I fully agree that just querying whether a library exists somewhere in the environment is unpalatable. This RFC does NOT make builds any less hermetics that they already were.

This RFC does mean that the behavior of one library changes based on which other libraries also participate in the build -- and crucially, whether they enable a specific feature or not -- but... that is nothing new AFAIK.

The side-effects of enabling features -- that is, the fact that items suddenly pop into existence, the fact that trait implementations pop into existence, the fact that optional dependencies may set a global, etc... -- is already observable.

It's just incidentally observable, or should I say accidentally observable.

And it's not clear that making it UNobservable is a tractable problem. That is, if we take two libraries A & B, both depending on D, then A not observing that B enabled feature D/x which changes the definition of X could mean that A & B see different definitions of struct X, which would prevent A & B from both operating on the same instance struct X. Awkward does not start to cover it.

So, whether we wish or not, enabling features in one library has ripple effects on all downstream dependencies on this library already, and said libraries may already implicitly depend on this feature being enabled, without knowing it.

In fact, this is the very situation that the TiKV team complains about. Their build is constantly broken being some of the libraries implicitly depend on features that they do not activate, and thus fail to compile in isolation when said features are not activated for them.

The existence of an item is only part of what a feature can do:

  • A feature can alter the definition of an item.
  • A feature can also enable trait implementations
  • A feature can pull in optional dependencies.
  • ???

And perhaps more importantly, a feature has semantics of its own.

Attempting to divine which feature is enabled by poking at which side-effects (enabling items, trait implementations, etc...) a feature is expected to have is like attempting to divine the CPU model from the list of enabled instruction sets. It's brittle. Such side-effects are inherently indistinguishable from a version upgrade.

1 Like