Rendered on GitHub.
- Feature Name:
multiple_crate_versions_lint - Start Date: 2022-03-27
- RFC PR: N/A (yet)
- Rust Issue: N/A (yet)
Summary
Give maintainers a mechanism to declare that their library is unlikely to work (e.g. fail at runtime) if there are multiple versions of it in the dependency tree. When the constraint is violated, a compiler warning is emitted to inform users of the issues they are likely to encounter.
Motivation
Implicit arguments
In a simpler world, a function would declare all its required dependencies explicitly, as function arguments. In reality, this is often inconvenient, annoying or impossible—e.g. almost every function would have to require a logger as argument.
Various patterns have emerged to provide support for implicit arguments:
- Thread-local storage (e.g. retrieving the current runtime in
tokioor the current OpenTelemetry context inopentelemetry); - Task-local storage (e.g. retrieving incoming flash messages in
actix-web-flash-messages); - Request-local storage (e.g. the extensions type map in pretty much every single Rust web framework);
- Process state (e.g. the global dispatcher in
tracing)
There is a clear pattern: the implicit arguments are global values scoped to a context (a thread, a task, a process, an incoming request, etc.).
Runtime failures
All these patterns for implicit propagation break down at runtime as soon as the types do not line up, as it happens when two different versions of the crate are being used in different parts of the program.
The runtime failures can be either visible or silent.
tokio is an example of a visible runtime failure.
If your main function creates a runtime using tokio:0.3.x and somewhere in your program a future is spawned using tokio:1.x.y, you will get a panic with the following error message: there is no reactor running, must be called from the context of a Tokio 1.x runtime.
opentelemetry, instead, is an example of a silent runtime failure.
The OpenTelemetry context won't be propagated if the OpenTelemetry context on the incoming HTTP request was extracted using opentelemetry:0.15.x but the propagation code used by the HTTP client relies on opentelemetry:0.16.x. Everything compiles, there is no runtime error, but nothing works as expected.
Both failure modes are undesirable. Catching these issues at compile-time would be preferable.
Silent runtime failures, in particular, are tricky to debug if you do not have a solid understanding of Rust's type systems and the mechanisms used by these crates for implicit propagation. Beginners, in particular, are left puzzled and can waste a significant amount of time trying to troubleshoot these issues.
Guide-level explanation
cargo makes it possible for your project to depend on multiple versions of the same crate.
This behaviour can sometimes be undesirable: you can encounter runtime errors when depending on multiple versions of certain crates.
Library authors can opt into emitting a compiler warning when this is the case:
[package]
name = "mylib"
version = "1.0.0"
[lib]
multiple_crate_versions = "warn"
Compiling a project that depends on both mylib:0.3.0 and mylib:1.0.0 will lead to this warning:
warning: there are multiple versions of `mylib` in your dependency tree.
mylib v0.3.0
mylib v1.0.0
└── httpclient v1.3.5
└── httpserver v0.1.6
As a library author, you can go one step further. You can specify a custom warning message to explain to the users of your crate what issues might arise by depending on multiple versions of it at the same time:
[package]
name = "mylib"
version = "1.0.0"
[lib]
multiple_crate_versions = { level = "warn", message = "`MyType::build` will panic if called within a context managed by a different version of `mylib`." }
warning: there are multiple versions of `mylib` in your dependency tree.
`MyType::build` will panic if called within a context managed by a
different version of `mylib`.
mylib v0.3.0
mylib v1.0.0
└── httpclient v1.3.5
└── httpserver v0.1.6
The warning shows the different versions of mylib in your dependency tree and it highlights how they came to be there. In our example, the binary depends on mylib:0.3.0 directly, while mylib:1.0.0 is brought in as a transitive dependency of httpserver:0.1.6.
You can either try to downgrade httpserver to a previous version or upgrade your direct mylib dependency to 1.0.0.
As a user, there might be cases when you want to ignore this warning. You can do so by adding the following attribute in the entrypoint of your binary:
//! src/main.rs
#![allow(cargo::multiple_crate_versions(mylib))]
// [...]
Reference-level explanation
The parsing logic for Cargo.toml would have to be augmented to detect the new entry in the lib section.
The lint will be evaluated after dependency resolution: it does not act as a constraint on cargo's resolver.
Drawbacks
This RFC broadens the feature set of cargo, which might be deemed undesirable.
Rationale and alternatives
Implement the lint outside of cargo
There is enough machinery already available in the ecosystem (e.g. guppy) to write a third-party tool, outside of cargo, to enforce the lint described in this RFC. Some community-maintained tools provide, today, a very similar functionality (e.g. cargo-deny).
A third-party solution has various disadvantages:
- It is unlikely to be used or discovered by beginners, the cohort that is most impacted by the type of runtime failures that this RFC seeks to prevent;
- It would most likely be consumer-driven instead of author-driven[1], requiring a significant amount of due diligence by crate consumers. Crate authors are best-positioned to provide recommendations given their intimate knowledge of the inner workings of the libraries they maintain.
Abuse existing features
Instead of adding a new feature to cargo, we could nudge library authors to achieve the same objective via existing features.
Both mechanisms detailed below have the same drawbacks:
- They result in a compiler error. This prevents the consumer from choosing to use multiple versions of the same crate, a necessity in certain scenarios (e.g. when upgrading the library version in a large application, piece by piece). Furthermore, making this a hard error would prevent existing crates in the ecosystem from adopting this feature, at the very least until their next breaking release;
- The error message is confusing due to the fact that we are hijacking features that are designed for different usecases.
Link
The links section of the manifest is conceived as a mechanism to make cargo aware that a crate provides binding for a native library.
cargo will return an error if you try building a binary that depends on the two crates with the same links section.
Library authors could populate the links section using a non-existing yet sufficiently-unique name even if they do not link to a native library. This would prevent cargo from building a project that depends on two different versions of the library at the same time.
no_mangle
Library authors can declare a public symbol annotated with no_mangle in all versions of their library.
#[no_mangle]
pub extern "C" fn there_can_be_only_one_version_of_mylib_at_once() {}
This causes a compiler error when multiple versions of the symbol are in scope (i.e. multiple versions of the library are present in the dependency tree.)
Prior art
clippy
clippy defines a multiple_crate_versions lint, in the cargo lint group.
clippy's lint scans the dependency tree and gets triggered as soon as there is at least one crate that appears in the dependency tree with more than one version.
clippy's lint can be useful when working with deployment targets where bloat is a major issue (e.g. embedded). It is impractical for projects with a large dependency tree: there will almost always be at least one crate that violates the constraint and clippy's lint does not provide a mechanism to selectively silence the check (e.g. do not warn me about the cookie crate); you must instead allow the lint, disabling the check altogether.
Unresolved questions
Custom warning messages
If multiple versions of the same crate define a custom warning message, what should cargo show to the user?
The warning message from the latest version? All warning messages?
allow
I am not familiar enough with cargo's and rustc's internals to understand how cargo, where I imagine this lint would live, would become aware of the allow statements relevant to this lint.
Syntax
Is the lib section of the manifest the most appropriate location for configuring this lint? Should it be a top-level field or would we prefer to have it nested for future extensibility (e.g. inside [[lib.lints]])?
Is it even desirable to have the lint "definition" in the Cargo.toml file? Is there an alternative syntax we could use, without being ambiguous, to have it at the top of the lib.rs file?
Future possibilities
Compatibility across versions is not always clear cut due to the semver trick.
This lint could be extended to allow specifying which versions are incompatible:
[package]
name = "mylib"
version = "1.0.0"
[lib]
multiple_crate_versions = { level = "warn", conflicts_with = [">=0.2.0,<0.3.0"] }
-
An author-driven system, like the one detailed in this RFC, could only be achieved via a third-party tool if there was critical mass (in terms of adoption) behind a single third-party linter. At that point, we would probably be talking of upstreaming this into
cargoitself anyway. ↩︎