workaround MSRV-testing without MSRV-aware resolver (#9930) as there is a correlation between low versions and low MSRVs
When you have dependencies without declared MSRVs, -Zminimal-versions still might better serve some people
Once we have an MSRV-aware resolver, in theory that leaves "testing version requirements". An alternative solution to that is summed up as:
drag manifest forward rather than dragging lockfile backwards
The idea is we try to transition the ecosystem over to updating version-reqs by default (discussed in #10498) so that manifests always have the versions from your lockfile in them as the lower bound of your version requirement. At that point, there are no versions more minimal than what is in Cargo.lock and we don't need a minimal-version resolver.
Some users may want to hold back their version requirements as much as possible to give users more freedom in picking versions
As a program's dependency tree grows, the likelihood that you need to hold back a dep until a bug fix is available grows. If you need the latest version of a dependency that forces you to upgrade into the version that has a bug for you, this becomes unworkable.
As mentioned in https://github.com/rust-lang/cargo/issues/10498#issuecomment-2276604441, this would cause more churn in Cargo.toml that would cause more merge conflicts which some project may try to avoid (yes, this is also true of Cargo.lock but those merge conflicts are "trivial" to resolve)
We wouldn't have a workaround when dependencies lack package.rust-version. Ideally, if you have an MSRV then your dependencies would too, so you either get your deps to change or you change deps. Our hope is to lower the burden of maintaining an MSRV.
On the flip side, we could
drag manifest backwards rather than dragging lockfile backwards
We could allow people to opt-in to -Zminimal-versions (config?) and we'd update the version reqs to match what we actually resolve to. Maybe we even have it so cargo update with minimal-version resolution is still meaningful, we pick some strategy for doing updating the lockfile and then sync the Cargo.toml to that.
The answer might be "support them both" if we can design the two feature orthogonality. If this is all managed through config, then people in special cases (e.g. "lockfile is latest while manifest is MSRV-compatible") could still maintain that goal. They'll already be needing to opt-out of features to maintain that goal when we stabilize the MSRV-aware resolver.
To "support them both", we'd need a solid understanding of the use cases and justifications for why they should be first-class. "Because my project chooses to do it that way" is not a solid justification for first-class support so long as we don't "break' you.
I feel like this doesn’t quite address the situation where a workspace contains a library and a product (like a command-line tool). You often want to keep your library requirements minimal, but build your product with the latest versions of things. “Keep your library requirements minimal” is pretty vague, but in concrete terms it could easily be “the library’s MSRV is lower than the product’s”. Maybe we’re no worse off in that scenario than we are today, though; the equivalent of -Zminimal-versions becomes -Zignore-lockfile or something.
Quick thought experiment: in a perfect world, the dependency version requirements listed in the manifest would indicate the actual minimal bound required for a package to work as advertised. Testing that this is true unfortunately does require transitively minimal version resolution, since a behavior bug in a direct dependency might be caused/fixed by version selection of a transitive dependency. But ideally you'd also test with all dependencies maximally resolved to catch any bugs newly introduced there and measure performance impact in the configuration that matters more.
I don't know what this suggests to look for in practice, but it is the framing that I use to conceptualize the problem space.
I think it's reasonable for maintainers to want to minimize the maintenance and testing burden they take on. In theory, a maintainer could maintain a minimal version requirement in their manifest, while separately testing with the latest version. In practice, I think it's entirely reasonable to have the manifest dependencies record the version tested with, even if an older version might have worked; the manifest then represents a known-sufficient configuration.
The MSRV-aware resolver makes it possible for people who need older versions for MSRV reasons to get those older versions without the maintainer having to ensure the latest version of their crate supports older versions of other crates.
If the MSRV-aware resolver is available and stable, what is the use case and benefit of keeping your library requirements minimal?
Is this a question of whether the MSRV-compatible version of a library is perceived as supported rather than just existing?
If library base updates its MSRV in a minor version, and my library mylib uses base, I may want to continue releasing new versions of mylib that work with the older base, thus not bumping my own MSRV. At the same time, though, I want mylib-cli (a separate package in the same workspace, not just another target) to use a newer (but compatible) version of base, with a higher MSRV. And I want to be able to test both in CI.
I think ultimately this might just be a conflict between Cargo.toml being per-package and Cargo.lock being per-workspace, since the same sort of thing could happen with multiple libraries in a workspace as well. So maybe the answer is “your library and your binary should not share a workspace if they have differing min-requirements”. We’ve kind of opened up that direction anyway with workspace dependency specs, but I think it’d still be a shift from current usage.
As a program's dependency tree grows, the likelihood that you need to hold back a dep until a bug fix is available grows.
Specifically, if A depends on B depends on C, and C releases a version with a bug that affects A but not B's test suite, and B releases a version that bumps its minimum acceptable version of C to the buggy version, then A is worse off than if B had not done that.
It might not even be a bug, according to C’s judgement; it might be a “we don't promise this is stable” behavior change that A needs to take some time to adapt to. Or a conflict in global things like links, unfortunately-not-strictly-additive features, or new code that doesn't compile on a particular target (that is, unforeseen disagreement about what the set of supported targets for a library is).
FWIW, I've been testing in CI with -Zdirect-minimal-versions, and the only hassle this has given me is that if my direct version requirements are less than some transitive version requirement, then that is an error under -Zdirect-minimal-versions.
It would be nice if the “shares a target dir and lock file, and unifies features” functionality of workspaces was separate from the “allows workspace inheritance and running commands on the whole workspace” functionality, and the latter could contain (and even nest) many of the former. Until then, there are large ergonomic costs to splitting a project into multiple workspaces.
The question then becomes, should we solve that by trying to get B to not bump its dependency on C, or should we make it easier for A to keep using the older version of B until it's ready to upgrade C?
I've been in the position of A before, and I've been in the position of B before.
In B's position, I've sometimes wanted to pull in a new C with new behavior (e.g. a bugfix I need), sometimes not because it affects me but because I don't want to test all possible combinations of B and C and deal with the various disparate behaviors.
In A's position, I generally want to hold back C and hold back anything that would pull in the new C, and I'd like the tooling to help me do that.
Sure. There will always be reasons to bump versions that are debatable. But “B bumps its dependency requirement when its author sees some reason to” will result in fewer bumps overall — even if some package authors do so for every dependency every time they publish — than “B bumps its dependency requirement because the tooling defaults to doing that for all updates”.
My big-picture opinion here is that it is desirable to aim for the ecosystem having, on average, lower churn — fewer updates for the sake of being updated, that force dependents to update more things too. Keeping version requirements as minimal as is feasible to maintain (in the judgement of each individual maintainer, not as any set standard) serves that goal.
What's wrong with the status quo (other than the lack of stable -Z[direct-]minimal-versions)? Why do we need to “try to” have B do anything they aren't already? Is there anything that A needs to make their life easier besides perhaps a “don't auto-update this package in cargo update” feature?
I personally don't think there is anything wrong with the status quo, plus the MSRV resolver and without adding minimal-versions, but I'm trying to be open to alternative answers to that question.
I don't think we do, but when I said that I was specifically referring to minimal-versions. Part of the point of minimal-versions is to make it easier for B to not bump their dependency on C, by making it easier to test. We could also not ship minimal-versions, which will provide less support for B, making it more likely that they will reasonably decide to put the latest version (which is the they actually tested with) in their manifest.
That's exactly what I had in mind: either a "hold" mechanism (block all future versions until the hold is dropped) or a "forbid this specific newer version" mechanism, as transient state that affects cargo update.
I like the idea of pushing the manifest forward, but whenever I suggested that, I got feedback that it's too risky to depend on the very latest version, because it could get yanked
(cargo update will pull the very latest whenever possible, so copying the lockfile to the manifest also makes them very latest).
Perhaps this could have some time-based limit, such as updating to versions from 3 months ago? (of course within limits of what semver and msrv allows).
The crate index metadata lacks publication dates, but that information would be handy to have anyway.
Even after reading the explanation in the OP of this thread and clicking through some of the Cargo issue threads, I still find the benefits of moving the requirements in Cargo.toml forward pretty nebulous. Given a functional working (stable) -Zminimal-versions, what are they exactly? Assuming Cargo.lock files are versioned throughout the ecosystem, it seems there is very little benefit from reproducing the lockfiles contents in the manifest.
On the flip side, I've encountered issues a number of times where bumping the requirements to a just-released versions caused issues because the new version was yanked a few days later.
Lets say I list in my Cargo.toml that I need bar = 1.1.4. The latest version is bar 1.2.8, which is semver compatible and as such is what is in my Cargo.lock (assuming MSRV allows it etc). -Zminimal-versions would let me test in CI that both 1.1.4 and 1.2.8 works, but:
Twice the number of CI jobs. Might not sound like much but since to be sure I need to to test both of these on all supported architectures {linux, windows, mac} x {x86-64, aarch64} + RISCV and 32-bit architectures for Linux . Oh and times {msrv, stable, nightly} too. That is a lot of configs.
What about all the versions in between? Are there any bugs that affect those that make them not work with my code? There is no way I can realistically test that (especially not all combinations for all crates, maybe new tokio + old tracing + my crate doesn't work?)
For me, it reduces the support matrix significantly (since I do want to support many architectures) to just push forward. I also try to have as few Cargo features as possible for that same reason. It is just more work to try to be configurable in these ways.
Same. I have a CI check to make sure I always use default-features = false on my direct dependencies (and unused-crate-dependencies lint to only directly depend on what I use).
I think this is a library vs binary policy. I have workspaces with binaries that have some reusable libraries on the side. When I build binary releases (with prebuilt binaries uploaded to github releases) from those workspaces I don't want the oldest possible versions. Nor would I want that if users use cargo install with the locked flag. Or when a distro package is being built.
You are also assuming that semver is followed perfectly, which is not true. And us downstream users need to test that or the issues won't be found and fixed promptly. You can yank a semver break if the release was two days ago. Six months down the line you might break as many users as you fix.
And sure cargo-semver-checks help, but is far from perfect. In particular it can't find most breaks involving cross crate types, which has bit me several times as I split my crates for compile time and organisation reasons. Though looking at their bug tracker there are many other limitations as well.
Yes, I was gonna mention that as an option (it's in the issue). Ideally cargo behavior should differ depending on whether the crate is a library or a binary, but as explained in the issue, that's sadly not simple.
Yes, that's the same type of assumption when you assume your dependencies implement their API. Anything not following this is called a bug and can be fixed without discussion. It's ok for stuff to be broken as long as we agree they are.
Yes, but my understanding is that for binary crates, you just need to ship a lock file that works. So by definition there's only one thing to test. It's true though that compiler version may matter.
Those users are already broken, they just don't know it yet. So yanking would actually help them. They are on a parallel universe otherwise.
That is not a very pragmatic approach. If there are more users depending on the new behaviour than the old behaviour (they may even have started using the library after the change) it is not feasible to go out and break them all for an old mistake that broke one guy who couldn't even be bothered to check with current versions in their CI...
It gets even more complex in workspaces where I have:
One (or sometimes several related) binary/binaries
Some supporting libraries not intended for direct public use.
Some reusable libraries (for example implementing file format parsers for specific formats, that were developed for use in the main binary, but could be reused by anyone interested).
What would the default cargo policy even be? Keep in mind that this is also a virtual workspace, so there isn't a designated top level crate in the eyes of cargo.
If the semver bug is that old (i.e. users don't update or don't use the buggy part), then it indeed doesn't help to yank. Instead, a new major version (possibly without any actual breaking change since the last version) needs to be released to disambiguate the situation. Users will just have to manually check if updating needs any change on their side, like for any other major bump. Users on the pre bug version can either temporarily pin or bump completely next time they update.
Yes, workspaces are just worse than crates. They have the same issues but at a bigger scale and thus at a higher probability. It's usually a bad idea to use workspaces, and they are overused in my opinion. There are some good ideas in there though, like sharing a target directory (which you can do without workspaces), sharing a lock file, and sharing dependency requirements (both of which don't have a workaround yet but there are open issues for it).
They help when you have multiple tightly related components evolving together, as often happens when you have internal helper libraries for a binary. Breaking changes are easier to do, especially between releases where there might be several breaking changes (you would have to mess with patch table on cargo to use fir dependencies otherwise)
Chiming in as a user who cares about having CI guarantee that we don't accidentally depend on features introduced in a version of our dependencies which is more recent than the one specified in our Cargo.toml. Don't have strong opinions on how this can be supported as long as it's supported somehow. Don't need a response to this message - just adding a data point.