Prerelease versions aren't usable for shared dependencies

Cargo treats version 1.2.0-alpha as strictly incompatible with a ^1.0.0 requirement.

Of course it totally makes sense to avoid picking unstable versions by default, but it seems to me it should only be a strong preference for stable versions, but not a hard incompatibility.

The incompatibility makes testing of prerelease versions of shared crates cumbersome. If a project asks for =1.2.0-alpha, but any other (transitive) dependency asks for ^1.0.0, the dependency resolution will fail with a conflict.

Such failure seems unwarranted to me. In practical terms (putting aside what semver spec said and what Version.cmp() currently does): 1.2.0-alpha is still significantly newer than 1.0.0. It's still a 1.x major version. From API compatibility perspective, I would expect that a version that is a prerelease of 1.2.0 is trying its best to be API-compatible with 1.1.* releases. It could have breaking changes compared to other 1.2.0-pre releases, but not (intentionally) relative to previous stable releases.

This hard version incompatibility has been a pain for a couple of crates I maintain: rgb and boring-sys. These are shared crates used across multiple dependencies and should be tested in a larger context. The inability to force use of a prerelease version for transitive dependencies wanting old stable versions creates a chicken-egg problem: 3rd party dependencies only want stable releases (I can't expect everyone in between to make their own prereleases with prerelease dependency requirements), but I can't confidently make a stable release until it's properly tested across the ecosystem.

AFAIK currently the best way to test new unstable versions of crates is to give them a stable version and use [patch.crates-io]. This isn't elegant. It would be more straightforward without patching if dependency resolution supported upgrading to prerelease versions.

10 Likes

Cargo's version requirements require opt-in to pre-releases which means you have to edit yours and all dependency version reqs to use prerelease versions which is one of several known issues with prereleases (another major one is that they are treated as compatible with each other).

Tracking Issue for `update --precise` with pre-release · Issue #13290 · rust-lang/cargo · GitHub is one attempt at fixing this but

  • --precise doesn't help when prereleases depend on prereleases
  • there is some uncertainty about the version requirement semantics in this mode
1 Like

It looks like the update --precise issue got stuck trying to define how semver requirement syntax should be transformed to match prerelease versions, which ended up having a bunch of tricky edge cases (which is expected, it is a tricky problem).

I wonder if this can be approached differently by changing how the resolver works, rather than changing how semver works in general.

I was thinking about logic like this:

  1. Perform dependency resolution as usual.
  2. If it fails due to a conflict, and the conflict is due to prerelease versions not matching stable version requirements, then fudge the specific affected requirements to be satisfied (e.g. treat 1.2.0-pre crate as being 1.2.0) and goto 1.

The current resolver has something very similar where it detects conflicts and backtracks to other candidates. However, the conflicts and backtracking happen with fine granularity during resolution, and at that point the resolver can't accept a prerelease if there's no other option, it can either accept it when it sees it or reject it forever.

So I'm worried that simply including prereleases in the list of candidates (even if they get put at the end of the list) could still make them selected too often. Ensuring that prereleases aren't picked if there are non-prerelease solutions looks to be difficult to implement in this algorithm for the reason that ensuring only a single semver-major version of each crate is used – once a dependency is satisfied, and nothing forces it to backtrack, then it won't, and will keep whatever was locally optimal/first to activate, without having opportunity to choose between solutions that are more globally optimal.

I was hoping the resolver could pick prereleases when necessary by itself, to make them just work based on Cargo.toml. cargo update --prerelease can do the job, but automatically avoiding dependency resolution errors would be a nicer interface.