Suggestion: cargo yank is a misfeature and should be deprecated and eventually removed

tools and infrastructure cargo
1

I'm going to make a bold suggestion: cargo yank should be deprecated and eventually removed.

The rationale is the following:

  1. Cargo yank is very difficult to use without being very disruptive to downstream users. And there have been many examples of such disruption.
  2. Cargo yank doesn't actually do what many crate maintainers think it does. It often has secondary effects that they don't anticipate.
  3. Cargo audit, and the rustsec advisory repo, are a superior solution to this problem. We should get rid of cargo yank, and tell maintainers to file rustsec advisories against crate versions with security problems. For crate versions that don't have security problems, there is little rationale to yanking them.

  1. The oldest example of ecosystem disruption is most likely Why were all versions prior to 0.14 of this crate yanked? · Issue #774 · briansmith/ring · GitHub. I posted another post about frustration with yanking 4 years ago: https://www.reddit.com/r/rust/comments/ctayew/rust_crypto_developers_please_stop_the_yanking/

  2. Many maintainers seem to think that, yanking a crate will force users to update. But of course this is not exactly the case. Anyone with the yanked crate in their cargo lock will still be able to use it, so many projects may continue to use the yanked version. Any projects that don't have a cargo lock committed, may now be unbuildable without changes, but this may very from user to user depending on if they have a cargo lock in their local checkout already. So it is very hard to predict how many of your users will actually notice the yank and do anything.

Many maintainers seem to think of yanking as an "undo button". For example, if you screwed up your release and your crate doesn't build in some configuration, surely you should yank it after you publish the new version. But if we think about it, this doesn't really make sense either. Users who encounter build problems can easily update to the latest version of your crate to try to fix it, likely even a cargo update would fix it if you make a new patch release. But users who depend on dependencies that depend on your previous version, may not be able to build at all without coordinating with those maintainers, or forking the packages that depended on you. So we just traded one form of build breakage for another, harder-to-solve form of build breakage.

Additionally, if I have a rust library and I do not commit it's cargo.lock file to the repo, (which is the recommended practice FAQ - The Cargo Book), yanking crates may break my ability to use git bisect to debug my library. This is because if I go far enough back in time to a point where somewhere in my dependency there are yanked crates, cargo may not be able to build my library at all, even if it once could. I could try to compensate for this by committing cargo.lock anyway, but this has other downsides as explained in the FAQ.

  1. Most of the time when an important crate is yanked, the result is github issues that ask "why was this crate yanked?" This is because yanking doesn't have a place for maintainers to explain the rationale, and there's no way for users to look it up. cargo audit and the rustsec advisories fix this.

If we made cargo yank instead return a message "cargo yank is being deprecated, please file a rustsec advisory instead, which your users discover by using cargo audit. If there is not a security problem, please just publish a new patch version.", it will have a number of benefits:

  1. Way more people would use the cargo audit tools and report rustsec advisories. That's great because these are great, well thought-out tools.
  2. Users would be able to easily get information to understand what the problem is with the crate, which they can't do with cargo yank.
  3. Cleanly separate "cargo build" from the issue of security advisories. It is very important to learn about security advisories when preparing for a software release process, but for development and testing, it is more important for developers to have precise control over what is built. Building old software is not bad, and may be necessary if using git bisect to investigate a bug. Even if security issues were discovered much later, preventing them from building in order to run tests may prevent them from doing their job. Releasing old software is bad, and that's the stage at which we need to discover outdated / broken dependencies.
9 Likes
2

If I have a crate version 3.4 and publish an update as version 3.5 noticing after the fact that what I published was actually breaking API changes so what I actually want is to have a new major version, then I would yank version 3.5 and publish version 4.0. By yanking, cargo update for users won't accidentally break their code by choosing version 3.5.

Is this a reasonable use case for yanking? What alternative approaches would exist without a cargo yank feature?

21 Likes
3

If I have a crate version 3.4 and publish an update as version 3.5 noticing after the fact that what I published was actually breaking API changes so what I actually want is to have a new major version, then I would yank version 3.5 and publish version 4.0.

I think what I would suggest is, publish 3.5.1 that is essentially the same as 3.4 (reverting whatever breakage was in 3.5.0), then release 4.0.

(It's not clear to me that in some configuration, cargo won't pick 3.5 even if it was yanked and then break the build, if there isn't a patch version that can be picked by semver in the 3.5 line, so I'm not sure if the yanking solution you proposed without such a patch release, would not be breaking your users in some case.)

It seems to be widely accepted that you should not yank unless there is a semver compatible release available, but this is not enforced.

8 Likes
4

If you yank the most recent version in a semver compatibility bracket, that will break version resolution for anyone with a Cargo.toml dependency on specifically that version, yes. If the manifest lists a range compatible with an older version, cargo update will actually update to the older version.

My personal guideline is that yank is fine iff there's a later version available within that semver compatibility bracket. This matches the (FAQ) guidance from the semver spec on what to do if you inadvertently break compatibility: release a new minor version restoring compatibility and document (but do not modify) the offending release as broken.

What do I do if I accidentally release a backwards incompatible change as a minor version?

As soon as you realize that you’ve broken the Semantic Versioning spec, fix the problem and release a new minor version that corrects the problem and restores backwards compatibility. Even under this circumstance, it is unacceptable to modify versioned releases. If it’s appropriate, document the offending version and inform your users of the problem so that they are aware of the offending version.

I also consider it reasonable to yank an entire semver compatibility bracket if it's properly unfit for any purpose (e.g. cannot be used without causing UB). This is, however, extremely uncommon, and if this is properly the case, I'd actually consider releasing a new version which is just a compile_error! explaining the crate's unfitness, rather than just yanking.

It's also worth noting that you can still ask cargo to use a yanked version, by specifying the precise version to use with cargo update --precise, I believe. If I recall incorrectly, there should be a "I know what I'm asking for" button to allow setting a yanked version to the lockfile.

If we do go about deprecating or even just restricting the use of yank on crates-io, I would like some other way to hide a crate. Yanking all versions of a crate has a convenient side effect of marking the cratesio page as "this crate has been yanked" and an immediate indication that the crate shouldn't be used if/when it shows up on cratesio. librs respects badges.maintenance="deprecated" and probably surfaces unmaintained rustsec advisories, but cratesio doesn't currently offer any way to mark a crate as "do not use" other than setting the README and description or yanking all versions.

Perhaps there's some space also for a "deprecate" command which won't outright yank a version, but will still have cargo update print a warning whenever it resolves to that version, rather than making it fully unselectable / an error.

As much as rustsec and cargo audit are the "correct" way to communicate such "yank a whole compatibility range" desire, it's still not an official built-in part of cargo. If cargo audit were a default subcommand, it'd be easier to suggest. Especially if we would do something like probabilistically run audit automatically during a build when it's not been run on a project for some reasonable time period[1].

  1. Complete spitballing: on any non-fresh cargo build, if $CI is not set, cargo is not offline and would be allowed to touch the network for dependency resolution, cargo has not alerted an audit on this machine this week (tracked by fingerprint in $CARGO_HOME), and cargo has not successfully completed an audit on this workspace in the past month (tracked by fingerprint in $CARGO_TARGET_DIR), then with no greater than 1% chance, run a cargo audit as well as the build (ignoring a network error and printing nothing extra on a clean audit result). (Might require some extra state finagling such that shared a system-wide shared target dir doesn't disable audits for unaudited workspaces improperly; we're tracking the system-wide cooldown separately.) A failed audit shows as a build warning, and does not error the build. Note I've deliberately queued this on build only and not check, but also that test includes a build if necessary. ↩︎

8 Likes
5

Unless it’s been added recently, this isn’t possible. To add a yanked version you have to manually edit the lockfile to enter the version number and checksum.

6 Likes
6

I am of the opinion that cargo yank should work similarly to the way it does on npm. There is a short (several days) grace period where you can yank freely, reserved for the case when someone messes a release and pushes a semver break or a horribly broken version. This grace period may be increased a bit (e.g. up to several weeks) if there are no known dependents for that crate version on crates.io. Once the grace period expires, yanking a version becomes impossible, because it is very likely to break someone's build somewhere (and it becomes much more likely that someone uses that version in a private crate).

This covers the "hot-patch a mistake" use case, while guaranteeing long-term stability of the ecosystem. It makes impossible stuff like the maintainer deciding that an error or UB in some obscure function warrants yanking 20 old versions altogether, after several years of being used by the people.

12 Likes
7

The last time I tried it, cargo accepts lock files without checksums (fills in missing ones) so it's actually pretty easy to manually edit the lock file in this way.

8

I think yanking is great. It's a great idea to yank releases that nobody should be using any more, e.g. releases with known vulnerabilities or incompatibilities with current rust versions. I think it should be used more.

The yanking feature should be extended — support custom message for yank reason. Perhaps even support optional forced upgrade to fix the problem of users accidentally staying locked to a yanked version.

9 Likes
9

I think yanking is great. It's a great idea to yank releases that nobody should be using any more, e.g. releases with known vulnerabilities or incompatibilities with current rust versions. I think it should be used more.

This illustrates my point -- many maintainers think that yanking because of incompatibilities with current rust versions is a good practice, and do this. But this is disruptive to anyone with a large project that can't upgrade rust versions without a lot of work. If there's nothing wrong with the previous release there's no reason to yank. Rust seeks stable and repeatable builds because many large projects critically need that.

Cargo gives you the right tools -- you can set a MSRV in your cargo toml file, but maintainers reach for yanking inappropriately, simply because it is there.

9 Likes
10

I am of the opinion that cargo yank should work similarly to the way it does on npm. There is a short (several days) grace period where you can yank freely, reserved for the case when someone messes a release and pushes a semver break or a horribly broken version. This grace period may be increased a bit (e.g. up to several weeks) if there are no known dependents for that crate version on crates.io. Once the grace period expires, yanking a version becomes impossible, because it is very likely to break someone's build somewhere (and it becomes much more likely that someone uses that version in a private crate).

This would be such a great change, thank you for suggesting this. It could be like, if you try to yank a crate that's more than a week old, it refuses and suggests you file a RUSTSEC advisory instead if there is a security problem.

3 Likes
11

You can still use yanked versions with a Cargo.lock if you need to reproduce old builds. This is where crates.io differs from npm, which used to delete releases completely.

3 Likes
12

The supported MSRV is for the opposite situation, when your code requires a newer version of rust. Yanking is for when an existing release of a crate no longer compiles or has miscompilations on a new version of rust (e.g. if it contained UB that was known not to be exploited on an older version), this would need the other MSRV (Maximum Supported Rust Version).

6 Likes
13

You can still use yanked versions with a Cargo.lock if you need to reproduce old builds. This is where crates.io differs from npm, which used to delete releases completely.

This is actually quite difficult to do, you would need to make a tool to do this if you need to do it as part of a bisect in a large project. It's a pretty bad user experience and it's not clear what benefit counterbalances this.

The supported MSRV is for the opposite situation, when your code requires a newer version of rust. Yanking is for when an existing release of a crate no longer compiles or has miscompilations on a new version of rust (e.g. if it contained UB that was known not to be exploited on an older version), this would need the other MSRV (Maximum Supported Rust Version).

Again, I think yanking is not an appropriate way to handle this situation. If you yank it, then people who are not on the new rust version are broken, for no good reason.

If it doesn't build, make a patch release that fixes it. If it has UB, you should file a security advisory.

Yanking it does not remove the need to do either of these things. The only thing yanking accomplishes here is harming your downstream users.

4 Likes
14

The idea is that you have Cargo.lock versioned in your repository, so when you bisect, you get all the exact dependencies as they existed at the time of the commit. Otherwise you have no reproducibility guarantees even without yanking, because Cargo picks latest dependencies, and not the versions in Cargo.toml (i.e. dep = "1.2.3" doesn't use v1.2.3, but v1.999.999 if there is such version).

UB or syntax fixes don't automatically increase MSRV. It's possible to fix future-incompatibility problem in a backwards compatible way, keep the same MSRV, and yank the old broken release. For example, fn foo(u8) stopped compiling in Rust at some point, but fn foo(_: u8) compiles on all Rust versions.

Note that the Rust project does not support use of any old Rust verisons. Only the latest stable is supported. This is why I find it more important to yank crates incompatible with the only supported Rust version, rather than keep old unsupported code that only works with old unsupported Rust versions.

13 Likes
15

imo they have to be pretty severe to justify being yanked. Vulnerabilities are not one-size-fits-all bad. For example, the regex vulnerability doesn't matter if I'm using it within my tests. Other vulnerabilities might be in a part of the crate I'm not using (I wish cargo audit could report only for whats in use). Let cargo audit and cargo deny deal with reporting vulnerabilities. Let's improve their integration into users workflows.

As was mentioned, the official docs and cargo new discourage committing Cargo.lock for libraries. So long as a semver-compatible upgrade is available, it at least shouldn't break people without a Cargo.lock. I do wonder if we should change this recommendation though to make life easier for git bisect and other workflows.

12 Likes
16

We absolutely should not. In fact, we should go further and state that Cargo.lock should never be committed at all. Committing Cargo.lock, regardless of crate type, makes bisection and reproducible builds marginally easier, but makes all other uses of the source repo significantly less ergonomic.

(I have said this before, e.g. in Feedback on `cargo-upgrade` to prepare it for merging - #122 by zackw. I promise to post an actual explanation for my position ... probably as a new thread ... sometime this week.)

2 Likes
17

I would be interested to see that reasoning. I prefer a checked-in Cargo.lock because I find the value of consistency between collaborators to be far higher than anything you get from not checking it in - if I see a given warning, all my collaborators will be seeing the same warning, since we're all using the same rust_toolchain.toml (hence same compiler version) and the same Cargo.lock with the same workspace.

9 Likes
18

Note that the Rust project does not support use of any old Rust verisons.

There's a big difference between saying, we don't offer support for previously released versions of rust, and "no one should be using this anymore".

You don't get to decide when your users update -- they do. They have to have this right, because they may be a large company with a huge code base, and they just can't move as fast as you. If using a previously released version of rust is "wrong" then rust stability is meaningless.

So yes, it may be true that "rust 1.55 is unsupported" in that no one is making patch releases or bug fixes for that, and if you find bugs that affect only that release but not the latest release, no one is going to care. It is extremely inconvenient to users who are still on that release if you yank crates just because you personally don't think anyone should be on rust 1.55 anymore. There are completely legitimate reasons to still be building code at rust 1.55, even if that release of rustc is "unsupported".

11 Likes
19

Yep; why were prior versions yanked? · Issue #37 · stephank/hyper-staticfile · GitHub is one instance of this. In my case, the vulnerability was Windows only, and we weren't using the software on Windows and never will. In that case, some sort of better default cargo/rustsec integration would indeed have been much nicer from my PoV.

5 Likes
20

At Cloudflare we power good chunk of the whole Web, we have several large Rust codebases, and we update the compiler the day it comes out.

Rust has committed to stability without stagnation and it largely works. It is however at odds with the completely opposite "Debian" approach that is more geared towards dealing with brittle code, and preferring to keep old known bugs rather than get bug fixes.

5 Likes