Suggestion: cargo yank is a misfeature and should be deprecated and eventually removed

I haven't committed a Cargo.lock for any of my Rust libraries for the maximal possible amount of time (i.e., since Cargo was released). Many of those libraries are serious and the CI is quite effective.

To be clear, I don't have a super strong opinion here and I don't mean to suggest one shouldn't commit a Cargo.lock. If I'm being honest, it probably would be useful to test both with and without a Cargo.lock in CI. Testing without is useful because it alerts you to upstream issues in dependencies at the cost of reproducibility.

But my point here was to counter-balance what I see as extreme hyperbole in your position.

3 Likes

I believe you are also in a unique position of maintaining fundamental libraries with few dependencies. I can understand how you wouldn’t necessarily benefit from a lock file, if that is actually the case. And yes, fundamental crates are fairly serious projects; no disrespect intended.

For other libraries like bevy plugins, which could pull in many hundreds of dependencies, the reproducibility guarantee becomes exponentially more important, and practical.

4 Likes

What if crates.io rebuilt all packages itself, and didn't publish anything that wouldn't build?

Stronger version: What if crates.io accepted only a source upload plus the expected hash of a tarball of the build artifacts, rebuilt the package itself, and wouldn't publish if it didn't get exactly the same bits you got?

Rate-limit if necessary to stop people from using crates.io as a build oracle.

crates.io already processes an upload bandwidth where this isn't very practical. docs.rs only does docs builds, which are less resource intensive than a full build, and somewhat regularly ends up with a queue backlog.

Plus, crates can and often do end up relying on system libraries to build successfully (e.g. *-sys packages). These aren't necessarily available on whatever builders crates.io would use. Or perhaps the "doesn't build" is on a non-default combination of feature flags.

Additionally, the stronger scheme is implicitly reliant on reproducible builds. Rust is only mostly reproducible when built from linuxes, and often not on Windows (because of temporary paths leaking into the debug info, iirc).

1 Like

I think I agree with that. But having the Cargo.lock in the repository is a prerequisite for doing this!

Currently, what even is the best way to test without a lockfile? There is no --unlocked so I supposed it's rm Cargo.lock?

3 Likes

I use cargo update --aggressive in my "updated dependencies" CI jobs .

FYI I have an active proposal to the cargo team for checking in the Cargo.lock which includes encouraging people to test with the latest dependencies.

At least from the docs, --aggressive sounds superfluous

2 Likes

Yeah, it's supposed to only do stuff with -p, but it makes it sound like it'll do more :grin:

Just going to mention briefly here that after an offline conversation with @epage and thinking about it a bunch more, I no longer think that checking in Cargo.lock is universally a bad idea. However, I think that it is still often a bad idea and should not be done without careful consideration of the consequences, and, in particular, that the cargo documentation is (in its present state, never mind the pending proposal) significantly more sanguine about it than I would be.

Most of my concerns about the practice would go away if all cargo commands would consistently, by default, discard and re-create any existing Cargo.lock unless the workspace has already been built at least once. You could still ask for it to be used on the initial build of a fresh checkout by specifying --locked.

I think it's probably best if further discussion of dependency locking moves to the proposal linked by @epage; this was supposed to be a thread about cargo yank, after all :slight_smile:

1 Like

I thought I understood yank, and in fact I had just read the section in the Book about it last weekend, but in fact I didn't realize that even without --locked, there's no easy way to make Cargo find and fix yanked dependencies without just deleting the lock file or running cargo build.

Especially in the context of the linked proposals for checking in Cargo.lock in more cases, this seems like a bit of a footgun to me. yank implies, to me, that reproducible builds are the only valid reason to continue using a specific version; the developer has taken somewhat drastic action to stop people from using it in the future. But anyone who has a lockfile with the yanked version will not only get that version, they won't even see a warning about it, even if cargo re-downloads the package from the registry. If the developer doesn't want to risk updating every dependency just to fix the yanked ones, then the only solution I'm aware of that's even partly automated (and indeed, the way I actually discovered the above issue) is to use a third-party tool such as cargo deny to determine what has been yanked and then update those specific packages.

I realize that in general, developers should be able to trust the ecosystem enough to run cargo update pretty freely. But there are certainly cases when it would be nice to know that a cargo update is warranted (even without using a third-party tool like audit or deny), and there are also cases when a minor or patch release can inadvertently cause a breaking change downstream, making a blind cargo update less appealing.

6 Likes

I've opened a feature-request for cargo audit fix to un-lock yanked package versions: Request: `cargo fix` should un-lock yanked packages · Issue #917 · rustsec/rustsec · GitHub