Performance of `cargo package` / `cargo publish` and the purpose of the verify step

tools and infrastructure cargo
1

In short, I'm looking to feedback on the following questions:

  • What packages or workflows are sensitive to cargo package or cargo publish times?
  • Should cargo publish's verify step focus on smoke testing the packaging process or be a last-ditch quality check in case there isn't a CI?

cargo package and cargo publish run a verify step by default. When testing cargo publish --workspace (cargo publish multiple packages at once · Issue #1169 · rust-lang/cargo · GitHub), I noticed how slow the verify step is and realized it is running cargo build instead of cargo check. Publishing packages is an infrequent task and the performance likely doesn't make much of a difference (well, except for aws-sdk). I did go all of these years without realizing thinking more about how slow it is.

Where cargo publish time can make a difference is in dry-run support. Previously, workspace release tools, like cargo-release, had to skip the verify step in dry-run modes because there wasn't a way to overlay the newly generated packages on top of the registry to build dependent workspace members. With cargo publish --workspace --dry-run, that is now possible. This makes what was a fast dry-run to be very slow. Dry-run can be run part of the development cycle while tweaking the release process. The full build isn't relevant to the that, especially when tweaking non-publishing steps like regex-based custom version updates. Users would hand-tweak the command to get the subset of functionality that is relevant but that requires extra work on the users behalf.

Switching the verify step from cargo build to cargo check would reduce the amount of time it takes and give the same amount of coverage for "is this packaged correctly". What it loses is any post-monopolization diagnostics (including linker errors). With Cargo Check T-lang Policy by Lokathor · Pull Request #3477 · rust-lang/rfcs · GitHub, cargo check is officially not guaranteed to report all errors and code that passes only cargo check is not subject to Rust's compatibility guarantees.

Whether the differences between cargo check and cargo build are relevant depends on factors like:

  • Our general assumption of what users do before publishing (cargo test? run all of CI?)
  • The likelihood that a step from above is missed before release
  • The likelihood of post-monopolization errors being introduced in that gap
  • The number of affected users if a problem does slip through (are more popular crates likely to have more rigorous processes?)

This also gets into the larger question of what the role of the verify step, whether its

  • Is this packaged correctly
  • Last-ditch quality check
    • Currently only builds (no clippy, miri, etc) the default build-targets using the dev profile with one feature set on one target-platform
    • Doesn't run tests (see earlier for problems here)

We could potentially make the verify step configurable (see also `cargo package` could run tests in the packaged tree · Issue #14685 · rust-lang/cargo · GitHub) at which point this becomes a question of defaults before and after this is configurable.

2

Without offering opinions on the larger issue, skipping cargo build skips linking, which is a huge amount of the time, but also often a source of issues for a certain kind of project (executables that link against system libraries probably being the main set). Whether that’s desirable is separate, but it probably shouldn’t be glommed in with “post-monomorphization errors”, even though technically…

(I personally would expect cargo publish --dry-run to do a dry-run build as well. But again, I’m not a crate maintainer, so my opinion isn’t so important here.)

2 Likes
3

One type of post-monomorphization error is a constant evaluation error. Constant evaluation errors could result from packaging mistakes (e.g. if using a macro like include_dir!() and processing the found files in constant evaluation).

In general, I think that cargo publish is something that should not be casual and it isn't a big deal if it takes longer than strictly necessary.

2 Likes
4

I've updated it to call out that it includes linker errors.

1 Like
5

Keep in mind that this isn't just cargo publish but

  • cargo publish --dry-run for the use cases I called out
    • I'd also love to see this integrated into more CIs once we have the --workspace flag which will be slower than a regular cargo build because (1) it will build serially, (2) it won't reuse intermediate build artifacts (we could look at loosening that limitation)
  • cargo package which I've worked with people who were it an integral part of their workflow
1 Like
6

Did you mean this the other way around?

7

Fixed

8

The pre-publish check has saved me on a couple of occasions when I was too hasty to publish.

I work with sys crates a lot, so linking check is useful to me.

If you're looking for a compromise, then cargo check may be ok for crates that don't have any -sys deps, or maybe even for crates that don't have build.rs themselves (under assumption that build.rs may be compiling extra stuff and checking it links is important).

But in general I don't mind the pre-publish check. When I'm in a hurry or scripting my own test+publish, --no-verify is there.

3 Likes