`cargo install --locked` or not locked middle ground?

cargo install crate-cli has a dilemma:

  • Respecting the crate's lockfile makes dependencies more predictable, match what the crate author has tested with, and less likely to break due to semver violations in dependency updates.

  • Ignoring the crate's lockfile makes dependencies more up-to-date, possibly include security fixes, and less likely to break due outdated dependencies that became incompatible with newer compiler or operating system releases.

Different developers value/judge the stability vs risk issues differently, so there are strong opinions in favor of either option (e.g. having future security fixes may be more important than any trouble from updated packages, or that the released binary should keep working forever).

End users can choose the behavior with the --locked flag, but the default behavior is important — novice users may not know about the flag or its tradeoffs. Developers may be annoyed they need to instruct end users to add the flag (and another group of devs could be annoyed if the other option was the default and a different flag was needed for the other behavior).

Can cargo install have best of both somehow? Some ideas (assume every one of them can still have its own flag/config/opt-out):

  • run install without a lockfile, but if the build fails, retry with the lockfile. IMHO that would have best of both in the majority of cases (except an edge case where two different dependencies need opposite strategies at the same time), but at a cost of sometimes much longer cargo install time. It should be pretty easy to implement.
  • use the lockfile, but update yanked dependencies (perform equivalent of cargo generate-lockfile before install). This would depend on authors yanking broken crate versions (unfortunately some authors are unresponsive or prefer to keep all versions available in case anybody used them anyway). It should be relatively easy to implement too.
  • allow registries to explicitly mark releases that are security updates and update only these dependencies (IMHO insecure versions should be yanked, but some crate authors think that's too harsh, so a softer yank could solve it). This might even be integrated with rustsec or other vuln databases. It won't help with compiler compatibility issues.
  • Update locked dependencies' patch versions only (treating every locked version as x.y.* requirement), which would hopefully avoid updating to bigger breaking releases, but still allow crate authors to release bug/compatibility fixes. Currently the crates-io ecosystem doesn't really pay attention to patch releases for minor versions, but if Cargo had such behavior, there would be a motivation to support it.
8 Likes

What about allowing authors to specify the default for their crate? A configuration in Cargo.toml that says "use the lock file when installing this".

8 Likes

run install without a lockfile, but if the build fails, retry with the lockfile. IMHO that would have best of both in the majority of cases (except an edge case where two different dependencies need opposite strategies at the same time), but at a cost of sometimes much longer cargo install time. It should be pretty easy to implement.

Idea that's probably too hard to implement: What about run without the lockfile and for every dependency that fails (or somehow causes compilation to file) downgrade just that crate to the lockfile version and recompile that crate (and it's dependents) only. Saves a bit to a lot of time depending on the dependency

If it is a shared dependency, you might have to find some intermediate minimal version to satisfy it. Or downgrade other packages.

Imagine:

  • A -> B
  • A -> C
  • B -> C

A's lockfile says B 1.1 and C 1.0. Latest is B 1.3 and C 1.4, but C 1.4 fails for whatever reason, you can't use the lockfile because B 1.3 says it needs C 1.2 (say, for a new API). So you need to do a full solve again for this.

1 Like

cargo update -p --precise already has ability to partially update the dependencies. The bigger problem may be in identifying which crates are at fault, because the crate that fails to compile may not be the crate that caused the problem, and cargo doesn't have insight into the code to understand root cause of errors. Imagine if serde made a breaking API change: everything other than serde would have a problem. Trying to find working deps one by one by trial and error could make it much slower, since everything downstream from a changed dependency will need to be rebuilt.

Personally, I think I would prefer we just went with --locked always and required application authors to re-release when a security update is needed.

  • The lack of --locked only helps when installing. Most packages that I've cargo installed were installed years ago.
  • Most applications likely have to re-release anyways for pre-built binaries
  • cargo install isn't and (imo) shouldn't be treated as a general purpose installation tool, so having limitations is sufficient.
7 Likes

I've had cargo update -p --precise do nothing because of unsatisfiability problems. Alas, logging of why it failed to do anything is missing (no, I've not filed an issue about it; it is always already a sidequest to my actual goal and tacking more on is a great way to lose track of what that goal is).

I'd be thrilled with this, personally, as long as projects systematically updated and published regularly.

In principle, packages should always work with newer versions of crates, and cargo update should always work. In practice, there are things that break this. Most commonly not from crates that break API without a major semver break, but rather from dependency stunts (e.g. crates in the dependency graph that think an = or ~ or < dependency is a good idea).

2 Likes

There's one crate I maintain that recommends = dependencies, but that's because it binds to another API that may make changes that are Rust-incompatible. If usage is exposed, it is recommended to pub the crate again so that transitive users can use it without having to figure out a dependency version themselves. We have since figured out a pattern that is much more robust to Rust's idea of API stability, but the deprecation cycle for the definitely-not-API-stable patterns continues.

1 Like

Indeed I have found that cargo install has sometimes inconsistent behaviour. When I want to try a tool and the README says cargo install xxx and it fails with a compilation error, my first thought is : oh, ok this tool is not ready for production, forget it.

I learn by this discussion that it is not necessarily a negligence in developing the crate, but comes from tooling. So +1 for this discussion and finding a solution where when you release a crate, users can rely on reproductible installation steps.

1 Like

Hasn't it become the defacto default ? Except for rustup installed components, I don't recall other means of installation recommended by default by crate authors. It is systematically cargo install.

2 Likes

cargo install is the one and only Rust executable installation tool that works on all platforms where Rust toolchains can be run. So, it is generally suitable for installing command-line tools that are used for developing Rust programs, and that is why it is the default for that purpose. But:

  • It cannot install manual pages, Linux .desktop files, or data files that should accompany a program.
  • It cannot perform any build steps that would generate such accompanying files, if they are not already in the Cargo package, or perform steps which would modify or wrap the executable, such as macOS .app bundle creation.
  • It cannot create file associations or otherwise register the program with the OS or desktop environment.
  • It cannot integrate with existing application package management.

Therefore, there are many kinds of applications it cannot correctly install.

7 Likes