I'm curious to hear your thoughts on this. I've actually been meaning to propose the opposite: we should always check-in lockfiles, not just for binaries.
Hey, I'm also interested in your argument against committing lockfiles.
My opinion is that all projects should always commit their lockfiles. The reason is reproducibility.
crates.io).cargo update).In my opinion the benefits of long-term reproducible builds far outweigh the need to be explicit when you want to check behavior with the latest compatible dependencies. In particular they are invaluable when investigating regressions or trouble-shooting build errors. If a crate fails to build locally, but I see a commit with a lockfile and green CI it narrows down the source of errors to factors outside of the control of cargo.
To be honest I'm still a bit baffled by this recommendation in the cargo book:
Why do binaries have
Cargo.lockin version control, but not libraries?[...]
For libraries the situation is somewhat different. A library is not only used by the library developers, but also any downstream consumers of the library. Users dependent on the library will not inspect the library’s
Cargo.lock(even if it exists). This is precisely because a library should not be deterministically recompiled for all users of the library.
A Cargo.lock in a lib repo is obviously not for users but for lib devs. A library still has binaries, they're just called through cargo test instead of cargo run; why should they be non-deterministic by default? A Cargo.lock does not prevent running your tests against the latest dependency versions, it's easy. What's hard is manually resolving the dependencies at a given time when you forgot to commit the lock file in the first place.
This is also the stance of Yarn, which is IMO the most reliable package manager for Node:
Which files should be gitignored?
[...]
yarn.lockshould always be stored within your repository (even if you develop a library).
Yarn actually goes even further and recommends storing the Yarn version itself in the project. I recommend also reading their arguments (they have other ones, such as a missing lock file not being sufficient to ensure you use the freshest dependencies anyway). cargo can piggyback off rust-version so we probably don't need a cargo-version.
Also to have a message on the topic of this thread. A few months ago I posted a comment about a previous version of cargo upgrade complaining that it was harder to pull the latest version to test against them. Since then, cargo-edit was updated a few times and I'm very happy with the current iteration of cargo upgrade. Thank you for your work.
I'm a huge fan of this even though it seems relatively unpopular. Bisection is often difficult/impossible without a Cargo.lock. Even getting a clean build on a stale repo can be problematic.
This has lead to requests for a Cargo time machine that could try to do the resolution based on a historical index state, but there's a lot simpler solution: just check in Cargo.lock.
This is the approach Go's modules take, and I think lockfiles for libraries are maximally powerful combined with Go's notion of Minimum Version Selection (research!rsc: Minimal Version Selection (Go & Versioning, Part 4)). Obviously that's a much bigger change than simply including them.
I'm very curious to hear about this as well.
cargo-edit 0.12.0 is out. I have not (yet) revamped the UI but did smaller steps
--recursive changed from being on by default to the default being based on if compatible upgrades are being doneCommitting a lockfile doesn't mean it must be used. People are always free to rm it, do cargo clean, and build from scratch.
The lockfile is more like a documentation: it last worked with this particular set of dependencies. Useful for debugging all sort of problems with dependencies, like regressions and semver violations.
Looking to summarize a recent discussion on where to go from here. I've tried to go back and include some details from the thread but I likely missed some.
Expected user operations across cargo update and cargo upgrade
tokio_03)Note: "compatible" and "incompatible" are relative to the version requirements and not semver (though thats the default version requirement operator)
cargo update today
Cargo.lock, not Cargo.toml
cargo update -p foo@ver, @ver is used to disambiguate foo within the dependency tree (must be full version)
--precise ver takes a full version for replacing ver in the lockfile and must remain compatible--aggresive to recursively update foo
Considerations:
[[bin]] maintainers may want bulk updates of everything, most likely with a focus on compatible as incompatible might need hand updates[lib] normal/build deps: maintainers may want compatible updates by hand. incompatible bulk updates are good for checking of it works but sometimes they will need to be done by hand[lib] dev-only deps: like [[bin]]
-Zdirect-minimal-versions
-Zdirect-minimal-versions or a way to update requirements without updating the lockfileProposal 1: Deprecate cargo update in favor of cargo upgrade
cargo upgrade (formerly cargo update && cargo upgrade --to-lockfile)cargo upgrade --incompatible / cargo upgrade -i
cargo upgrade -p foo
-i, --pinned
cargo upgrade -p foo@verreq
--pinned needed?cargo update -p foo --precise ver?Proposal 2: Separate Commands
cargo upgrade only does incompatible (formerly cargo upgrade --incompatible allow --compatible ignore)cargo update --save does compatibleProposal 3: Merge Upgrade into Update
cargo update stays the same except...cargo update --save (formerly cargo upgrade --incompatible false --compatible true)
--save is needed feels off to mecargo update --incompatible (formerly cargo upgrade --incompatible true --compatible false)cargo update -p tokio_03 de-sugars the dep name to package name + version (tokio@0.3.12)cargo update -p foo --precise ver
--precise and not allow controlling the precision or operators, do we hack up --precises behavior, or find a new flag?ver is incompatible, do your need --save, -i, or either?cargo update --precise --save to fully specify all version requirementscargo update && cargo update --save --locked would be an error, mirroring cargo adds behavior which has --locked apply to the manifest as wellNote:
--edges flag which works when talking about graphs but not really in this contextlinks?
git2
I think the next step is updating cargo-edit to one of the proposals for us to gain first-hand experience with how it works out
I still don't think we should have --compatible true and --incompatible true; let's just spell those --compatible/-c and --incompatible/-i. If you pass either one, you just get that one; if you want both, pass both.
Of the various reasons given to not merge upgrade into update, the main one seems like the inconsistency of having to give --save to upgrade the manifest for compatible upgrades but not for incompatible upgrades.
I think that's worth not having two commands or a deprecated command, though, because I think it's a natural consequence:
--save would be redundant there.-c/--compatible option, and then having cargo update -c and cargo update -i both save changes to the manifest.So:
cargo update stays the samecargo update -c upgrades compatible and saves to the manifestcargo update -i upgrades incompatible and saves to the manifestcargo upgrade -c -i upgrades both compatible and incompatible and saves to the manifest.The options I gave dropped --compatible <value> / --incompatible <value>. The difference is in the details of what happens if no flags are passed in and whether both flags exist.
I disagree when it is as destructively described as --save.
However I think this is subject to how the flag is framed and framing it a different way (your --compatible) bypasses that. Sorry, I had forgotten to call this out when doing my write up.
I'm still a little cautious of this approach as there is still a gap in the concepts of --compatible and --incompatible . This isn't a "no", just a "eh".
Then I misunderstood your parentheticals; I thought you were giving "abbreviated form (long form)" rather than "new form (old form)".
Thanks for pointing out that unclear point; I've tried to clarify it
I kinda disagree here. For bins, if you're doing an update --incompatible at all, the goal is most likely to get all dependencies on their latest version, and there's no reason to not do so for dependencies that haven't released a breaking bump.
If you're being more precise about it, then you'd (either do so originally or back up and) probably do each incompatible dependency upgrade in an independent step and commit, not attempt to do all of them at the same time.
I still don't really see --incompatible true --compatible false ever really being a desirable workflow[1]; either you want to update all dependencies in one step, or you want to update single (trees of[2]) dependencies at a time to minimize the incremental patches needed to resolve the upgrade.
So my counteroffer would be I guess roughly
update writes to lockfile (as today)update --save writes to manifestupdate --breaking is an error, suggesting one of the belowupdate --breaking --save writes to manifest, upgrades allowed to cross compatibility rangesupdate --breaking --precise implies --save for just the specified dependencies (any transitive updates are lockfile only and not saved unless --save is also specified)A interestingly meaningful command might then be cargo update -bsp tracing-subscriber; this would upgrade your manifest dependencies on tracing-subscriber and tracing to the most up to date potentially breaking release.
Interesting (but potentially poor) idea: differentiate between version = "1.0.0" and version = "^1.0.0"; --breaking will update "default bound" version requirements across compatibility ranges but not move requirements using an operator across compatibility ranges. Rationale: using a non-default requirement mode is an advanced use case (which required manual editing of the manifest at some point rather than the use of cargo add and upgrade) and the user probably knows what they're doing and want better than the tool can guess it. As an escape hatch, a --force flag will change back to the default bound requirement again (if an update outside of the bound exists) and do the update.
That idea is why I used --breaking instead of --incompatible; I'm focusing on the fact that an incompatible upgrade of the default requirement mode is potentially breaking (and thus not done without an opt-in), rather than that it's incompatible with the version requirement.
Even for libs that want to keep low requirements within compatibility ranges, I'd describe what they want not as "upgrade incompatible to the most recent" but "upgrade all to the lowest compatible in their most recent compatibility range;" it's not a difference in the upgrade they're doing but in the version selection preference. ↩︎
Things are a bit more interesting when multiple dependencies related by a public/peer dependency are involved, as you do want/need to upgrade them together. ↩︎
With the current cargo-edit version is there any way to upgrade all dependencies for one crate within a workspace? Or even upgrade a version requirement in one crate but not another when multiple in the workspace have the same dependency? It seems like it's only capable of operating in --workspace mode currently. (I have a few repositories containing separate library and binary crates so I commonly want to treat their dependencies differently).
Currently, everything is workspace-wide. The motivations were
I think this is the first time since that change that someone has expressed interest in handling upgrades per-workspace-member. Feel free to come up with a counter proposal for how it can work. I'm assuming supporting this would push us away from merging cargo-upgrade and cargo-update.