Dynamically linking cargo-subcommands against cargo

[gnzlbg]

I use a lot of cargo sub-commands and maintain two of them (cargo-asm and cargo-llvm-ir), and I have only seen cargo subcommands that either:

• depend on cargo “as a library”, which requires re-compiling cargo, and statically linking it - this takes a while, particularly on CI, and each cargo subcommand becomes huge because of it - or
• do not depend on cargo - these allows the sub-commands to compile fast but often results in the sub-commands emulating whatever they need from cargo in some brittle way.

There has to be a better way. It should be possible for cargo subcommands to use the toolchain’s cargo as a library without having to recompile some potentially different version of cargo and statically link to it.

I mean, I just installed cargo-tree in 11m 49s , I can’t use cargo-travis on CI because it takes too long to compile (It used to take 15-20 min on travis-ci, I think now it takes ~5 minutes, but this is still nuts).

[bjorn3]

It should be possible for cargo subcommands to use the toolchain’s cargo as a library without having to recompile some potentially different version of cargo and statically link to it.

That would mean updating rustc breaks all installed cargo-subcommands and also a cargo-subcommand only works in combination with some versions of cargo the library, because it doesn’t have any stability guarrantee.

[gnzlbg]

That would mean updating rustc breaks all installed cargo-subcommands

This is already at least partially the case. Updating rustc allows, e.g., using cargo features that were not available before, like the edition key in the Cargo.toml. cargo sub-commands compiled against older versions of cargo chuckle on this, so recompiling cargo sub-commands on toolchain upgrades is a good idea anyways.

and also a cargo-subcommand only works in combination with some versions of cargo the library, because it doesn’t have any stability guarrantee.

Note that the sub-commands would still specify the cargo version that they are compatible with. Dynamically linking does not change that. It only changes the binary size of the sub-commands, and the ability to upgrade to minor semver version bumps of cargo without re-compiling the world.

[cuviper]

That would mean updating rustc breaks all installed cargo-subcommands

This is already at least partially the case. Updating rustc allows, e.g., using cargo features that were not available before, like the edition key in the Cargo.toml . cargo sub-commands compiled against older versions of cargo chuckle on this, so recompiling cargo sub-commands on toolchain upgrades is a good idea anyways.

It’s worse than that though. Your ~/.cargo/bin/cargo-foo subcommand would be linked to a particular toolchain, and would immediately have trouble if you’re using rustup to manage multiple versions. For instance, if you built that with stable, then you couldn’t use it in projects that are overridden to use nightly, unless you remember to do cargo +stable foo every time.

[cuviper]

There has to be a better way. It should be possible for cargo subcommands to use the toolchain’s cargo as a library without having to recompile some potentially different version of cargo and statically link to it.

Maybe we could just ship libcargo.rlib? You’d still have to deal with semver migration, but at least it wouldn’t have the dynamic linking problems. I think this would require rlibs for all of its dependencies too though, and I’m not sure we have any way to guard that for stability purposes.

[gnzlbg]

An alternative would be to have cargo maintain a system-wide cache of compiled crates. That way one would only need to compile each version of cargo once, instead of once per cargo subcommand. Which maybe hints that I should just start to always use sscache or similar.

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.