Add a cargo command for importing a crate into the workspace

tools and infrastructure cargo
1

Somewhat of a Pre-RFC to extend the cargo for better integration in larger build environments.

Short description

Add a new pull subcommand that allows resolving, downloading, and importing a crate into the current workspace as if it had been written as part of the workspace.

Interface

cargo pull [options...] <crate> <version>

Options:
    --path <path> The local path to extract the crate into.
    // and all the other common options.

Motivation

I'm currently working on $system for $employer and we made the decision to have one of the internal modules written in Rust. (Why and what doesn't really matter). This revealed a bit of a pain point in tooling compared to a full-stack Rust or using Rust/cargo as the top-level build tool. Basically, a C++ component using CMake needs to make use of a specific Rust crate as a staticlib. This works pretty well on its own using Corrosion when all sources are local.

However, we want to make the Rust code an entirely separate crate, published and versioned independently. The CMake tooling should download the latest fitting version of a crate using the usual registry integrations configured in cargo. All of its commands, more or less, depend on some local root crate being available, which is not the case. Even cargo vendor, used for fetching and extracting crates to the local cargo workspace, requires one local root crate to specify dependencies.

There are other tools for downloading crates but they do not use its inner workings. Basically, alternate registries and other .cargo/config generally does not work as expected. This makes automation and credential management extra difficult.

Functionality

Resolves the name with the version specifier to a crate. Then downloads the crate, extracting it to a relative directory named after crate. Then adds the crate to the local cargo workspace in which the command has been ran. This should do some basic checks to ensure that we don't accidentally overwrite anything. It will also compose well with cargo vendor when one wants to import a whole tree of dependencies.

Unresolved questions

  • Bikeshed the name. I've used docker for orientation but that's possibly controversial.
  • How does this interact with other crates that are already in the workspace? In particular, when depency bar of the pulled foo is present, should we rewrite the dependency to point to it by path?
2

By "local workspace" you mean a cargo workspace? Or a cmake workspace?

3

I mean a cargo workspace (edited the text). As in, for the use case I would create an empty skeleton workspace and populate it with all root crates by pulling them, one-by-one. Then afterwards it's possible to run Corrosion on them without further issues. Or vendor all of their dependencies in one swoop.

4

I'd personally love cargo new inside workspace to cause the new crate to be added. Adding it manually is annoying.

1 Like
5

If your root Cargo.toml contains

[workspace]
members = ["crates/*"]

then you don't need to add any crates to the workspace. Just run cargo new crates/new-crate.

EDIT: I mean that by using a glob in the members field, you don't have to update the Cargo.toml every time you create a crate. I'm not saying that cargo new should download and vendor crates.

1 Like
6

An interesting idea to leave out adding, does this address the fetching part? Are you meaning to imply that new might just be expanded to perform that task?