We are struggling with how to set up cargo. The way we publish things inhouse, the cleanest would be one registry per internal organisation. Artifactory has federated registries for other languages but not for cargo. This seems to be because cargo itself does not offer such a feature.
Currently that goal requires more configuration, needing an up-to-date config.toml to be shared. Fine for CI, where it is in our container images. But not for projects that want to be buildable locally, hence have their own config.
Also every dependency then needs to specify the registry to load it from. That would almost prevent moving an inhouse crate responsibility to another organisation.
At least the per-dependency overhead would go away with:
[registries.federated]
members = ["inhouse-a", "inhouse-b", "crates.io"]
Since Cargo.lock remembers the source, crates would not automatically do a switcheroo, if the same name also gets published in a registry that comes earlier in the list. IMHO cargo should issue a warning if several federated indexes contain the same dependency name. The warning would go away by explicitly specifying the actual registry to fetch that crate from, as is done now.
If this were easy for Artifactory & Co. to build upon, that would solve all problems, by centralising this. In that case collisions should be resolveable both in the federating server and in Cargo.toml.
Is this a good idea to implement in cargo? Or are we missing another solution that is already possible
As a heads up, it isn't fully clear what you mean by terms like "federated registries" and say that Artifactory doesn't support something because Cargo doesn't without explaining how the two parts intersect.
Judging from this statement:
Since Cargo.lock remembers the source, crates would not automatically do a switcheroo, if the same name also gets published in a registry that comes earlier in the list.
I assume what you are looking for is being able to implicitly read from multiple registries without having to specify them. This is avoided in Cargo because it leads to Dependency Confusion Attacks. I had gotten the impression that Artifactory had a way of supporting this on the registry side but I have no idea how accurate that is.
I guess you got me right. We want one registry comprised of crates.io, plus any crates published inhouse (technically to several registries, because of admin & permissions.)
Artifactory does offer that, e.g. for Java. But if cargo doesn’t offer the backbone for this, they would themselves have to merge indexes and implement the sparse protocol for that. And they haven’t done that.
The Dependency Confusion Attacks would be avoided by sticking to Cargo.lock if available. While cargo add would refuse without --registry if ambiguous. And cargo build/update would warn if Cargo.toml doesn’t clarify where from to fetch a crate that has since become ambiguous. This warning could optionally be promoted to error, e.g. to avoid ambiguity in CI/CD.
In order to figure out if a dependency has become ambiguous, you need to ask all registries if it has entry by that name. this side channel leak is one of the methods the original blog post about dependency confusion attacks used to find the internal secret names of internal secret libraries used inside companies.
If you want to be ambiguous about where your dependencies come from you need to use source replacement feature, and point the default registry at a provider that implemented that functionality. Artifactory chose not to provide such functionality. Others have implemented that functionality. Without making an endorsement, CodeArtifact is/was proud of their implementation.
If you actually need to move a crate to a different registry, you can publish a package to the old registry that uses the package from the new registry as its own dependency and re-exports it (pub use new_package::*).
Having to specify the exact registry you want is much safer. crates.io is open and allows anyone to create new packages at any time. This can be dangerous even when the names are technically unambiguous, because crates.io could have typosquatted names that you'll accidentally run if you misspell your private crate names.
The syntax isn't too bad. You use the nickname you gave your registry in the config, so you can make it something short. package = { registry = "a", version = "1" }.
Having to deploy the config file everywhere is a chore. Some things (like auth tokens) can be specified via env vars, which makes it a bit easier in CI setups.