Pre-RFC: Cargo alternative registry authentication

There have been several attempts at getting support for authentication for private registries. I'd like to get that discussion going again. This RFC takes in to account the recent developments in HTTP-based registries (RFC2789), and attempts to address details that were missing from previous proposals.

I have a prototype of most what what's described here, and would be available to continue the implementation.

Summary

Enables Cargo to include the authorization token for all API requests, crate downloads and index updates (when using HTTP) by adding a configuration option to config.json in the registry index.

Motivation

Organizations need a way to securely publish and distribute internal Rust crates. The current available methods for private crate distribution are awkward: git repos do not work well with cargo update for resolving semver-compatible dependencies, and do not support the registry API. Alternative registries do not support private access and must be operated behind a firewall, or resort to encoding credentials in URLs.

There are many multi-protocol package managers: Artifactory, AWS CodeArtifact, Azure Artifacts, GitHub Artifacts, Google Artifact Registry, and CloudSmith. However, only CloudSmith and Artifactory support Cargo, and they resort to encoding credentials in the URL or allowing anonymous download of packages. This RFC (especially when combined with the approved http-registry RFC) will make it significantly easier to implement Cargo support on private package managers.

Guide-level explanation

Alternative registry operators can set a new key auth-required = true in the registry's config.json file, which will cause Cargo to include the Authorization token for all API requests, crate downloads, and index updates (if over HTTP).

{
    "dl": "https://example.com/index/api/v1/crates",
    "api": "https://example.com/",
    "auth-required": true
}

If the index is hosted via HTTP using RFC2789 and Cargo receives an HTTP 401 error when fetching config.json, Cargo will automatically re-try the request with the Authorization token included.

Reference-level explanation

A new key, auth-required, will be allowed in the config.json file stored in the registry index. When this key is set to true, the authorization token will be sent with any HTTP requests made to the registry API, crate downloads, and index (if using http). If a token is not available when attempting to make a request, the user would be prompted to run cargo login --registry NAME to save a token.

The authorization token would be sent as an HTTP header, exactly how it is currently sent for operations such as publish or yank:

Authorization: <token>

Interaction with HTTP registries

The approved (but currently unimplemeneted) RFC2789 enables Cargo to fetch the index over HTTP. When fetching config.json from an HTTP index, if Cargo receives an HTTP 401 response, the request will be re-attempted with the Authorization header included. If no authorization token is available, Cargo will suggest that the user run cargo login to add one.

To avoid the overhead of an extra HTTP request when fetching config.json, the user can optionally configure Cargo locally by setting auth-required = true in the [registries] table. If the local auth-required flag is true then Cargo will always include the Authorization token fetching config.json over HTTP -- skipping the initial unauthorized requiest and HTTP 401. The local configuration option does not impact other operations, such as API requests or downloads. It also does not impact git-based registries.

[registries]
my-registry = { index = "https://example.com/index", auth-required = true }

Security considerations

If the server responds with an HTTP redirect, the redirect would be followed, but the Authorization header would not be sent to the redirect target.

The authorization header could only be included for requests using HTTPS, or requests targeting localhost. Since the authorization header needs to be kept secure, it should only be transmitted over a secure channel. For registry development, localhost is also permitted, since the token would not leave the machine.

Interaction with credential-process

The unstable credential-process feature stores credentials keyed on the registry api url, which is only available in after fetching config.json from the index. If access to the index is secured using the authorization token, then Cargo will be unable to fetch the config.json file before calling the credential process.

For example, the following command would need to download config.json from the index before storing the credential. cargo login --registry my-registry -Z http-registry -Z credential-process

To resolve this issue, the credential process feature would use the registry index url as the key instead of the api url.

Since the token may be used multiple times in a single cargo session (such as updating the index + downloading crates), Cargo should cache the token if it is provided by a credential-process to avoid repeatedly calling the credential process.

Command line options

Cargo commands such as install or search that support an --index <INDEX> command line option to use a registry other than what is available in the configuration file would gain a --token <TOKEN> command line option (similar to publish today). If a --token <TOKEN> command line option is given, the provided authorization token would be sent along with the request.

Prior art

The proposed private-registry-auth RFC also proposes sending the authorization token with all requests, but is missing detail.

NuGet first attempts to access the index anonymously, then attempts to call credential helpers, then prompts for authentication.

NPM uses a local configuration key always-auth. When set to true the authorization token is sent with all requests.

Gradle / Maven (Java) uses a local configuration option for private package repositories that causes an authorization header to be sent.

git first attempts to fetch without authentication. If the server sends back an HTTP 401, then git will send a username & password (if available), or invoke configured credential helpers.

Drawbacks

  • There is not a good way to add the authorization header when downloading the index via git, so the index authorization will continue to be handled by git, until the http-registry RFC is completed.
  • Requires a breaking change to the unstable credential-process feature, described above under "Interaction with credential-process".

Rationale and alternatives

This design provides a simple mechanism for cargo to send an authorization header to a registry that works similar to other package managers. It would even work on a static HTTP server configured with basic authentication, since the token could be set to Basic <base64_encoded_credentials>.

Alternatives:

  • Don't add any configuration options to config.json or the [registries] table and rely on the auto-detection method for everything by first attempting an unauthenticated request, then on HTTP 401, the request would be re-tried including the token. This carries more risk of the token being sent when the server may not be expecting it, but would avoid a configuration option for the registry operator. It also would require more HTTP requests, since each type of request would need to be first attempted without the token.
  • Don't add a configuration option to config.json and rely only on the local configuration in the [registries] table. This avoids the auto-detection, but requires configuration from the user, which could be set up incorrectly or missed.

Unresolved questions

  • Do registries need a way to specify which API requests require authorization? Or is true/false sufficient?

Future possibilities

The credential-process system could be extended to support generating tokens rather than only storing them. This could further improve security and allow additional features such as 2FA prompts.

6 Likes

I have a pre-pre-rfc that I was toying with, it's not done yet, slightly snarky in places.

Differences to the above seem to be:

  1. Makes private registries first-class.
  2. Don't require config.json changes
  3. You don't need an index
  4. You can generate header values using credentials (specify arbitrary headers)
  5. both of ours might be amenable to having a bearer token leaked on a hijacking, but I think maybe this one is worse in that it's just going to send one if it gets a failure.

The more I think about this, the more I think we should be able to configurably validate https certs (so, local cargo trusts some pub cert, then the server can give you a pub cert to trust for the api, and both are overridable with configuration), but I don't want to pend this particular feature (I see it as a fix for a lacuna) on that.

The argument against mine is complexity, probably. I think the config paths are sufficiently different that they don't conflict.

IMO, manually configured certs are a sharp edge and an antipattern. If you want to trust an internal CA, it should be in the system trust store.

Much of the tooling around rustls, for example, makes the system trust store the preferred and only supported way of using an internal root CA certificate.

6 Likes

This is a reasonable point.

In plenty of cases Rust is installed in user accounts, that don't have access to the system certificate store.

It would also be helpful to use client certificates for authentication.

1 Like

Agreed. Another use case is to help prevent state-sponsored CA shenanigans by giving a program a smaller set of certificates to avoid some of the moreā€¦problematic entries in the global store (e.g., I give offlineimap the exact CA cert chains needed to communicate with my email hosts).

It's true that a user might not have access to modify the system certificate store, but in many corporate environments, they might reasonably expect that the necessary root certificates are already in the system trust store.

But I'm wondering if the certificate discussion is necessary at this point. I don't think anything in the pre-RFC precludes a future enhancement related to certificates.

Personally, I'm interested in using my companies internal gitlab instance as a place to host a registry. This is already possible today if the gitlab instance/project is configured to not require authentication. With the changes proposed in this pre-RFC, I believe cargo would have everything needed to work with gitlab authentication. So :+1: from me.

2 Likes

Hi! I've added authentication for HTTP registries some time ago in a Cargo branch and implemented the HTTP registry feature in Kellnr. It differs a bit from the proposed implementation, as it was not available back than and never merged as it was just my attempt at getting into the code of Cargo. But if you need help to implement the final RFC in Cargo, I'll be happy to help!