[Pre-RFC] Domains as namespaces

tools and infrastructure cargo
1

Summary

Provide a namespacing facility which addresses the following requirements:

  1. Allow claiming a namespace exclusively, thereby restricting which crates can be published under this namespace. This allows addressing the squatting issue.

  2. Avoid disputes over namespaces, which can take massive amounts of time and resources to manage.

  3. Guarantee that namespaces are unique, solving the issue of "clashing" namespaces. Non-unique namespaces work poorly and cause disputes, as demonstrated by npm's kik conflict.

  4. Avoid creating a hard dependency on external services. The recent GitHub service interruption is a good example why this is important.

  5. Make receiving a namespace as seamless as possible for users. Namespaces should be an additional benefit crate authors receive, not some annoying bureaucratic hurdle they have to jump over.

Design

  • Use (sub)domains as namespaces. Maven has shown that this approach has been working exceedingly well for more than a decade.
  • Control of a (sub)domain is established using domain validation (DV) and RFC5785. Let's Encrypt is successfully using this approach to generate millions of certificates each year.
  • Crates.io maintainer will never need to get involved in namespace disputes, as ICANN's Uniform Domain-Name Dispute-Resolution Policy deal with this.

Basic User Experience

Basic task: I want to publish my abc crate under xyz.github.io.

  1. I add organization = "xyz.github.io" to my Cargo.toml.
  2. I run cargo publish.
  3. The command prints the following text to the console:

    Crates.io couldn't verify that you own xyz.github.io. To prove ownership, place a file with the following contents

    ZG2ioiTY3tMgwDYfmb6p

    at https://xyz.github.io/.well-known/cargo-publishers

  4. I place the file "cargo-publishers" with the randomly generated string from crates.io at the specified address.
  5. I rerun cargo publish. crates.io checks that the string matches and publishes the crate under the namespace.

Details

Upon the first request to publish a crate with a namespace, crates.io generates a random string that is user-, crate- and domain-specific.

On each following request crates.io checks whether a file named "domain-owner" at the given domain exists and verifies that the contents are identical to what crates.io has on file for the specific user/crate/domain combination.

The ability to publish a crate is not limited to one person: If a different person wants to publish a crate to a domain, crates.io returns a similar reply:

Crates.io couldn't verify that you own xyz.github.io. To prove ownership, add the following text to the "domain-owner" file at your domain root: Io3487xY3tjg54Ef9Opz

This person would then contact the owner of the domain, who could decide whether to accept or reject this request (by adding or not adding the line to his "domain-owner" file).

This also serves as a very easy way to allow external contributors to publish add-on crates to a crate in an organization, without allowing them to publish to the main crate. E.g. authors of serde could publish their own crate under serde.rs/serde and allow vetted, "official" add-on crates to be published by external contributors under e. g. serde.rs/serde-abc or serde.rs/serde-xyz by adding those users' verification line to their "domain-owner" file.

Changes

This requires some changes to cargo and crates.io:

  • cargo needs to support

    • adding organization keys to Cargo.toml files and treat them correctly
    • adding dependencies on namespaced crates to Cargo.toml files

  • crates.io needs to be extended

    • to understand organization keys and namespacing
    • to validate control of (sub)domains as specified above
    • to correctly display such crates on the web

No changes to the language are required. Source code of crates does not need to be changed to enable namespaces.

Further Suggestions

It is recommended that / is used as a separator between the namespace and the crate name.

This would mean that a fully-qualified crate would look like an URL, making it easy to find further documentation or the project's homepage:

Consider a crate called abc with the namespace xyz.github.io. The fully-qualified name would be xyz.github.io/abc. The namespace owner could then place documentation under this URL, forward to the crates GitHub page, or redirect to doc.rs.

Future Tasks

This proposal does not discuss what should happen with the global namespace. This is intentional – there are plenty of policy decisions that can be made with various advantages and disadvantages.

There are valuable discussions to be had about this, but I think it makes sense to discuss this separately, so that disagreement about how to handle the global namespace doesn’t impede the debate about making namespaces as good as possible.

FAQ

This FAQ is not meant as a complete enumeration of all possible scenarios, but is intended to show how this proposal functions, and which potential policy decisions can be made:

Do I need my own domain?

No. Any subdomain is sufficient. Everyone participating here already owns one, and they are not hard to come by.

I no longer care about maintaining the domain, what should I do?

Ask your maintainers or contributors to take over the domain. (This is not much different from a situation where the original author loses interest in maintaining his/her crate.)

My domain was hacked!

Use your domain registry's support channels to resolve this issue. In the meantime, it might make sense to let crates.io maintainers know, so that the publication of new crates or versions can be disabled until the issue is resolved. (This is not much different from your crates.io account being compromised.)

Someone else owns my domain now!

Pick a new domain. If there is an acute danger of misuse, let crates.io maintainers know, so that publications of newer versions of existing crates under that namespace can be blocked.

18 Likes
2

That’s an interesting approach. I have a few questions, though: Would new crates necessarily need a domain associated with them? And what about existing crates? Would it go effective the first time they publish with the domain?

3

Is there a standard for domain verification available anywhere? The only one I can find is just the general ACME draft standard, but that’s a much more complicated protocol than what your example shows.

Personally if domains are to be used as namespaces (which I’m not really for or against), I would much prefer to use DNS validation than have to update files on my HTTP server.

1 Like
4

My main concern would be that domains can expire and change owners. Let’s say I publish my crates under my domain, then loose interest in Open Source work. My domain expires and someone else buys it. Do they now control my crates? Will I ever be able to publish new versions of my crates?

Also, as mentioned on reddit as well, the “domain owners” file should use a more specific name and go into “.well-known”.

Additionally, I believe that the “domain owners” file should have a more defined structure, or at least support some kind of comments, so that when editing the file, one can know which verification code corresponds to which user or crate (in case one wants to revoke specific permissions).

Another note: While maven uses domains to identify packages, I don’t think they actually validate that one owns the domain.

21 Likes
5

(post deleted by author)

7

I hope it will never be accepted.

Right now I don’t own any domain - I don’t deserve to publish a crate?

If domain of my project is “nudebeachgirls.fun”, will users use my crates for cryptography?

If domain is expired I can’t update crates anymore - it’s just ridiculous.

If I buy domain “rustlang” in some exotic tld, will it increase my chances for phishing activity?

It’s bad idea.

5 Likes
8

I really like the idea of a play/test namespaqe that’s easy to get permission to write under that won’t clog the main scope

1 Like
9

the global scope can remain open, along with many other semi official, open domain scopes. that are easy for users to publish to

10

I admire the enthusiasm, but I have some reserve with the process with regard to long-term maintenance and security.

  1. If the “owner” loses access to the domain, which happens relatively often, then what? Do they have to acquire a new domain, move the crate over to that domain, and ask all clients to migrate over (manually)? That’s a lot of churn!

  2. There are several issues around security/authentication that I can of:

    • If I take over a domain, can I publish the same crate (name) with my own user?
    • If I have an official looking domain (rustaceans.cam, rustlang.cam), it really opens up phishing.

I do agree that some what of having “organizations” or “namespaces” would be great; but I am not clear that your proposal is significantly better than just letting users claim organizations and namespaces (like Maven does).

7 Likes
11

The core point is that crates.io would be a collection of facts along the lines of “at the point in time crate X was published, the author Y controlled domain Z”

I fail to see what the domain information actually adds to the picture. Another way to frame what you're saying is that crates.io would maintain a time-varying mapping of domain aliases to crates.io accounts—you'd be using a (domain name, time) tuple to look up an account, then retrieving the crate with the given name.

Why not just use the crates.io account name for namespacing directly? That would eliminate that extra layer of indirection and vastly simplify the implementation. It still hits all of your requirements, as GitHub usernames are already namespaced, and doesn't appreciably increase our dependency on GitHub, since crates.io uses GitHub OAuth already. A service interruption might affect the community's capacity to publish new packages, but it wouldn't affect the namespacing information we would already store.

12

domains as namespaces without domains as independent, federated registries isn’t a very good solution IMO.

I’ve said it before: federated means you can use crates.io as a caching proxy for other, official project registries.

by using .well-known, we would also allow the registry to exist on the main website, e.g. diesel.rs could setup a registry on diesel.rs so that you don’t have to use a subdomain when specifying the crates.

the main benefit is that you get official registries, rather than using rust’s generic/general purpose registry. as a side effect, you also get independent moderation, and the crates.io moderation team would no longer be overwhelmed with moderation tasks. in the end, both of these contribute to a better organized registry “fediverse”, and IMO it would be more efficient than simply “domains as namespaces”.

the main risks (which IMO are benefits) is that crates.io loses some control over crate availability. however, the fact that crates.io would work as a caching proxy for the other registries, means crates.io could run whatever malware filters they want on the crates, as well as disallowing the mutation of published crates.

1 Like
13

How is this any different than existing possibility of phishing on existing crates, like serde vs. serdi, for a bad example. It seems like this doesn't make it better or worse, does it?

I really like this idea, but I'm a little biased. I think working through the concerns about domain transfers as they relate to ownership is an important detail. Not a blocker in my opinion, but definitely the biggest issue I see with this idea. In addition to that, I wonder about management in large organizations. Let's randomly pick google.com as an example. How do things work in the event that there are multiple groups inside the same large organization, but each independently owning their own set of crates? Would they need separate sub-domains?

1 Like
14

(post deleted by author)

1 Like
15

What's the plan for referring to namespaced crates in rust code? There was a lot of focus on this issue over on the packages as namespaces thread, which alternative proposals could mine for ideas. The one that I'm going to move forward for for that proposal is mapping / to ::>, which would probably work fine here as well. There does need to be some way to mangle the . that can show up in domains.

I guess one alternative would be to say "You can't depend on both ethanpailes.github.io/somecrate and soc.github.io/somecrate at the same time. Then you name somecrate in rust code with the word somecrate. I worry that such an approach erases some of the benefits of namespacing.

Avoid creating a hard dependency on external services. The recent GitHub service interruption is a good example why this is important.

It sounds like we would end up encouraging many users to use their github pages subdomain as a namespace. That wouldn't create a hard dependency in theory, but it might in practice. It's worth considering at least.

17

Then it doesn't address anything.

18

You're being somewhat counterproductive here, imo.

Regardless of whether they're tied to domain names, what I'd idealistically like to see is the addition of namespaces on top of what would become the 'default' namespace (all the packages that are on crates.io at the moment). This would only add options - you would be able to either publish a crate in the default namespace (exactly the same process as at the moment), or under a namespace tied to an established project that may have lots of crates within it. This proposal provides one option along these lines.

This addresses a lot of the concerns I've seen raised about squatting, including the legitimate case of squatting crates with a prefix such as the win32- crates

19

No surprises here, I said already I'm against this idea.

Really? How it will prevent spam in "default" namespace?

prefix is not a domain.

Companies change domains - will it affect projects who use crates from changed domain?

This idea if implemented, will just make system of crates more complicated, with more fragile points, that's all. It doesn't prevent spam, squatting or phishing (it even makes phishing easier).

1 Like
20

it’s supposed to help offload moderation from the crates.io mod team

21

(post deleted by author)

1 Like
22

(post deleted by author)