[Pre-RFC] Domains as namespaces

[matthieum]

I admire the enthusiasm, but I have some reserve with the process with regard to long-term maintenance and security.

1. If the “owner” loses access to the domain, which happens relatively often, then what? Do they have to acquire a new domain, move the crate over to that domain, and ask all clients to migrate over (manually)? That’s a lot of churn!

2. There are several issues around security/authentication that I can of:

   • If I take over a domain, can I publish the same crate (name) with my own user?
   • If I have an official looking domain (rustaceans.cam, rustlang.cam), it really opens up phishing.

I do agree that some what of having “organizations” or “namespaces” would be great; but I am not clear that your proposal is significantly better than just letting users claim organizations and namespaces (like Maven does).

[mammothbane]

The core point is that crates.io would be a collection of facts along the lines of “at the point in time crate X was published, the author Y controlled domain Z”

I fail to see what the domain information actually adds to the picture. Another way to frame what you’re saying is that crates.io would maintain a time-varying mapping of domain aliases to crates.io accounts—you’d be using a (domain name, time) tuple to look up an account, then retrieving the crate with the given name.

Why not just use the crates.io account name for namespacing directly? That would eliminate that extra layer of indirection and vastly simplify the implementation. It still hits all of your requirements, as GitHub usernames are already namespaced, and doesn’t appreciably increase our dependency on GitHub, since crates.io uses GitHub OAuth already. A service interruption might affect the community’s capacity to publish new packages, but it wouldn’t affect the namespacing information we would already store.

[Soni]

domains as namespaces without domains as independent, federated registries isn’t a very good solution IMO.

I’ve said it before: federated means you can use crates.io as a caching proxy for other, official project registries.

by using .well-known, we would also allow the registry to exist on the main website, e.g. diesel.rs could setup a registry on diesel.rs so that you don’t have to use a subdomain when specifying the crates.

the main benefit is that you get official registries, rather than using rust’s generic/general purpose registry. as a side effect, you also get independent moderation, and the crates.io moderation team would no longer be overwhelmed with moderation tasks. in the end, both of these contribute to a better organized registry “fediverse”, and IMO it would be more efficient than simply “domains as namespaces”.

the main risks (which IMO are benefits) is that crates.io loses some control over crate availability. however, the fact that crates.io would work as a caching proxy for the other registries, means crates.io could run whatever malware filters they want on the crates, as well as disallowing the mutation of published crates.

[bluejekyll]

If I have an official looking domain (rustaceans.cam, rustlang.cam), it really opens up phishing.

How is this any different than existing possibility of phishing on existing crates, like serde vs. serdi, for a bad example. It seems like this doesn’t make it better or worse, does it?

I really like this idea, but I’m a little biased. I think working through the concerns about domain transfers as they relate to ownership is an important detail. Not a blocker in my opinion, but definitely the biggest issue I see with this idea. In addition to that, I wonder about management in large organizations. Let’s randomly pick google.com as an example. How do things work in the event that there are multiple groups inside the same large organization, but each independently owning their own set of crates? Would they need separate sub-domains?

[soc]

Right now I don’t own any domain - I don’t deserve to publish a crate?

This proposal gives you more options to publish your crate, not fewer.

I intentionally used a sub-domain in the example above to demonstrate publishing without owning the whole domain.

Given the fact that you can post messages here, it’s highly likely that you already control such a sub-domain suitable for publishing your crate.

You also can keep publishing your crate to the global namespace.

If domain of my project is “nudebeachgirls.fun”, will users use my crates for cryptography?

That’s a decision made by your potential users.

If domain is expired I can’t update crates anymore - it’s just ridiculous.

What’s ridiculous about that? If your domain expires, you also can’t serve your web pages on it anymore, or receive your mail sent to that domain.

If I buy domain “rustlang” in some exotic tld, will it increase my chances for phishing activity?

There is nothing stopping crates.io maintainers from banning malicious domains. It’s even easier than before, because registering a new domain is more work for attackers than creating a new user name somewhere.

[ethanpailes]

What’s the plan for referring to namespaced crates in rust code? There was a lot of focus on this issue over on the packages as namespaces thread, which alternative proposals could mine for ideas. The one that I’m going to move forward for for that proposal is mapping / to ::>, which would probably work fine here as well. There does need to be some way to mangle the . that can show up in domains.

I guess one alternative would be to say "You can’t depend on both ethanpailes.github.io/somecrate and soc.github.io/somecrate at the same time. Then you name somecrate in rust code with the word somecrate. I worry that such an approach erases some of the benefits of namespacing.

Avoid creating a hard dependency on external services. The recent GitHub service interruption is a good example why this is important.

It sounds like we would end up encouraging many users to use their github pages subdomain as a namespace. That wouldn’t create a hard dependency in theory, but it might in practice. It’s worth considering at least.

[OZ_]

This addresses the squatting issue.

This proposal gives you more options to publish your crate, not fewer.

Then it doesn’t address anything.

[IsaacWoods]

Then it doesn’t address anything.

You’re being somewhat counterproductive here, imo.

Regardless of whether they’re tied to domain names, what I’d idealistically like to see is the addition of namespaces on top of what would become the ‘default’ namespace (all the packages that are on crates.io at the moment). This would only add options - you would be able to either publish a crate in the default namespace (exactly the same process as at the moment), or under a namespace tied to an established project that may have lots of crates within it. This proposal provides one option along these lines.

This addresses a lot of the concerns I’ve seen raised about squatting, including the legitimate case of squatting crates with a prefix such as the win32- crates

[OZ_]

You’re being somewhat counterproductive here, imo.

No surprises here, I said already I’m against this idea.

This addresses a lot of the concerns I’ve seen raised about squatting

Really? How it will prevent spam in “default” namespace?

including the legitimate case of squatting crates with a prefix such as the win32- crates

prefix is not a domain.

Companies change domains - will it affect projects who use crates from changed domain?

This idea if implemented, will just make system of crates more complicated, with more fragile points, that’s all. It doesn’t prevent spam, squatting or phishing (it even makes phishing easier).

[Soni]

it’s supposed to help offload moderation from the crates.io mod team

[soc]

If the “owner” loses access to the domain, which happens relatively often, then what? Do they have to acquire a new domain, move the crate over to that domain, and ask all clients to migrate over (manually)? That’s a lot of churn!

Then the past owner won’t be able to publish new crates anymore. It’s not different from a user losing access to their GitHub account, email account or crates.io account etc. Does it suck? Yes! Is it expected that this happens? Yes! Does this approach deal with it gracefully? Yes.

Migration can be accomplished by either

• supporting a forward_to = "new.domain.org" attribute in Cargo.toml. This would also enable to creation of “awesome crates” collection crates, which only consist of forwarders to other crates.
• asking crates.io maintainers to create such a forwarder. (This is expected to be exceedingly rare. See the approach Maven took.)

There are several issues around security/authentication that I can of:

• If I take over a domain, can I publish the same crate (name) with my own user?
• If I have an official looking domain (rustaceans.cam, rustlang.cam), it really opens up phishing.

That’s a policy question. There are multiple approaches with various degrees of automatism I can think of, let’s pick the one people feel most comfortable.

I do agree that some what of having “organizations” or “namespaces” would be great; but I am not clear that your proposal is significantly better than just letting users claim organizations and namespaces (like Maven does).

This proposal is significantly better than just letting users claim organizations and namespaces (like Maven does) because

• Maven does not allow any “choose-your-own” organizations or namespaces anymore as far as I know. There are some legacy artifacts left, but no new ones can be created (or at least heavily discouraged). Even old Apache Commons artifacts like commons-lang are slowly being migrated to domain-namespacing. Do you have contradicting information?
• “Claiming” in Maven space usually works by:
  • manually creating a JIRA account on Sonatype
  • writing a ticket, requesting a new project
  • having some person manually review your request
  • etc.: https://central.sonatype.org/pages/ossrh-guide.html

I proposed this approach, because I’m aware of how horribly Maven is when it comes to publishing artifacts. Using domains as namespaces (as Maven does) works exceedingly well, but many other parts of Maven are really really terrible mistakes that shouldn’t be repeated.

This triggers creation of your repositories. Normally, the process takes less than 2 business days.

and

Please do not deploy until after you have received an e-mail notice indicating that the ticket is Resolved . One of the most common problems related to new projects is premature deployment, which misroutes your artifacts to a catch-all repository.

[soc]

What’s the plan for referring to namespaced crates in rust code?

There isn’t any. This is intentional, because doing this would be a terrible idea.

Allowing people to refer to namespaces from within the source code would make it impossible to switch out unmaintained crates with maintained ones (who have been forked and live under a different organization) without having to touch every single source file.

This proposal makes sure that namespaces and crate names are decoupled, this is a core property of this design.

I guess one alternative would be to say "You can’t depend on both ethanpailes.github.io/somecrate and soc.github.io/somecrate at the same time.

Users can rename crates they depend on in your Cargo.toml.

It sounds like we would end up encouraging many users to use their github pages subdomain as a namespace. That wouldn’t create a hard dependency in theory, but it might in practice. It’s worth considering at least.

Absolutely. Though I fairly certain that most established projects will prefer using their existing domains and many people who might want something “nicer” than “xyz.github.io” will do the same.

My main motivation here is that this approach is more open than e. g. using your GitHub name/org as a namespace as it has been suggested in the past.

[soc]

Really? How it will prevent spam in “default” namespace?

See the comment regarding the scratch namespace.

Companies change domains - will it affect projects who use crates from changed domain?

This has been addressed multiple times already, please have a look at the other replies.

This idea if implemented, will just make system of crates more complicated, with more fragile points, that’s all. It doesn’t prevent spam, squatting or phishing (it even makes phishing easier).

It addresses all three points. I encourage you to read my explanations and please feel free to ask questions if you don’t understand something!

[pythoneer]

It was addressed several times but i haven’t seen a convincing answer yet. One problem – and i believe its currently the major one – is “what if i loose access to the domain?” and the general answer was “its as bad as if you loose your github account”. But i don’t believe that, it is worse and a possible security concern. If i loose my github account, i can’t publish any new version of the crate – that’s unfortunate but fine.

Lets imagine i have a domain “usefullstuffforrust.com” and i publish my important “is-even”, “is-odd” … crates here. Now nearly everyone uses my useful crates and they are among the most popular ones but they don’t need much maintenance because the crates are pretty much “done”. So over the years i lost interest in maintaining them but they don’t loose importance to the Rust ecosystem – i stop using and paying for the “usefullstuffforrust.com” so i loose the domain and its open for anybody to pick it up and publish new version of my crates? With added “features” like cryptominers and keyloggers?

[mbrubeck]

Moderator note: Everyone, chill out. If you want to participate on this forum, you must do it without sniping at other users.

[ethanpailes]

There isn’t any. This is intentional, because doing this would be a terrible idea.

Allowing people to refer to namespaces from within the source code would make it impossible to switch out unmaintained crates with maintained ones (who have been forked and live under a different organization) without having to touch every single source file.

That’s an atypical understanding of namespaces. Not that it is necessarily bad. I think the ability to swap out unmaintained packages for maintained packages has a certain appeal, but what you are proposing does run counter to what people will be used to. In particular it is not what the other main domain-based namespacing system does (the JVM ecosystem). It is also not how npm scopes work.

Including namespaces in how you refer to packages in code is the standard way to do things. Calling it a terrible idea out of hand is unjustifiably dismissive. The position that conflicts won’t happen much and when they do you can just use crate renaming is an interesting one. It has the benefit of keeping the language the same. I really like the idea of keeping the changes confined to the package manager.

[Soni]

If you go federated, you can sign packages and cargo can verify the signatures. The server signature would be stored somewhere like .well-known, and cached locally both in crates.io and in Cargo.lock (or maybe a Cargo.sig – we don’t wanna leave out library packages!). This means if a domain changes ownership, this won’t lead to a potential security issue.

Ofc, it provides little to no protection against that domain/server being hacked, and servers are generally easier to hack than a crate developer’s computer. This is one point where a centralized registry is beneficial - if you have many registries, it’s likely that at least some of them will have security vulnerabilities. If you have a centralized registry, it’s a lot easier to patch those vulns.

In any case, federation wouldn’t necessarily make the attack surface any smaller - crates.io would still be a single point of failure, as it’d be the default registry. If the ability to use alternative registries takes off, tho, it could lead to a smaller attack surface, as different registries could be using different software with different vulnerabilities or even a strong layer of hardening. Some registries could even be served over static pages + a cron job that checks for updates, making hacking them quite a task!

[Ixrec]

Maybe this is a tangent, but this term “federated” has shown up enough times that I want to stop and ask what that word means.

The only definition I was able to find via googling was Julia’s:

Federated package ecosystem. This means that multiple independent parties can maintain both public and private packages and registries of them, and that projects can depend on a mix of public and private packages from different registries.

which sounds like exactly what we’re going to get from cargo supporting alternative registries, so whatever @Soni means by “federated” must be more than just that. Right?

[Soni]

it means the ability to publish crates depending on example.org’s whatever on crates.io, and having crates.io fetch the crates from example.org and cache them in crates.io, and having cargo fetch example.org’s packages from crates.io, so it’s NOT what the “alternative registries” RFC proposes.

the “alternative registries” RFC proposes: registries are independent and fully isolated, so they can’t ever talk to eachother. dependencies from different registries aren’t allowed to be mixed together.

it’s completely different. both involve independent parties and independent moderation, but, for benefits of federation: (non-federation lacks these, so they can be considered drawbacks for non-federation)

• reliability, as if a registry goes down you can just switch to another and everything’s cached
• deduplication, rather than publishing (separately) on multiple registries, you only publish once and it gets automatically distributed
• namespacing (as talked about in this post)
• there are more I can’t currently think of, but I encourage ppl to talk about them as well (:

otoh, federation has the following drawbacks compared to non-federation:

• wider attack surface, as you can hack another registry and the hacked contents will get automatically distributed as with a non-malicious crate. (malware scans help mitigate this problem - remember: always sanitize your inputs, even if they come from a federated server ;))
• um, there are probably more that I can’t think of? (“change of domain ownership” etc aren’t a problem if the server signature (public key) also changes, so I’m not sure if it should be counted here. but do note that the signature is independent of the domain name!)

[soc]

I think the ability to swap out unmaintained packages for maintained packages has a certain appeal, but what you are proposing does run counter to what people will be used to. In particular it is not what the other main domain-based namespacing system does (the JVM ecosystem).

Maybe we are talking about different things, because this is in fact exactly what Maven does.

One arbitrary example: jaxb-core was previously published under the following coordinates:

<dependency>
    <groupId>com.sun.xml.bind</groupId>
    <artifactId>jaxb-core</artifactId>
    <version>2.3.0.1</version>
</dependency>

Since a while this package has been migrated to:

<dependency>
    <groupId>org.glassfish.jaxb</groupId>
    <artifactId>jaxb-core</artifactId>
    <version>2.3.0.1</version>
</dependency>

For users of this artifact no changes except updating the groupId have been necessary.

This is what I’m proposing. Fortunately, Rust doesn’t even have the rather arbitrary (and unenforced) package naming convention and I’m not suggesting that Rust should adopt it. (That design smells a bit like the stuff suggested in [Pre-RFC]: Packages as Namespaces though, which I think is a poor idea for the obvious reasons.)