Pre-RFC: User namespaces on crates.io

Web Presence
1

crates.io currently places all registered packages in a single flat namespace. This simplifies some aspects of package discovery, but adds friction for users who prefer to name packages after their purpose rather than use an opaque codename. Additionally, due to the large number of registered packages, it is often difficult to find a name that is short enough to be ergonomic when referenced as a symbol.

The typical solution to this problem is package namespaces, and the topic has been covered before (links copied from HN discussion):

These proposals have focused on one type of namespace, the "organizational namespace", which would allow "@example/foobar" to be a crate managed by Example Inc. Then they struggle to make progress on difficult non-technical aspects such as allocation of names.

I propose adding a more limited form of namespacing, for user accounts only:Jane Doe, who created her account by signing in with the GitHub account jdoe, will be able to publish crates with names prefixed by "~jdoe/". This is in addition to the existing non-namespaced crates policy.

The tilde prevents conflicts with existing crate names, is separate from any future proposals for organization namespaces, and has that charming mid-90s Geocities aesthetic.

This syntax will also be recognized by Cargo, which will drop the prefix when assigning a package to a crate name. In other words, these two [dependencies] sections would be equivalent:

[dependencies]
foo = { version = "0.1", package = "~jdoe/foo" }

[dependencies]
"~jdoe/foo" = "0.1"

This makes the use of user-namespaced crates more ergonomic -- the dependent crate does not need to manually rename.

12 Likes
2

This is not valid TOML, you'd have to use something like "~jdoe/foo" = "0.1"

1 Like
3

That's unfortunate, but done.

4

Do we already have more packages than npm? They survive somehow. I don't see why developer can't add "jdoe-" prefix to achieve what you propose.

3 Likes
5

NPM has namespaces, which they call "scopes". Many large projects use scopes to avoid name conflicts and restrict publish permissions.

4 Likes
6

  • GitHub names are not permanent, but crates.io crate names are permanent. It's possible to rename username of a GitHub account, and there are users in the crates ecosystem who have done that. How do you propose to solve that without running into issues such as deadnaming?

  • How does namespacing by user work with crates with multiple owners, and changes in ownership? What if the original user is removed from the owners?

  • crates.io tries not to have any hard dependencies on GitHub. It currently favors GitHub, but that's only due to lack of resources to implement other authentication schemes. How would namespacing work for GitLab users? Or if crates.io added something like a general OpenId account support or its own accounts?

28 Likes
7

Somehow, I also feel that people are going to be hit with shell-escaping problems. I know this is primarily used in Cargo.toml and Cargo.lock, but:

cargo install ~user/project

Will most likely not do what you expect it to.

11 Likes
8

For the moment I'm going to avoid talking about the specifics of how crates.io manages namespaces and concentrate on the implementation. After all, if Rust supports namespaces then it allows alternative registries to experiment with different approaches.

In your proposal rustc avoids supporting namespaces itself. Any conflict with library names can be resolved manually by aliasing. The downside of being namespace unaware is it may be less obvious from the code which crate is being used. For example, today if you see tokio you generally know what that's referring to whereas the difference between rachel/async and jacob/async will not be so obvious when it appears only as async in the code. But this may be an acceptable trade-off. After all even today and it's possible for crates to be renamed or loaded from sources other than crates.io and even different versions of the same crate can potentially be very different from each other.

Cargo would need some awareness of namespaces. It would need to support:

  • registries with and without namespaces
  • a global namespace for compatibility with un-namespaced crates
  • some way to specify the namespace in cargo.toml

Your solution is to use the ~ character to disambiguate a namespaced dependency from an un-namespaced one. To be honest I'm not the biggest fan of magic characters in configuration strings (whatever that character is). Though I admit I can't think of a better solution off the top of my head. We could add a namespace key but that may be considered too verbose as it would preclude the short form way of specifying a dependency.

9

One solution for this which has been proposed by several people, which is quite different from the solution proposed in this pre-RFC, is adding namespacing purely on the crates.io side (or an alternative registry's side) in a way that doesn't require any changes to the Rust module system or even its knowledge/awareness of namespaces (i.e. a registry-side-only change).

This could be accomplished by letting someone claim the foobarbaz-* / foobarbaz_*-prefixed namespace if and only if there are no published crates with foobarbaz-* / foobarbaz_* prefixes already and the owner requesting the namespace controls the foobarbaz crate.

This would be a codification of what people already do in order to have a pseudo-namespace for their crates.

10 Likes
10

Interesting. Are there any experimental repositories that explore this idea?

11

While this would be a partial solution, doing it opt-in is not enough.

One of the reasons to have namespacing is allow users to trust a crate based on which user/organization owns it. Perhaps I would have more trust in a crate under tokio/ or serde/ than just a random crate with tokio or serde in its name. If the namespacing is opt-in, just looking at serde-json, I wouldn't know whether it is owned by serde-rs.

1 Like
12

What if it were clearly marked in the https://crates.io web UI? (and potentially, in the cargo CLI via e.g. bold, something like rewriting the crate name into something like @serde_json, different colors)

13

If it's just about ownership, nothing stops tools from checking the owners!

For example:

cargo add @dtolnay/serde_json

could use the crates-io API to check if the serde_json crate is owned by dtolnay.

This of course isn't namespacing the crate names, but OTOH it naturally supports crates with multiple owners, and changes in ownership.

3 Likes
14

I like the proposal for reserving prefixes, like tokio-*. This could be reinforced by crates.io website displaying crates a with prefix as explicitly belonging to the prefix owner:

tokio-foo (owned by tokio-*)

Maybe there could be a microsyntax for this, i.e. if a crate belongs to tokio-* prefix, then tokio-foo can be written as tokio/foo (but continues to map to tokio_foo symbol in Rust source).

Combining idea of prefixing crates with a user name, and idea of crates-io helping namespace things:

If you name your crate with a prefix of $githubusername-, then crates.io could recognize this and automatically change the TOML snippet displayed on the crate page to:

[dependencies]
foo = { package = "jdoe-foo", version = "0" }

This could be combined with crates.io automatically reserving jdoe-* prefix for user who logs in with jdoe username (it will move crate name squatting to username squatting, but it will work as namespacing while keeping backwards compatibility).

8 Likes
15

That could work, esp if there is also some indicator in the Cargo.toml file that it is a namespace.

It still leaves the problem of what to do with crates like serde-*. They are the kind of crates which will benefit a lot, but can't use namespaces (without renaming), because there are already unrelated crates with that prefix.

16

Seems tricky. It'd probably be good to start with a minimal implementation that can work for new projects, and then maybe circling back on how to retrofit it for old ones.

2 Likes
17

I've assumed prefix reservation would be checked only when a new crate is created (published for the first time). This means that reserving serde-* now wouldn't affect any crates, but would prevent new serde-something from being published by unauthorized users.

The check-on-create-only rule also allows owners of prefixes to act as a registrar. They can create new crates, and then transfer crate ownership to others to manage them.

2 Likes
18

So your proposal is to add optional namespaces (like in npm), not required? Then it's a good idea.

1 Like
20

That's correct. The new namespaces would be optional, and users could continue publishing non-namespaced packages per current policy.

3 Likes
21

I'm not a fan of magic characters in configuration files (~ meaning user, or @ meaning org), especially with TOML's limitations. However, I really like the idea being able to reserve a prefix foo-*. This would prevent someone not associated with the tokio project using an official looking crate tokio-router for example. This doesn't do much with the current name squatting, but I think it's a step in the right direction that addresses other issues only tangentially related to namespacing. This is all with the caveat of things like the ability to transfer/share ownership of prefixes, etc.

7 Likes