Namespacing on Crates.io

[naftulikay]

I am moving this discussion here from this thread about a Crates.io incident.

Why did Cargo/Crates.io choose not to namespace crates by author/organization name?

Squatting for crate names would not be feasible if there was user/org namespacing. I’m not pointing fingers, pushing blame, or asking anybody to change, I’d just like to understand what the reason was for choosing not to namespace.

[kornel]

This has been discussed also in Crates.io squatting mega-thread.

There are cases where namespaces would help (e.g. tons of winapi crates), but there would also be squatting of namespaces.

There have been proposals to just use GitHub as the namespace, which doesn’t solve squatting, but it does make it GitHub’s problem.

[DanielKeep]

Not to mention here (from 2014 thru 2018) and here (from 2015 thru 2016).

[sgrif]

Why did Cargo/Crates.io choose not to namespace crates by author/organization name?

Very very short version: “I don’t want to have to learn your username to use your project”

[bascule]

Has there been any work on (pre-)RFCs or the like for adding an organization-like namespacing mechanism to crates.io in addition to the current global namespace, similar to what npm has done?

[DanielKeep]

Here is a visual summary on the numerous proposals to add a form, any form at all, of namespacing over the years.

[kornel]

But more seriously, even if we agreed namespacing is the way to go, the big problem would be in migration of existing crates. I estimate there are about 5000 useful crates, and there may be a long tail of crates whose owners aren’t active.

How do you move a non-namespaced name to a namespace? In a way that’s less of a headache to manage than squatting, and won’t encourage namespace squatting?

1. Don’t move, just allow namespaced crates alongside
   • Makes all existing non-namespaced crates special. If non-namespaced crates can be created after switchover, it doesn’t solve squatting. If they can’t, it creates a distinct set of old-school blessed crates.
2. Map non-namespaced crate to $crate/$crate or $crate/main, etc.
   • Establishes a nice pattern, so people will want vanity namespaces to follow the pattern and get shortcut names.
3. Move all crates to a single namespace old/$crate
   • Huge premanent split between old and new crates. Crates like libc are going to stay forever, so the special legacy namespace will be around forever.
4. Map to github-owner/crate
   • It creates a hard dependency on GitHub. Currently GitHub is critical for crates-io, but in theory it’s separate and completely removable.
   • A crate can have multiple owners, who can be added or removed at any time, so it’s unclear how to handle multiple-owner crates and what to do when owner in the namespace is removed from the crate.
5. Force everyone to choose a namespace and move their crates to $namespace/$crate
   • More org and account management to implement for crates.io
   • Still a problem what to do about all the crates that don’t have active maintainers

[bascule]

But more seriously, even if we agreed namespacing is the way to go, the big problem would be in migration of existing crates.

That isn’t what I was suggesting:

…adding an organization-like namespacing mechanism… in addition to the current global namespace

In other words, this isn’t an either/or dichotomy. npm has a global namespace of crates that works just like crates.io does today, and separately named @organization/package names for namespaced packages.

[eaglgenes101]

My idea is to leave the current crates.io namespace as the free-for-all it currently is, but separate crate names and crate idenifiers. Current crate names match to identifiers, and identifiers still must be distinct, but crate names can be changed, and are allowed to conflict. Identifiers are what gets put into cargo.toml, but names are strongly preferred for human-facing purposes such as displaying information.

(This post was originally replying to the replied-to poster, but has since turned into something of its own right. Sorry.)

[Soni]

Add a DNS record (or .well-known) for crate names.

if your website/DNS has a .well-known or w/e, you can use it instead of a crate name.

big projects would heavily benefit from this.

it’s sadly not as nice as true federation, but it still benefits big projects so I don’t care much.

[logannc]

Would anyone be interested in a curated registry? An expanded stdlib, of sorts? This inverts the problem in that squatting is not an issue and namespacing is less needed because the registry maintainers resolve conflicts.

I found a particularly nice domain name for it earlier today after the incident, but I could use some help setting something up.

[CAD97]

https://crates.rs/ does exist today, let’s not forget.

I don’t know what criteria @kornel used when setting it up, but it does today serve as an alternate frontend.

An alternate registry would be cool to have, but I suspect it unfortunately wouldn’t have much influence on name squatting on crates-io, as it’s still the default registry, and having a different name in different places would be unideal at best.

[Uther]

About option 5, You could keep the current crates in a read only “legacy/$crate” repository. But you would have to choose a namespace to publish a new version.

[Centril]

For those advocating namespacing on crates.io… how would namespaced crates map to crates as understood by Rust the language (specifically name resolution)? Would the namespace simply be dropped? Would the namespace and the crate name be concatenated with some modifier…?

[Nemo157]

The name on crates.io and the name Rust sees can already be independent, the latter is determined by the lib.name field in the generated Cargo.toml, with a default of package.name | tr - _ if not set. The rename-dependency Cargo feature will additionally allow overriding the name Rust sees for a specific dependency (useful if you have multiple packages that use the same lib.name like futures and futures-preview.

I guess there’s still the issue of what to do about the default, currently that will generate a non-identifier name if namespacing uses something like / or @ to separate namespace and internal name. Another alternative would do to do nothing and force the user to decide what they want via an explicit lib.name property.

[Centril]

The name on crates.io and the name Rust sees can already be independent

All true.

I guess there’s still the issue of what to do about the default

Yes; I think this is the interesting question here.

Another alternative would do to do nothing and force the user to decide what they want via an explicit lib.name property.

That doesn’t seem like great UX to me. Worse yet would be to force reverse dependencies to pick a name.

[Uther]

For backward compatibility, and usability too, I believe the namespace part should not appear in the source code. You will rarely have to use two crates with the same name and if you do, you can use lib.name to disambiguate.

[kornel]

I think that’s the “don’t move” option 1 I’ve listed.

[kornel]

It has been mentioned previously to use a domain name as the namespace. My initial reaction was that it’s just a long way of specifying git dependencies, but maybe there is something there after all.

Federated cargo

Cargo should support foo@example.org dependencies, in addition to the default foo@crates.io dependencies. Crates.io should allow crates with non-@crates.io dependencies. Y’know, so we all can have different opinions but still be on the same community.

The foo@registry syntax is intriguing. It’s terser than current alternative registries syntax, and feels easier to use than .cargo/config changes or extra registry = url key.

That could solve “what if someone squats some company’s trademark/project name(space)” problem, because the really official crates could be in form of crate@company.com. e.g. you could install servo@mozilla.org without having to check if servo or mozilla/servo is owned by mozilla or squatted.

I think publishing of a custom crate index could be made really easy. As easy as publishing a static website, e.g.: have a directory of .crate tarballs, run cargo magic there, and have it generate the index for them that you can upload anywhere, e.g. to github pages. There are actix.rs and tokio.rs domains already, and they could become registries & namespaces.

[Soni]

Not only that, the registry key would be reserved for using an alternative front-end registry.

crates.io, the default registry, would become a caching proxy for the other registries. so every time you make a crate using, say, servo@mozilla.org, rather than fetching directly from mozilla.org, it would fetch through the default registry.

Crates on the default registry wouldn’t need the @example.org part.

It’s also better to have active registries, that push to federating registries. Otherwise, if you had say 100 or 1000 registries, they’d be regularly hammering your registry for crate updates.

Do take a look at mastodon / ActivityPub. It’ll probably help to clear things up. (note that Activitypub Over Static Pages (AOSP) isn’t really a thing. also note that AP allows both the caching and the backing servers to block eachother, so for things like proprietary no-redistribute registries, they can simply block federation.)