Namespacing on Crates.io

This indeed sounds like it would address the problem without placing undue burden on cargo.

1 Like

I think it stands to reason that a crate-collection like winapi would like “winapi” to appear as a prefix in the Rust code when referring to one of those crates.

We are moving away from users having to wrtie extern crate; the new way of doing things is just:

use mycrate::bar;

Yes; as @Nemo157 noted; but you still need a way to refer to the crates in paths.

This indeed adds more weight to the idea of simply reserving “pre-fixes” as the way of name-spacing:

  • tokio-* (prefix is tokio and once reserved, only the owner of that prefix can create more packages with that particular prefix)
  • serde-*

No new support in cargo or the compiler or the language required (it seems at first blush). Only thing needed is changes to crates.io:

  • How to reserve a prefix (first come, first serve)?
  • Limit number of prefixes a single user/org can reserve without intervention
  • Do not allow reserving a prefix that already has existing crates with that particular prefix unless the owner of said crate is the one asking for the reservation. If there are multiple crates with independent owners both eligible for the same prefix, then, the owner with the most creates starting with that prefix is granted the prefix. Existing crate that are not authored by the author are either grandfathered in, or (better), redirected to a new name and the old name becomes a reserved/dead name until the original owner surrenders it.

This probably requires more thought, but, something like that should work and would be backwards compatible as @kornel already pointed out.

Reserving prefixes seems to me a reasonable alternative and would give @retep998 the ability to get all the winapi- crates without having to make them all (perhaps moot since the crates already exist now…).

This doesn’t really require namespacing.

1 Like

The aggressive “ad hoc namespaces” would be to prevent publishing of the crate /([^-_]*)?[-_].*/ where the crate $1 exists.

It does probably make sense to limit namespace ownership to some value n per linked account, though. But I think it could (and maybe should) require ownership of the crate to claim that crate’s namespace.

1 Like

Keep in mind that not everybody wants to control their prefix. I explicitly do not want a crate being called diesel- to be restricted to projects maintained by Diesel. There are dozens of crates that other people have made called diesel-* or diesel_* to extend Diesel in various ways, and I would have no interest in preventing them from existing or using that name. Indeed, by not having an explicit namespace which implies control by a person or group, people are able to more easily find crates that are meant to extend or integrate with Diesel, without those projects having to be blessed by the Diesel project.

1 Like

Ideally, those would be called: somebodyelsesprefix-cratehelper-diesel. So, if I as the owner of the “cookiemonster-” prefix wanted to create a diesel helper that targeted the “countchocula” database, I’d name it:

  • cookiemonster-countchocula-diesel

Now, I know that’s isn’t how things have been, but, it would be a better place to be (IMHO).

That’s the thing. I don’t want to force that distinction. If Diesel would name a crate diesel-foo, and someone else wants to create a crate for that, I want them to be able to use the same name Diesel would use.

I’m not sure why this should work any differently than the rest of software. If I create a program that cleans the Windows registry, I’m allowed to call it “Windows Registry Cleaner”, regardless of whether or not it’s officially tied to Windows.

I don’t personally care whether some crate is an “official” add-on to a certain project. The stuff I care about with my third party libraries are things like documentation, a pit-of-success API, being actively maintained, etc, etc. None of that stuff is strongly correlated with official-ness.

The only situation where I could imagine caring is if I had to enter into a legal, contractual arrangement with all maintainers of code I consume. Or if I decided the right way to audit all my transitive unsafe code is to vet the maintainers rather than auditing code. But I dunno how either of those would ever end up being the case.

It’s just my opinion, so take it as that, but I would protest against namespacing without a slash or colon.

As @sgrif mentioned, if you create a framework and want to encourage others to write plugins or extensions, sandboxing diesel-* would make things counterintuitive, I’d have to name my crate naftulikay-diesel-extension, which doesn’t seem like an improvement on naftulikay/diesel-extension or diesel-extension.

I don’t have a perfect way forward, I don’t know that one exists, but hyphenating would seem to only add to the confusion. Any solution to this problem is going to be disruptive, but honestly I for one don’t mind go get github.com/naftulikay/gro or specifying that as a dependency. Since only the tail end of the repository is going to be used as the crate name, use gro::*; still works as expected.

If I had a vote, I’d vote for just biting the bullet and moving toward $VCS/$OWNER/$REPO. Leave the current crates as is, but start making new crates on Cargo register the full VCS link. This would prevent new squatting from happening and we’d be able to slowly weed out and get rid of the existing squatted crate names.

Just my two cents and entirely my opinion, so take it as that, an opinion :slight_smile:

There seems to be somewhat of informal name-spacing already happening using hyphens though. If you were to formalize it using the following rules, it might not be disruptive at all:

  • Any crate that starts with [A-Za-z][A-Za-z0-9]+-: the owner of that crate claims that prefix (say foo-bar, the the owner of “foo-bar” gets the “foo-” prefix assigned to them
  • If more than 1 crate owner has crates with the same prefix, disambiguate as following as: 1) owner of the most crates with that prefix gets that prefix, 2) other crates with that same prefix owned by other owners get the next compete using the same rules for the next sub-segment, 3) rounds continue until all crates starting with the original are either assigned an owned sub-prefix (up and including the final segment)
  • After this is done on the initial rollout, no one is able to create new crates using and already owned prefix
  • A new user needs to register one or more prefixes for their use before they can create new crates. All new crates must be created under one of their owned prefixes (either auto-assigned during the initial cut-over, or requested and assigned)

Now, this leaves some legacy crates in sub-namespaces of a larger namespace, but, there is nothing that can be done about that without breaking things backwards compatibly.

Another useful thing would be that an owner of a prefix can delegate/assign sub-prefixes to other owners if they choose to. So, the owner of the “cookiemonster-” prefix could delegate assign “cookiemonster-kiebler-” to another owner. This “sub-prefix” would work just like sub-prefixes previously assigned. Now, the owner of “cookiemonster-” can no longer create crates called “cookiemonster-kiebler-something” because on the owner of “cookiemonster-kiebler-” may create such crates.

This means that the owner of tic prevents anyone from publishing tic-tac, who then prevents anyone from publishing tic-tac-toe. Though the real complaint here is that you’re basically describing a hierarchical directory permission structure, but instead of using a path seperator, you’re using -, which people do not typically use for that purpose. Some people use - to replace _, for example. (And you still don’t stop anybody from using tictactoe.)

1 Like

I just don’t see a difference between using "-", "_", ":", "::", "/", "\" or whatever as a directory separator. I’m just saying to formalize what is largely an informal convention anyway (I think). If it isn’t really, then maybe this isn’t as desirable. I don’t think it is useful to get hung-up on what is and what isn’t a “directory separator” as there are lots of different conventions in different contexts.

And, no, the owner of “tic” would not claim the “tic-” prefix". The owner of “tic-tac” might claim the “tic-” prefix (if they’re the one with the most crates starting with “tic”. No existing single segment crates would become a prefix owned by anyone on their own. Those crates would persist in the global namespace and no further one-word (without a “-” in them) would be permitted to be registered going forward.

I don’t think it’s a good idea to enforce any rules that use names to express relationships between crates. Names should stay nominal, and human organizations should orchestrate any crate relations via documentation or other publications. If you’re handing out permissions over resources, you have to get humans to negotiate the facts regardless.

I also don’t think namespacing is really a good idea… I see the appeal – it makes it feel like everyone has more ‘space’ without stepping on each others’ names – but it actually only does that by forcing people to use longer crate names. If we all decided to use a minimum of 40 characters for our crate names, probably we would not have anybody squatting on them either.

2 Likes

I disagree. So, there can be only one “tic-tac-toe” crate, right? So every other implementation of “tic-tac-toe” on Crates.io is a second-class citizen right now. Under the new regime going forward it would be:

  • orgprefix1-tic-tac-toe
  • userfoo-tic-tac-toe
  • etc.

With no special “tic-tac-toe”. Now, this doesn’t fix the past, but, it improves the future in a backwards compatible fashion.

Yes, that is correct. But now, the owner of the “serde-” namespace or the “mysoftwarecompany-” namespace has some control of who can publish within that namespace and “Squatting” becomes not as useful or easy and there is some level of accountability introduced that self-organizes.

This argument was addressed in the initial policies of crates.io four years ago. I get that you disagree with the conclusions, but continuously relitigating what has already been discussed to death is exhausting for the team.

EDIT: I’m bad at discourse apparently. This was intended to specifically reply to the comment above mine, not the OP of the thread

1 Like

I don’t see the problem caused by a special tic-tac-toe or why rust-tic-tact-toe, std-tic-tac-toe, or anything of the sort would not also be special.

1 Like

So, in that case, we can just ignore this thread and close it down. It’s already been decided. No further discussion needed/wanted/allowed is what you are saying. Someone opened this thread, others are chiming in with opinions about it and possible backwards compatible ways forward. Saying, “we’re not doing name-spacing and that’s that” seems a little off-putting.

My apologies for offering an opinion in support of others with similar ideas/opinions.

If I’m to understand what you are saying correctly, then, any further discussion of this matter is moot and future threads on this should immediately be shutdown and debate ended. OK. I guess that could work.

Namespacing on crates.io is off the table permanently. Got it.

I still believe that it’s important to discuss all this. It has certainly been enlightening for me and I have found it productive, even if it doesn’t manifest in a real change.

The fork in the road here is that on an individual level, people will stop using crates.io and just hardcode in the Git repository URL into Cargo.toml. Invariably, forks will happen and crates will diverge, and this will end up in not working in crates.io, and people will go elsewhere.

There are a few avenues for the future:

  • change nothing: we could try to automate detection of crate squatting but still have to have manual user intervention to address violations
  • make a hard action to stop accepting “root-level” crates and sandbox under some scheme: this would stop squatting from continuing, if namespacing is done right. we could solve the existing squatted crates and not have this be a perpetual problem.

The discussion thus largely groups into one of these categories. Within the latter category, it’s a discussion on how to do namespacing, like based on VCS, based on some prefix, based on an org/user name in GH, based on a org/user name that crates keeps track of, etc.

I have found the discussion useful. I’m grateful to everyone who has participated so far.

Definitely. Nobody is trying to shut the discussion down. I’m just asking that folks try to keep the discussion above “I disagree on the conclusion that has been drawn in the past”, and try to bring new information to the table. There’s been a lot of discussion around this lately, and the team does try to keep on top of it.

Probably worth re-iterating that any actual change would need to be the result of an RFC. Also keep in mind that such an RFC would not only need to describe why this is a change we want, but also why it is important to prioritize this right now