[Pre-RFC]: Packages as Namespaces

This is my personal opinion. It should not be taken as a statement from the crates.io team, nor does it imply that anybody else on the team shares my opinion.

In general, I like this technique, although I think we need more user research to make sure this is solving the problems we hope it’s solving. Namely, as said other places in this thread, this really only addresses the “shared branding”/officialness problem. Is this a problem worth solving? Is this the best solution to that problem?

In addition to the questions already raised about sigils/naming/etc, I’d like to see more details around permissions made explicit in an eventual RFC along these lines: it seems like whoever is designated as an owner (either directly or via team ownership) of crate “foo” should be able to create new crates “foo/whatever”. Then should crates “foo/whatever” automatically get the same owners of crate “foo”? That seems like the common case and would save folks time having to remember to add the other owners, but are there scenarios where that wouldn’t be desired? If, after creation, the owners of either the crate “foo” or the crate “foo/whatever” are changed, should those changes be synced to the other crates in the group? Only if the top level crate’s owners are changed? What would folks expect?

EDIT: Also, thank you @ethanpailes for taking the time to write this up! :heart:

6 Likes

I like this style of proposal since it gives two desirable properties at once: affiliation and namespace security. I wonder if symbol mangling could be done in a backwards compatible manner by using double underscore as a mangled separator: the crate foo/bar becomes extrn crate foo__bar;. Icky, but if nobody has a crate name with double underscores already it could be done without changes to the ecosystem.

With regard to Carol’s questions of namespace author permissions, the obvious answers are either “make comprehensive security permissions that can be twiddled to do whatever a user might want”, or “crate namespace groups should trust the members of that group and do their own governance”. I like the latter solution.

Something else that I don’t think has been brought up yet: at the risk of making a hairy issue more complicated, is there any reason the level of nesting should stop at one?

@carols10cents, the note about permissions is a really good one. I had definitely planned to flesh the permissions model out more in the full proposal. I think @icefoxen’s point about basically trusting the maintainers of a top level package to manage their own community is a good one. Selecting good defaults is probably the key thing for this RFC.

With respect to user research, I think that looking at why npm adopted scopes can provide a lot of value. They have already done some of the legwork about this sort of thing and talked about it in the open, so that’s great. I started this discussion because of the flood of chatter about this stuff on /r/rust and other boards, and I think those discussions can also be mined.

I promise I’ll write up the actual RFC real soon now.

If we did have multi-level nesting, I wonder if it could start one level up with the registry. In order words: @foo/bar would be short for @cratesio/foo/bar and registries would just be namespaces that happened to have their own URL.

Sourcegraph search of the index for __

repo:^github\.com/rust-lang/crates\.io-index$ __

One crate, teardown_tree___treap, which is

Do not use - intended for internal use of teardown_tree crate

Which seems like it would enjoy being teardown_tree/treap instead?

This is the singular occurrence of a double underscore in the index, and is indeed a triple underscore.

(This post is merely meant to be informative and not issue support one way or another)

3 Likes

Awesome datapoint!

An interesting use of namespaces would be to relax orphan rules. This RFC states that nobody can write diesel-chrono crate that integrates the two crates that would like to stay independent of each other.

However, if there was diesel:bike:chrono namespaced crate, then it would be known that all the diesel:bike:* crates are from the same author/owner/org, so they would have power to enforce their own orphan rules across their own crates.

This way the diesel crate could remain abstract and not depend on chrono, and the diesel:bike:chrono could add chrono support by being privileged to implement traits that currently only diesel can.

1 Like

As an FYI, crates.io team member here, I worked for npm for 3 years. I have been and intend to continue sharing the best practices I learned from that team. If you have any questions, feel free to direct them to me.

As for why name spaces were originally introduced, it was to help create a paid product, private packages. Very few people used them, until around 2 years later when we introduced another paid product, organizations. Free and paid organizations are the primary use case that folks use and it’s largely as a result of the increased and more ergonomic permissions controls. crates.io already affords this by letting folks leverage GitHub teams. So strictly from a npm point of view, crates.io does not currently have a motivation for this feature as we don’t need to make money and already give folks the ability to organize permissions via GitHub organizations.

3 Likes

Background: crates.io team member, npm engineer 3 years

I’d like to share that package disputes on npm are handled by a rather large team of paid support engineers. Disputes are by far the largest time burden they have. They are handled manually. I believe, through my discussions with Bundler/RubyGem maintainers that they do not have this policy because they cannot support a support team large enough to thoughtfully handle these requests.

At the moment, the crates.io team also does not have this set of support engineers. Until the team is able to grow (I have advertised joining the team multiple times in the past few weeks with literally zero people taking me up on the offer), I cannot imagine how we would be able to take on the workload of supporting this.

7 Likes

That’s a cool point about orphan rules. I think I’m not going to address it in the proposal I put forward, but if optional package based namespaces are adopted it would be a cool follow up.

@ag_dubs, that sounds like a point in favor of this idea. Hopefully optional namespaces would reduce the number of disputes that could come up and take some of the burden off the crartes.io team.

Until the team is able to grow (I have advertised joining the team multiple times in the past few weeks with literally zero people taking me up on the offer), I cannot imagine how we would be able to take on the workload of supporting this.

I’m a little confused about what you mean my this. Are you talking about the code changes required to allow crates.io to understand namespaces? I’m not proposing any sort of dispute resolution mechanism, so this proposal would not impose any sort of mandate on anyone to start playing judge.

I suggested a more aggressive name spacing using epochs here:

Best namespaces proposal so far. One addition: would be great to let owner of the “parent” namespace to add collaborators who also have right to publish/edit their crates.

Does anyone have thoughts on what to do about name mangling? I’ve been trying to figure out how the module path separator is mangled currently, but I didn’t find any quick answer.

I’m sorry for letting this thread die off. In the process of writing this idea up into an actual RFC, I became more and more concerned that this was going to add too much complexity to the language, and especially the naming system. Recent posts about rust language design being mostly finished except for a few big things that are still in the pipeline have further convinced me that making any changes to rust-the-language would be a bad idea.

I still think that packages-as-namespaces is the best approach to namespaces, but I just don’t think namespaces are needed enough that it is worth spending the complexity budget in the language itself. Packages as namespaces might still be a good idea, but it would have to be handled completely within crates.io.

Apologies for this aside, but which posts are you referring to here? This is the first time I read this, and I try to keep up with all things “high level language design decisions / directions” out of a personal interest.

Many of the recent blog posts about people’s wishes for rust 2019 have focused on the need to focus on stabilizing and working on holding the ground that has been taken. I think there is a general sentiment that too much churn is not good, and once async/await, const generics and a few other things land the focus should be more on improving the ecosystem. There might come a time when it becomes a good idea to switch gears back to making big language changes, but for now it would be nice to slow down.

Obviously I’m in no position of authority, but that’s been my read of the chatter in the past month or so.

https://readrust.net/rust-2019/ is a pretty good list of all the known “Rust 2019” posts so far. @ethanpailes is definitely correct about there being a strong emerging consensus that “we” don’t want as many big language features in the near future.

https://www.jonathanturner.org/2018/12/the-fallow-year.html seems to be the most quoted, linked, and imo persuasive of the posts explicitly making this argument, so if you only read one Rust 2019 post I’d go with that one.

Ah thanks. Yes, I did read those, and I agree with all of them, I just never took it as “we’re done”, but as “we need to slow down and polish what’s there, before taking the next big steps”.

Either way, I get where you are coming from now. Sorry for the aside.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.