Pre-RFC: Packages as Optional Namespaces

[withoutboats]

I think this proposal is quite good, including making no change to rustc. The choice to convert to foo_bar makes sense; I'd point out that ambiguity is unlikely, though not guaranteed not to occur, except by the explicit choice of the organization authors (which would probably only be projects converting from the current ad hoc foo-bar system to the new system).

The only thing I really dislike about it is having to have non-bare strings as dependency names! I thought the . proposal was a rather interesting for its interactions with TOML. I wonder if we could enumerate the drawbacks of using . as the separator in TOML but converting to underscore in rustc.

[passcod]

If using . in toml would this be a valid way to use multiple packages from a namespace?

[dependencies]
serde = {
  json = "1.2.3",
  derive = "1.0.55",
}

or the also equivalent

[dependencies.serde]
json = "..."
derive = "..."

I could see either of these being very confusing to novices, even without bringing version, features, etc conflicts into it.

[Manishearth]

@withoutboats

I think this proposal is quite good, including making no change to rustc. The choice to convert to foo_bar makes sense; I'd point out that ambiguity is unlikely, though not guaranteed not to occur, except by the explicit choice of the organization authors (which would probably only be projects converting from the current ad hoc foo-bar system to the new system).

Thanks!

Yeah, the ambiguity is only really a problem for typosquatting, e.g. squatting icu-num and tricking people into using it over the Extremely Trusted icu/num library.

@withoutboats

The only thing I really dislike about it is having to have non-bare strings as dependency names! I thought the . proposal was a rather interesting for its interactions with TOML. I wonder if we could enumerate the drawbacks of using . as the separator in TOML but converting to underscore in rustc.

Yeah i dislike the quoting too.

I'm definitely interested in this, but my understanding is (please correct me if wrong) that . in toml only works with [], i.e. you HAVE to do the following:

[dependencies.foo.bar]
version = ..

# or
[dependencies]
foo = {bar = "1.0"}

and cannot do

[dependencies]
...
foo.bar = ...

I don't really like that that's the case, it feels weird to be forced to put some dependencies in a pretty different format. Though I suppose we could make "foo.bar" = "1.0" work?

The other drawbacks in my eyes are:

• It feels more likely to me that people will expect foo.bar to work in rust code (this is not a major concern)
• This does block git, version, branch, features from ever being a subcrate name (no biggie)

I am not opposed to this solution, however. I can add a section to the pre-rfc about this, but I would also like to see some more discussion of the tradeoffs here!

[withoutboats]

@Manishearth

• This does block git , version , branch , features from ever being a subcrate name (no biggie)

Actually, it seems like more of a concern to me that this would block us from ever adding new fields to the dependency object in the format. That sounds like a non-starter.

[Manishearth]

Oh, derp. I knew there was a reason I had rejected this approach before.

[samsieber]

So I was thinking the namespace root is a real crate, but then the crate you depend on is actually a facade that reexports all the namespace root's public items as well as any enabled children crates as mods.

So if serde was a root and you were using serde and serde/derive too then you'd have these in your dependency chain:

• serde (synthetic)
• serde
• serde/derive

Where the lib.rs for the synthetic serde looks like this, (not sure about the name mangling):

pub use serde::*;
pub use serde_derive as derive;

But again, it's very odd solution. But it would remove any ambiguity from the parsing of use statements - they would be just normal use statements with the backing structure being a generated facade.

[drewkett]

Not sure I saw this anywhere else but a random idea for the namespace is to require the namespace to start and end with - or _ (ie -tokio-core where tokio is the namespace). The leading underscore is not ideal in code use given its current meaning in code, but it is currently valid for modules to use. I would think it’s minimally invasive since I’m assuming there’s very few (no?) crates that start with - or _. And as long as there’s no conflict with existing crates, it wouldn’t be ambiguous.

[CAD97]

I just want to explicitly note that package::subpackage is exactly the behavior that unic (I assume icu4x) and basically any super-modular crate family wants. Especially given that the parent crate necessarily owns the subcrate, I'd be perfectly happy if even the parent crate were required to not define a module at that path.

The main downside to that approach that I see is that it would require changes to rustc as well as crates-io and cargo.

To be honest, were unic still in active development, I would've petitioned to swap it from being many subcrates to just being a stupidly granular feature matrix, especially since we version all the crates together.

If the components are more "plug in modules" versioned independently, it's a bit more interesting of a decision, but it still seems to fit most uses of subcrates.

[Manishearth]

@CAD97

I just want to explicitly note that package::subpackage is exactly the behavior that unic (I assume icu4x) and basically any super-modular crate family wants. Especially given that the parent crate necessarily owns the subcrate, I'd be perfectly happy if even the parent crate were required to not define a module at that path.

Yeah, though I'll note that not all uses of this are for super-modular crate families. And icu4x will probably want to be more than just reexports at the top, but we can still make that work.

@CAD97

To be honest, were unic still in active development, I would've petitioned to swap it from being many subcrates to just being a stupidly granular feature matrix, especially since we version all the crates together.

There's actually a separate feature some folks are talking about, "workspaces as packages", which could potentially get us that. Still early days.

[jmillikin]

Given that the package name foo/bar would be translated to the crate name foo_bar, this proposal doesn't seem like it's related to namespaces at all. Would it be better to name it something like "reserved package prefixes"?

I think it would be good to keep the term "namespace" available for proposals that are designed to enable multiple packages to register the same crate name, similar to how mod { } or C++'s namespace { } allow multiple public functions to have the same name but different internal identifiers.

This may also help to disambiguate discussions designed to restrict which package names can be used (e.g. for authentication or reputation management) vs those that try to increase the set of allowed package names.

[Manishearth]

@jmillikin

Given that the package name foo/bar would be translated to the crate name foo_bar , this proposal doesn't seem like it's related to namespaces at all. Would it be better to name it something like "reserved package prefixes"?

No, this is being translated to the crate name foo/bar as well (and will be foo/bar in Cargo.toml), just that foo/bar maps to foo_bar before being handed to rustc. We do something similar for foo-bar already.

This does not prevent a package foo-bar or foo_bar from being registered. This strictly increases the set of allowed package names.

[ratmice]

I think one way to get rid of this ambiguity in features is by adding a prefix, like @foo/std, that would seem to allow you to specify @namespace/crate/feature, and know that @namespace/crate specifically refers to a crate.

Then for the separator mapping @ would be erased I suppose.

[jmillikin]

@Manishearth

No, this is being translated to the crate name foo/bar as well (and will be foo/bar in Cargo.toml), just that foo/bar maps to foo_bar before being handed to rustc . We do something similar for foo-bar already.

Sure, but it's still the same crate name from the compiler's perspective. It wouldn't be possible for two users to register packages containing a crate named bar, the name would implicitly be foo_bar or someuser_bar or so on.

[Manishearth]

@jmillikin

Sure, but it's still the same crate name from the compiler's perspective. It wouldn't be possible for two users to register packages containing a crate named bar , the name would implicitly be foo_bar or someuser_bar or so on.

Worth noting: you can already publish crates that have the same crate name from the compiler's perspective, cargo.toml gives you the freedom to set a lib name that is different from the package name.

I would ask you to go through the first two sections of my original post again. It acknowledges that namespacing is a means to different things for different people, and explicitly calls out the thing you seem to be asking for as out of scope.

This is still, conceptually, a namespace. Yes, it looks the same to rustc, but from a package management point of view, this is a namespace.

[josh]

@mbrubeck

One minor issue with / as the namespace delimiter is the following ambiguity:

[dependencies]
"foo" = "1"
"foo/std" = { version = "1", optional = true }

[features]
# Does this enable crate "foo/std", or feature "std" of crate "foo"?
default = ["foo/std"]

This is a really unfortunate ambiguity.

We do need an improved syntax for cargo features, though, because the current cratename/featurename syntax means both "enable the feature" and "enable the crate", with no way to say "enable this feature but don't enable the crate if it's not already enabled". I think we could fix both in one pass, by saying the following:

• In the [features] section, by default, / always serves as a feature separator, and you can't reference a package with / in it. If it looks like you might be trying to, we can give a better error.
• Add a new feature separator, say @, such that foo@bar means "the bar feature of foo", but does not enable foo itself. So, if you want to enable foo and its bar feature, you'd write ["foo", "foo@bar"].
• The new syntax would then allow features to enable crates that contain / in the name.

[Manishearth]

@josh

The new syntax would then allow features to enable crates that contain / in the name.

Wait, how? The @ is only for subfeatures, and as you said / is always a feature separator.

This is an interesting idea, but how about this for resolving the ambiguity: In feature lists, foo/bar is a feature separator, foo/bar/ is a crate. This works fine with an extension scheme like the one you propose for conditional feature dependencies.

[TBieniek]

@Manishearth

User / org namespaces

Another way to handle namespacing is to rely on usernames and GitHub orgs as namespace roots. This ties crates.io strongly to Github -- currently while GitHub is the only login method, there is nothing preventing others from being added.

Furthermore, usernames are not immutable, and that can lead to a whole host of issues.

There is also the option of having those orgs just within crates.io and making them immutable. We could still optionally couple them with GitHub orgs if we wanted to, but I currently don't see a strong benefit in doing that. It would certainly mean a bit of work on the crates.io side to implement these orgs with user roles and management, but on first glance this looks manageable.

@Manishearth

Separator mapping

The proposal suggests mapping foo/bar to foo_bar , but as mentioned in the typosquatting section, this has problems. There may be other mappings that work out better:

• foo::bar (see section above)
• foo::crate::bar
• foo::/bar
• ~foo::bar

and the like.

There is probably a reason for it that was discussed in one of the other RFCs already, but why not @foo::bar or @foo/bar? Does it conflict with existing syntax?

If the @ prefix is not available, then I think using ~ might be the next best option if it resolves all of the ambiguity concerns.

[josh]

@Manishearth

Wait, how? The @ is only for subfeatures, and as you said / is always a feature separator.

I should have expanded on that: foo/bar@ would enable the crate foo/bar. But I like your idea better:

@Manishearth

This is an interesting idea, but how about this for resolving the ambiguity: In feature lists, foo/bar is a feature separator, foo/bar/ is a crate. This works fine with an extension scheme like the one you propose for conditional feature dependencies.

That sounds much better, and decoupled.

[Nemo157]

There is an existing unstable namespaced features feature which separates crates and features. It seems like this could be updated to take this into account (and made a default in a future edition).

[features]
foo = ["crate:foo"]     # Activate the optional dependency `foo`
bar = ["foo/bar"]       # Activate the feature `bar` of the dependency `foo` (without activating `foo`)
baz = ["crate:foo/bar"] # Activate the optional dependency `foo/bar`

[dhardy]

@Manishearth

While I cover a bunch of different separator choices below, I want to call out foo::bar in particular. If we went with foo::bar , we could have the same crate name in the Rust source and Cargo manifest. This would be amazing .

Except, of course, crate foo::bar is ambiguous with module bar in crate foo (which might actually be a reexport of foo::bar in some cases).

This can still be made to work, e.g. we could use foo::crate::bar to disambiguate, and encourage namespace-using crates to ensure that mod bar in crate foo either doesn't exist or is a reexport of crate foo::bar . I definitely want to see this discussed a bit more.

I really like this proposal. For the most part the name conflict isn't even an issue: the owners of foo know that foo::bar is a sub-package, and therefore if the bar submodule is shadowed it doesn't matter.

The downside is the potential for confusing errors when foo::bar is not a dependency:

use foo::bar::bam;
// error: item `foo::bar` does not exist
// or, worse in the case of a bar module:
// error: item `foo::bar::bam` does not exist