Pre-RFC: Packages as Optional Namespaces

Isn't this just about as confusing as forgetting to enable a feature already is? If subcrates are featured prominently on crates.io and in the docs of a crate (probably best by some new rustdoc feature so you can list subcrates in your main source file that get presented in the generated docs right next to the crates's top level modules for easy discoverability) they ought to be at least as easy to discover as features are currently, right?

1 Like

How about using workspaces as a namespace?

If you own a crate named foo, you can make it a workspace, and publish the entire workspace at once with all the sub crates in it. Then / in the crate name/path would look into the workspace.

There are many projects that already use workspaces and monorepos, and when they publish, it's a chore to publish all of the individual crates. Lots of crates have -derive and -core versions. This is a pain to maintain, so e.g. tokio switched from tokio-foo to just one crate with feature flags. But if crates-io supported workspaces, then workspace == namespace would solve two problems at once, and it'd be a novel useful functionality, not just a fancy mapping of slashes to underscores.

Here is the problem I see. As of 1.0.0 I have a crate foo, with important functionality in the bar module. Now bar is getting more complicated and can be split out. For 2.0.0 I want to make a foo/bar crate, and remove the foo::bar module. During the upgrade period a user depends on foo="1.0" and "foo/bar"="2.0" and has use foo::bar in their code. :boom: rustc needs to have a "ambiguous use statement error", and so this RFC has changes for rustc.

1 Like

I dislike this, because it forces you to organize your code a certain way to be able to get namespaces. This is something I hate from C++, for example: C++ forces you to organize your code a certain way if you wish to use inlining and templates. One should not be forced to use a monorepo to get namespaces.

I am very much looking forward to proposals that allow you to use workspaces to co-version projects as a single "package" containing multiple co-versioned "crates": I know someone working on such a proposal. I do not think this proposal should do this, I consider it to be orthogonal.

Actually, there would be no such error, foo::bar would default to the namespaced crate if both exist (foo::crate::bar is how you get to the module), and an informal contract we ask people to maintain is that if namespaced foo::bar exists, mod bar in foo should either not exist or be a simple reexport.

However, this still needs rustc changes because rustc needs to be able to handle crates with :: in their name and understand what that means.

There is no such limitation? All valid crate names are valid namespace names. I don't see where you get this from.

You’re right, there isn’t, consider it withdrawn. I think I was confused by lack of hyphenated examples and the above sentence talking about async-std wanting the async namespace.

Yeah, that comes from my impression that the "informal namespace" async-std seems to be occupying is async-* not async-std-*. There are non-async-std crates in that "informal namespace" (as there are under serde-*), but most of the crates published by that organization follow async-*, and I figure they'd want async/

The fact that it's a keyword makes async a bad namespace example. :slight_smile:

Well, it is, but it's not one of the original ones, so crates.io allows the name, and there is a crate squatted there which they could ask to take over. But yes, not a great example.

Is there a reason to not use reverse-DNS notation for the package names? "." is not currently a valid crate name character so it can be treated as 'reserved'. e.g. com.gitlab.ehiggs.rust-gst.

For backwards compatibility I think it makes sense to infer any non-reverse-DNS crates to be io.crates.<crate-name>.

I think the only thing we need is to lock the crate name prefix in crates.io, such as tokio and tokio-, to forbidden the other unofficial people using this prefix.

DNS isn't stable, it changes hands, and verification is annoying. Furthermore having package names that long is kind of painful, I always hated that in Java.

Yes, this could be done, except we already have crates where foo and foo-bar are owned by different groups, and as mentioned in the pre-RFC, sometimes that is intentional.

6 Likes

AIUI you don't register a domain, you register a prefix on the Maven Central servers. This is why you can have com.github.manishearth even though you don't own github.com.

But verification and signing are cornerstones of managing packages aren't they? What code am I importing into my project? Was it middleman and injected? Is tokio-naughty a project that is a trusted as tokio-core? How can we do namespacing without managing verification?

This is something the tooling handles so its surprising that this would cause pain. An alternative is C#'s [Organization].[Technology].[Feature]

Just a note: GitHub, GitLab and BitBucket all allow renaming your user account, and other users can claim your old usernames.

3 Likes

As some clarification here: async-std doesn't claim the async namespace (the naming background is "an async version of std" and not "the async.rs projects std" and tries to make sure most async- crates published are only based on futures and generally usable. It would not be named async, even if the name were available.

So it is indeed a good example to talk about there, because it is an interesting configuration to talk about.

As someone involved in the naming, I am retroactively much more annoyed that we picked async.rs as the projects name than calling async-std the name it has.

It's not at all clear to me how you'd be proposing to use DNS in package namespacing. Can you be more concrete?

1 Like

I am not proposing to use DNS in package name spacing. I am proposing reverse DNS notation. This is just to have a fully qualified namespace for the packages which can often be based on the domain.

Basically what I am proposing is what Java has. It's not novel or exciting but it works extremely well and scales very well and covers a lot of the issues that need to be solve: provenance, and security.

I think the confusion is that people might believe that there is a solid link between DNS and the reverse DNS notation being used. In reality, the OSSRH process involves manual labor where accounts need to be approved. The approval process involves having someone check that you have e.g. an email account on the domain. Here is a nice video explaining the process.

1 Like

There was some discussion about OSSRH on another thread. As the crates.io team doesn't have a full-time staff to curate namespaces like Sonatype does, an approach which needs something like a JIRA ticket queue and manual review of each namespace is probably not going to work.

1 Like

Yes, but this is worse, because this forces everyone to register on their domain or github username.

Let's say that I am perfectly fine with having my projects be unnamespaced (which I am). I'm still going to have to scramble to register net.manishearth and com.github.manishearth so that others don't use it. Projects wishing to have a project identity will likely have to register a domain to cement their identity. Sure, they don't have to, but they'll likely want to.

So you either need verification (and thus force people to buy domains), or you don't (and force people to proactively squat their own usernames and domains, and still force people to buy domains when they want a nice project identity).

The point of my proposal is that it's optional -- if you don't want to use namespacing you don't really have to think about it, and that's why it's entirely tied to existing project names -- the namespaces do not derive from anything external to crates.io.

My proposal already solves this, directly at the source. tokio-naughty is not trusted, tokio/naughty is. We do not need additional verification tied to a whole other ecosystem (DNS) to make this work. It is easy to understand ownership as being tied to other packages.

How do you know that rs.tokio.foo is actually a tokio package in this world? What if the tokio project did not actually register that domain?

I've written a fair amount of Java without an IDE. I've written almost all my Rust without an IDE. Java is far more inclined to make choices that require tooling support (and is clunky without it).

4 Likes

The understaffed crates.io team has been historically very reluctant to any kind of manual verification. I do not speak for them here, but I think this proposal is a non-starter for that reason alone.