Pre-RFC: User namespaces on crates.io

I read and quoted that comment.

No, you didn't. You are talking about a different comment.

I do not think they have been shown incorrect at all. I wonder what you are referring to here.

I'm referring to the believe that non-DNS-based namespaces achieve some platonic ideal of immutability.

This is not supported by any facts and is such an extraordinary claim that needs extraordinarily strong evidence (of which none was provided), lest it be readily dismissed.

As mentioned before, a reasonably achievable scope of immutability is to limit it to a recording of facts of the past, e. g.

  • Version 1.0.0
  • ... of crate foo
  • ... in namespace example.com
  • ... has been published on 2012-09-12T12:34:56
  • ... by a user with username X
  • ... who could prove ownership over that domain at that point in time.
  • Here may be the artifact, if it hasn’t been removed by legal requirements.

Anything making immutability promises beyond that is misleading and dangerous if used as a core assumption for a namespacing system.

It did not contain any arguments for how to achieve immutable names with DNS, or for why using DNS has sufficiently many benefits to make it worth dropping immutable names.

The promise of better immutability, "if only crates.io was acting as its own naming authority (instead of using an external one)", are simply not grounded in reality. I fear users are going to learn it the hard way.

You seem to value proper immutable names less than other concerns, which is fair, but what is not fair is expecting everyone to agree with that.

I think it's important to align

  • the expectation/intuition of users
  • the promises made by crates/cargo/...
  • the actual reality

in terms of immutability.

Seeing a substantial number of people thinking that namespaces would be "more immutable" if they weren't DNS-based indicates how poorly your preferred approaches are doing in this regard.

So, no, I'm not valuing "immutable names less than other concerns"; I'm simply not over-promising immutability that cannot be realistically achieved.

The benefits of this design goal have been stated repeatedly (most importantly: no "left-pad"-style issues)

No. "left-pad" is a problem caused by repositories allowing the deletion of published releases – nobody is arguing for that.

The "kik" problem (that caused "left-pad") on the other hand – now that's a problem that can only exist without namespaces, or with "made-up" namespaces; but not with DNS-based namespaces.


(This is intended to be my last comment on this topic and is largely written for the benefit of future software archeologists asking "why is everything so broken?". Thank you for your understanding.)

1 Like

One problem with a DNS-based solution: domain names cost money. Would we really be comfortable telling someone if they want a namespace then they need to pay up?

This has been addressed.

Link? I don't recall seeing that anywhere in this thread, and imo it's a major issue.

There’s plenty of public suffixes, like github.io, neocities.org, and js.org, for anyone that wants a domain for free.

I'm not seeing a good rationale for using domain names. If a crate namespace owner has to prove that they own a particular web domain, what problem does that solve? How is it better than just using arbitrary identifiers?

Also, since many domain owners shield their ownership from public view by using a privacy service (e.g., https://www.dreamhost.com/domains/private-registration/), how is ownership to be proved in such cases?

This has been addressed.

This has been addressed, too.

Moderator note: If you don't want to discuss it any more, then don't. Announcing that you're done with a thread like that is already needlessly dramatic. Continuing to participate afterward only makes it look worse.

Telling everyone "this has been addressed" like a broken record is the kind of behaviour that gets you in trouble. You are not acting your best here.

12 Likes

I agree With @burntsushi's comment above regarding high level design - seems like most "Pre-FRC" style discussions waste a lot of energy on implementation details and cause a lot of redundant and fruitless frustration. As the saying goes "Can't see the forest for the trees".

So let's start from first principles and define our goals. Here's my take:

  1. As a Rust developer I'd like to be able to easily find and "install" crates.
  2. As a Rust developer I'd like to see a clear retention policy and local caching measures so that I can avoid the "left-pad" fiasco.
  3. As a Rust developer I'd like to make available crates I own to be searchable and usable by other developers, while retaining ownership and control over the code and associated policies (licensing, retention policy, etc.)
  4. As a rust developer I should be able to change my hosting provider, be able to rename my project or transfer ownership to someone else without impacting other users unreasonably.

Non goals:

  1. Crate names do not have to be globally unique - global properties are expensive in complexity terms. It is sufficient to require that the local set of crates on my machine is unique (local reasoning is a better design). For example, this allows to have mirrors and backups and competing crates (think: "facebook/log" vs. "google/log"). Two crates named "foo" would conflict and therefore Cargo should fetch only a single instance of "foo" after resolving my mirror list for the crate "foo". In case of name conflict, I, the user, must resolve the conflict locally by e.g. renaming one of the crates.
  2. Immutable repository/global singular repository/ etc - These are all unnecessary and they complicate other aspects of the design. Keeping crates indefinitely is an overreach and raises legal questions. It is sufficient to have a clear retention policy that is well publicised and explained to users. Currently, cargo downloads sources from crates.io and compiles locally and the sources are stored on github. This is both redundant if my repo is already on github, and insufficient if I prefer to use a different provider or if github T&Cs conflict with my circumstances. Also, A federated-like system where I retain control over my own code is better suited to confer ownership. it is sufficient if crates.io can search and redirect users to my own repository from which I vendor my code. Ideally, there should be some cloud-like hosting solutions to alleviate some of the hassles involved.

As alluded to above, we need some aspects of a federated system. For starters, It confers trust explicitly in my local list of trusted repositories. crates.io would not own the namespace in any way, it simply provides a plugable way for me to add my repository with my retention and other policies available to be searched. The user must make an explicit decision to trust my repository and accept my policies before downloading artefacts. If I'm an OSS project owner I can allow crates.io to be an official mirror and cache my artefacts or if I work for a company such as google I may very well prefer to host my artefacts on company servers with company retention policies. Instead of registering the company name/user name/project name/dns addres on creates.io I rather have an API to plug-in my repo to the official index and make it explicit to users that the artifacts are from my repo and adhere to my retention policies.

5 Likes

To add to the above, I'd envision the following scenario:

Let's say, I have a project with multiple crates that, for instance, is managed under a github organisation. I should be able to fork a template crate.io style repo project and host it at my_project.github.io. I would than need to connect my "my_project.github.io" repo to crates.io. I can then explicitly set my retention policy so that crate.io will retain a cache of artefacts.

My crates will then be available to crate.io website users with a web-page that explicitly explains the above retention policy and a link to a "port-of-origin" (my_project.github.io) as well as instructions how to add the repo. The repo itself should also be searchable by name.

Users would need to explicitly add "my_project.github.io" to their trusted repos list. users can then depend on "my_project/foo" which is mapped locally to my_project.github.io.

There need to be a certificate or something that both crates.io itself and the end user both use to identify my crates, which would allow cargo to resolve to other mirrors (including crates.io itself) in case my repo isn't reachable.

I really like the reframing of the discussion as problems and use-cases to solve.

Namespaces are a particular solution, but they aren't a problem to solve.

This hits the XY problem pattern of miscommunication. We've had a lot of posts written from the perspective of "I want this solution, here's how you implement it", but without explicit definition of why, and what problem does it solve.

Maybe the same problems that are solved with namespaces could be solved in different ways? Probably not one single thing, but a combination of them.

  • Cargo already supports package aliases, names of libraries independent of crate names, and alternative registries. Those are existing tools to work with. If they're not sufficient, maybe they'd work if they were extended/improved?

  • Crates are tied to crates.io website. The website could do a lot more to display ownership, groups of crates, TOML snippets with package aliases.

  • There's crates.io API with ownership information. Crates can verifiably associate themselves with GitHub repos via repository link. There are many ways in which tooling could surface and leverage that information.

Please brainstorm ideas around these things, instead of assuming a particular solution from the beginning and then trying to jam it in.

5 Likes

I mean, no it isn't an extraordinary claim. Immutably is not actually difficult -- just do not provide any way to mutate something (except if the original crate owner explicitly transfers ownership). That's what crates.io does. End of evidence. Contrast this with DNS where changing the owner of a domain without the consent of the old owner is an explicit feature called "domain expiry". (Not sure what platonic ideals have to do with this though, that sounds like a strawman.)

But anyway, since your idea of "evidence" seems to be "it has been addressed [somewhere on the internet but I won't tell you where so you have no way to verify]" (many of your comments do not have links to back up your claim, like this and this and this), which is not evidence at all, and also since you seem unable to accept that different viewpoints than yours can legitimately exist, there is no point in continuing this.

11 Likes