The Great Module Adventure Continues

[withoutboats]

I definitely think this is a part of the dynamic, but I do think being overly lengthy is a problem. For example, the extern syntax makes something like this, which I do often, feel quite bad:

impl extern::std::fmt::Display for MyType {
    fn fmt(&self, f: &mut extern::std::fmt::Display) -> extern::std::fmt::Result {
        // ...
    }
}

[Ixrec]

To me ::std “feels wrong” mostly because of the many, many weird things in C++ that involve having or not having a leading :: on very specific paths. Of course, Rust doesn’t have things like ADL, so personally I don’t trust my intuition on whether or not that syntax would be “genuinely wrong” for Rust. All I can say for sure is that I haven’t yet seen any alternatives that I like better than extern.

[matklad]

Totally agree about verbosity being a problem!

It seems like ::std::foo and ::crate::bar would be pretty close to the minimal possible verbosity, while still being explicit.

[steveklabnik]

I’m feeling pretty good about Variant 1, honestly. I also hear @withoutboats here, but would expect that I’d reach for use a bit more often.

[withoutboats]

It seems like ::std::foo and ::crate::bar would be pretty close to the minimal possible verbosity, while still being explicit.

Unfortunately my understanding is that double colon prefix is out because it already has meaning. :-\

A part of why I like : is that its very similar to ::, though that’s also why other people don’t like it.

[rpjohnst]

I am a huge fan of variant 2, and I’m a bit taken aback by this thread because I thought there was more consensus around that version from previous discussion.

The issue for me with variant 1 is that it “unbalances” the namespace. I think of the crate:: prefix as a stand-in for current_crate_name::, which is important because 1) idiomatic Rust uses multiple crates with the same name (a library and a binary) and 2) it’s like self in that it enables you to rename a crate without a huge diff. (This is also why ::self and ::super don’t really make sense to me- they’re not crates and thus not top-level items in the namespace hierarchy.)

With this in mind, variant 2 places crate:: and some_dependency:: at the same level in the namespace hierarchy, while variant 1 and the status quo place the contents of the current crate one level higher in the namespace hierarchy. Variant 1 avoids some specific pitfalls here, but it feels klugy to me because it doesn’t match other languages which place every library’s namespace at the same level.

One other important point that came up in previous discussion is that std::-in-expressions only works in the root today because of an implicit extern crate std which follows the same behavior as other dependencies. To make it work with extern crate, we could simply add use std to the prelude and let it work everywhere, making it less of a special case without contorting the namespace hierarchy.

[withoutboats]

The issue for me with variant 1 is that it “unbalances” the namespace. I think of the crate:: prefix as a stand-in for current_crate_name::, which is important because 1) idiomatic Rust uses multiple crates with the same name (a library and a binary) and 2) it’s like self in that it enables you to rename a crate without a huge diff. (This is also why ::self and ::super don’t really make sense to me- they’re not crates and thus not top-level items in the namespace hierarchy.)

Several of the variants seem to resolve this concern by treating crate the same as the name of dependencies - @crate::foo::Bar, [crate]::foo::Bar, :crate::foo::Bar, crate:foo::Bar all treat crate as equivalent to regex or serde.

[rpjohnst]

As far as variant 2’s backwards compatibility and fallback, I do think that should be revisited. If I remember the previous discussion accurately, the only case where currently-valid code would actually change behavior under variant 2 is when you simultaneously a) have a top-level item with the same name as a Cargo.toml dependency and b) either don’t use that dependency, rename that dependency, or put its extern crate only in submodule(s).

This set of cases is small enough that any fallback should be able to check for those cases explicitly, at the definition site of the top-level items. If none exist, which I expect to be the common case, ::foo::/use foo:: can behave the old way with a deprecation warning (as would exist in variant 1 as well). If one exists, ::foo::/use foo:: can resolve to the top-level item rather than the dependency.

This arguably has better characteristics for people who are updating code to the new style. Variant 1 requires all absolute paths to change, either by adding a crate:: or by adding an extern:: (or whatever style from 1b). Variant 2 only requires internal paths to change, by adding crate::, and allows dependency paths to remain as they are.

Edit: This fallback would also enable the less-verbose and more-familiar ::dep/::std to keep working.

[rpjohnst]

Several of the variants seem to resolve this concern by treating crate the same as the name of dependencies - @crate::foo::Bar, [crate]::foo::Bar, :crate::foo::Bar, crate:foo::Bar all treat crate as equivalent to regex or serde.

That’s true, but they also introduce an entirely new syntax that has the same problem as vanilla variant 1 of requiring every absolute path to change.

IIUC, one of the reasons for these variants is to make absolute paths in expressions feel nicer. Most discussion I’ve seen around this is more specifically to make std:: nicer, which seems to be an issue when you start hitting the boundaries of what’s in the prelude. Like I mentioned previously, putting std itself in the prelude is enough to handle this case, while leaving arbitrary dependency crates to ::dep or, as I prefer, use dep.

[matklad]

Yeah, that’s true that :: is already used in the current system. OTOH, it would be a shame if we have to use a noisier syntax instead while :: is just deprecated and not reused at all.

[matklad]

Maintain backwards compatibility and avoid fallback

For implementation reasons

Are we 100% sure that this is a hard constraint? Could we just break things and epoch out of fallbacks?

“for implementation reasons” does not sound like a solid motivation in itself…

If this is not a hard constraint, I suggest seriously considering ::std::foo / ::crate::foo flavor of variant 1.

[kornel]

I don’t agree with the first goal being important (“Make it simple to see where an import is coming from”).

I’ve ran into almost every other module-related problem that has been discussed (and I appreciate the improvements being made), but never this one:

1. I don’t think it’s hard to remember what code is yours and isn’t. I know std and serde aren’t mine, but I have frobnicator module. Even for non-trivial projects the files on disk, often always visible in the editor, are a good reminder.
   While you could have both module and a crate with the same name and interchangeably use equivalents of use extern::cookie and use crate::cookie throughout the codebase, this will make all cookie::foo ambiguous on their own (checking top of every file each time is a hassle), so the use syntax isn’t helping enough, and you’re better off using mod mycookie instead.
2. I think the distinction is actually counter-productive, because it adds friction to graduating modules to their own crates (e.g. if my frobnicator module grows big, I’ll move it to be a crate in my workspace). For projects heavily using workspaces the this-crate/not-this-crate distinction is not useful. It’s almost an irrelevant implementation detail. A useful distinction could be my code vs 3rd party code, but the proposed paths syntax doesn’t indicate that accurately in workspace-based projects.

[SimonSapin]

Backwards compatibility:

This proposal is fully backwards compatible and does not require opt-in, though all existing code would be using the deprecated style.

I think this backward compatibility of Variant 1 is essential. But deprecating every single path that is legal today is extremely disruptive. I think this deprecation should be very very gradual: implement a lint, but make it silent by default at first. Consider only making it warn by default in the next epoch.

[djc]

I prefer variant 2 more than variant 1 because the current crate and other crates are referenced at the same level. That is to say, conceptually at a high level all crates exist in the same namespace (the package registry), and any particular crate selects a small subset of these crates to reference things from.

In variant 1, an “extra” level is created to reach outwards; this is nice because it helps with backwards compatibility – given that the current system encodes some things that don’t play well with global namespace view.

In variant 2, we put all the crates on a single level, and we have to live with some short-term incompatibility issues to work through in order to get to the more consistent conceptual solution.

[nikomatsakis]

“for implementation reasons” does not sound like a solid motivation in itself…

So, there are two things going on.

---

On the implementation complexity:

I was a bit vague there, and in part because I don’t remember the full state of play here. What I do remember from my time trying to under name resolution – particularly in the face of globs, macro expansion, and other things – is that it is a highly complicated, fixed-point sort of procedure.

Consider things like this:

• We resolve a path like foo::bar to some particular item.
• We use that resolution to expand a macro; this macro creates a new item that would have taken precedence.
  • (In this case, for example, a macro might introduce an extern crate.)

That said, my understanding of the current resolve algorithm is somewhat out of date. I’ll need to go bring things back into cache to say convincingly.

Let me also quote @petrochenkov here, commenting on the RFC:

The RFC gives the recipe:

First, attempt to resolve the name as an item defined in the top-level module. Otherwise, attempt to resolve the name as an external crate, exactly as we do with extern crate today.

It’s not as simple as that. In presence of globs and macro-expanded items the “attempt to resolve the name” can give indeterminate result. We have to loop through a fixed-point algorithm possibly trying to resolve other imports before we can resolve this one. If only determined resolution failure results in the extern crate fallback, then globs and macros expanded in the crate root may break the fallback mechanism - it will return resolution errors instead. Or we can speculatively search and load extern crates on indeterminacy, which is not desirable. Given that the fallback will have to be supported infinitely, according to the epoch policy, it would be nice if it could be avoided.

Other petrochenkov comments: =)

• https://github.com/rust-lang/rfcs/pull/2126#issuecomment-326914197

---

On the general desirability of writing use std::foo:

But let’s assume that we can make fallback work. Then we still have to decide if we want to. Personally, I am not sure. In particular, it fails to achieve the goal that one can take a path that is used in a use and use it elsewhere, at least on its own. I personally think that’s a worthwhile goal, though of course we have to decide if we find a version and we like, and it’s worth the churn.

[nikomatsakis]

I think this backward compatibility of Variant 1 is essential. But deprecating every single path that is legal today is extremely disruptive. I think this deprecation should be very very gradual: implement a lint, but make it silent by default at first. Consider only making it warn by default in the next epoch.

Yes, this is interesting. I also agree that massive and noisy deprecation, while technically backwards compatible, may not feel great. In the past we have sometimes thought about doing a kind of “silent deprecation”, in which e.g. tools like rustfmt just rewrite your code to use the now preferred form, but you never get warnings or errors about it. Those might come only later, if we actually decide (e.g., in the next epoch) to remove that deprecated form entirely in order to make room for something new.

[nikomatsakis]

Man, I had forgotten how many comments were on the original RFC. If I get some time, I think I will try to go and copy over some of the more persuasive arguments on these various questions. Seems like it’d be useful to avoid retreading ground.

A lot of the discussion there however focused on a slightly different alternative, one in which we introduce extern paths but (conceptually deprecate extern crate), leading to use foo::bar for the local crate and use extern::regex::Regex for others.

I personally find that less appealing than variant 1, but because of the increased asymmetry (you can think of extern::regex as a kind of “unit”) and more importantly because we lose the distinction between absolute vs relative paths.

[matklad]

In particular, it fails to achieve the goal that one can take a path that is used in a use and use it elsewhere, at least on its own.

Hm, I think in this regard ::std/::crate proposal is isomorphic to variant 1? Like, in V1 we deprecate use std::foo and require to write use extern::std::foo. In ::std/::crate we as well deprecate use std::foo and require to write use ::std::foo. The :: plays exactly the same role as extern::, but is strictly shorter

Am I correct that we don’t want to lean on epochs to make a breaking change here, because “to upgrade to a newer epoch, you must change every use statement” is unacceptable?

[nikomatsakis]

Perhaps you are discussing a Variant 3, and I am confused. Are you saying that I would write use ::std::foo and use ::crate::foo, or use std::foo and use crate::foo?

[matklad]

Yep, I am discussing variant 3, which is (if we forget about backwards compatability) a slight modification of variant 1, which makes syntax less noisy.

In variant 1, absolute paths always begin with extern::crate_name or with crate::module_name. In my proposal, absolute paths begin with ::crate_name or with ::crate::module_name.

So yeah, one would write

use ::std::foo; 
use ::crate::bar;

I am unsatisfied with variant 1, because it completely deprecates ::foo form of paths, which, at least to me, seems strictly better than extern::.

EDIT: “seems strictly better” on the syntactic level: I want to reuse ::foo syntax, but with a semantics of variant 1: it looks up foo crate (and not foo item in the root module), and the current crate gets the name crate.