The Great Module Adventure Continues

[matklad]

Another goals clarification question:

Allow paths that appear in use to also be used in fn bodies

Do we want the inverse to be true? That is, that you can take path from fn body and place it into use as is?

I am thinking about the following case:

mod foo;

fn f() {
    foo::bar(); // works
}

mod foo;
use foo::bar; // fails, you need self:: :(

fn f() {
    bar();
}

[nikomatsakis]

Yep, I am discussing variant 3, which is (if we forget about backwards compatability) a slight modification of variant 1, which makes syntax less noisy.

I see, my bad. It’s hard to keep it all straight. I already sort of regret using numbered variants, though I did try to give them names. Anyway, I agree that variant 3 achieves many objectives, and feels like a smaller delta on today, if we can resolve the backwards compatibility aspects of it.

It is an open question to me whether

use ::foo::bar;
use ::hir::tcx::map;

would be something I would eventually get used to. I expect so.

[nikomatsakis]

Do we want the inverse to be true?

I am not sure. I too have wondered this. I think though that if we shoot for an unambiguous syntax (variants 1 or 3) then it can be reached, though only after a “deprecation epoch”. i.e., we need some epoch transition where writing use foo::bar is a hard error (given the current meaning), so that then, later on, we can repurpose it.

[matklad]

FWIW, the smallest delta to hit all the goals except the first one is to just deprecate use foo::bar paths, without adding any new syntax

Do we want the inverse to be true?

I don’t know, by empirically, this is a nice goal, because it seems to me that people are more likely to add use declarations to get rid of qualified paths, rather than go in the opposite direction.

[nikomatsakis]

FWIW, the smallest delta to hit all the goals except the first one is to just deprecate use foo::bar paths, without adding any new synta

The other place we use absolute paths (for better or worse) is pub(in foo). So we would also require pub(in ::foo) there. I think though that effectively all of these approaches are deprecating that notation, if not formally, then informally. The syntax is already long.

I’ve been trying to "just use crate" more and more. I still run into privacy errors – which we were supposed to be putting an end to – which makes it hard to evaluate.

Even so, I haven’t decided how I feel about it. Definitely I find that it makes me draw broader boundaries than I really want. Often there is a more narrow boundary than what is written when I write crate. On the other hand, writing pub(in xxx) to specify the boundary very precisely feels long and tedious and makes the code kind of hard to read, especially when it’s reproduced many times.

Overall, writing crate, combind with ripgrep and some comments, may be the best thing.

[petrochenkov]

I think it’s important to actually implement the fallback and see how it goes and how often it causes resolution to stuck.

To avoid non-deterministic behavior the fallback has to work as a crate-global pass during import resolution, e.g. “1. Iterate import resolution until it saturates” -> “2. Simultaneously fallback all determined unresolved imports to extern crates” -> repeat “1” -> … See https://github.com/rust-lang/rfcs/pull/2126#issuecomment-327776317

(FWIW, after implementing the crate/extern stuff I’m not so vehemently against the RFC 2126 as written, i.e. Variant 2 with fallback, so I’m interested in how it would look in fully implemented backward-compatible form.)

[rpjohnst]

How is variant 3 (::std/::crate) different from variant 2, other than requiring the leading ::? Leading :: already has a meaning in both use/pub(in) and expressions, so wouldn’t it have the same need for fallback? (Although perhaps less frequently because use tends to avoid leading ::…)

Either way, looking at the old thread, one thing we missed here is that variant 2 don’t actually need fallback anyway if we gate the change on an epoch: https://github.com/rust-lang/rfcs/pull/2126#issuecomment-328998289

Instead, in this epoch, we:

• Introduce crate::
• Deprecate unused Cargo.toml dependencies
• Deprecate non-root use of extern crate
• Introduce Cargo.toml crate aliases
• Deprecate absolute paths that don’t begin with one of:
  • A potentially-aliased dependency name (which given the other deprecations, must now be a non-conflicting top-level extern crate item, if it is to avoid warnings)
  • crate (which is new, and so can avoid any ambiguities by simply picking the new resolution)
  • super or self (though these aren’t really absolute paths anymore)

(This last point basically deprecates absolute paths starting with top-level items.)

Then, given the “code with no warnings compiles with the same behavior on the next epoch” rule, the next epoch can simply switch to the new resolution rules and deprecate extern crate.

This, again, retains the advantage of less churn than variant 1. Most code will only need to add crate:: to local paths, rather than changing every absolute path.

[matklad]

How is variant 3 (::std/::crate) different from variant 2, other than requiring the leading ::?

In this version, absolute paths are not syntactically distinguished, but instead distinguished by where they appear.

In ::std/::crate variant, absolute paths are distinguished syntactically, and special casing of use/pub in paths is removed.

Note that crate::foo paths cannot be used in a fn body – rather you type ::crate::foo

In ::std/::crate variant there is just a single syntax for absolute paths that works everywhere.

[rpjohnst]

That doesn’t resolve the ambiguity within paths that start with ::. Today ::some::path can already refer to a local top-level item from both expressions and use paths. If we suddenly make ::some always refer to a dependency, that is not backwards compatible.

[nikomatsakis]

Either way, looking at the old thread, one thing we missed here is that variant 2 don’t actually need fallback anyway if we gate the change on an epoch: https://github.com/rust-lang/rfcs/pull/2126#issuecomment-328998289

Indeed, Variant 2 as I described it was meant to be gated on the epoch – and that is how it is implemented (well, it’s gated on the feature gate). The fallback is if we want to avoid that gating.

[kamalmarhubi]

I don’t think it’s hard to remember what code is yours and isn’t.

This works for you the primary author, but doesn’t work for me the contributor. It may not work for you the primary author a year down the line either.

I think the distinction is actually counter-productive, because it adds friction to graduating modules to their own crates (e.g. if my frobnicator module grows big, I’ll move it to be a crate in my workspace).

This is interesting. I kind of agree, but at the same time you’re already moving files around so the diff isn’t going to be tiny. Find/replace isn’t too different from the move and copy. If you must, pub(crate) use extern::formerly_module as formerly_module; at your main crate’s root.

[dhardy]

I was going to say I liked variant 3, but I mis-read it. Why not this, i.e. without the crate:: part for local names?

use ::std::time; 
use ::top_level_name;
use super::parent_item;
use child_mod::item; // or self::child_mod::item

Advantages:

• this works today
• the only required change is to deprecate usage of absolute paths without leading ::

Optionally also deprecate usage of relative paths in use and pub(in X), excepting super and self keywords.

This avoids the following problem too:

That doesn’t resolve the ambiguity within paths that start with ::. Today ::some::path can already refer to a local top-level item from both expressions and use paths. If we suddenly make ::some always refer to a dependency, that is not backwards compatible.

The only drawback is that it isn’t always immediately obvious whether ::foo refers to another crate or an item at the top of this crate (this is tentatively an advantage, allowing an external dependency to be moved to an internal module and customised easily).

[matklad]

Yeah, I originally thought that the only major problem with current module system is that paths in use are just alien, and that just removing this special case would be enough

By the way, if we remove speacial-casing of use, we could also remove self:: from paths, as it becomes unnecessary .

However, now I find the first goal compelling: it feels right that at the top level there are just a bunch of crates, and the one with name crate is yours.

[dhardy]

To me, that’s a valid goal, but not especially important; hence less breakage is better (strictly my opinion of course).

[rpjohnst]

use ::std::time; 
use ::top_level_name;
use super::parent_item;
use child_mod::item; // or self::child_mod::item

The framing of the problem for a lot of the original discussion was around “path confusion,” the idea that absolute paths (i.e. use, pub(in), or with variant 3 ::) overlap with in-scope paths in the root module, leading beginners to build incorrect mental models that aren’t broken until they go to add a new source file and get confused.

The last line of your example kind of undermines its ability to solve this problem- in the root module, top-level items and dependencies are still in-scope for undeprecated use, at least without further rules around what’s deprecated.

Another issue is that there was some level of support for use paths staying absolute by default, both for backwards compatibility and avoiding churn (use std::time, use regex, etc don’t have to change at all with the RFC/variant 2), and because a lot of uses cross the module tree, at least in some codebases.

Variant 2 helps here because it makes it immediately clear that use paths are absolute- you can’t use something out of a top-level item without prefixing it with crate, because your crate is just one among many.

To me, that’s a valid goal, but not especially important; hence less breakage is better (strictly my opinion of course).

In that light, I think variant 2 arguably has less breakage already. Variants 1 and 3, as well as your suggestion, all force every path to change. Variant 2 only forces internal paths to change, allowing external paths (which IIRC actually turned out to be the majority based on some numbers gathered in the RFC thread) to remain undeprecated.

(And to avoid retreading some ground from the RFC thread: we really can’t fix this problem without forcing some set of paths to change, based on the “path confusion” framing.)

[Rantanen]

A thing I haven’t seen discussed yet is referring to the local or extern crate with the same path. This is a use case that crops up in proc macros, where you might need to emit tokens, such as “::my_crate::SomeTrait”, and you still want to use that proc macro inside my_crate.

Currently I’ve been working around this by having mod my_crate { pub use ::* } inside my_crate, which allows ::my_crate::MyTrait to work either through the actual my_crate crate or the dummy my_crate module.

If the idiomatic way in the future differentiates between these two scenarios, then whatever the syntax for referring to extern::my_crate should also be able to refer to the local crate.

I’m expecting this to be just an implementation detail. I’m suspecting there are no real technical issues that would prevent considering the current crate as an “extern” crate as well when it comes to resolving crate-specific use statements/paths.

[rpjohnst]

Shouldn’t hygiene solve this, so you can just give the token the correct span to have it always resolve as if it were in my_crate?

[Rantanen]

Now that you mention; It might.

Truth is, I’m a bit lost when it comes to span sites and proc macros. Mainly because the concept of def_site seems mostly useless for them: The literal def_site is a proc_macro crate, which wouldn’t export anything anyway, so I’ve always considered call_site to be the only sane option for spans that come from proc macros.

I would love to learn more about the hygiene in proc macros, but this topic isn’t really the place to discuss it. I’m happy if the hygiene solves the above scenario and in that case there shouldn’t be any need to consider that scenario when discussing the module syntax.

[nikomatsakis]

I’ve been sitting on this for a while, and I have to say that “Variant 3” (as proposed by @matklad) is growing on me. This is basically changing two things from today:

• You write use ::std::cmp::min instead of use std::cmp::min
• You write ::crate::foo to select something from the root of your local crate instead of ::foo
  • A key question is whether we can continue (at least in most cases) to support ::foo via a fallback mechanism
    • @petrochenkov suggested this may be possible

Like Variant 1, this meets all the criteria in terms of distinguishing clearly where things are from, allowing paths-in-a-use to be a strict subset of paths-everywhere-else, etc.

It also has the benefit that one can write absolute paths calling out to the standard library, like ::std::fmt::Debug, and they are not unreasonable (no worse than today, basically).

(That said, I do find the leading :: a bit hard on the eyes, I have to say. Maybe that just takes time?)

[mark-i-m]

(That said, I do find the leading :: a bit hard on the eyes, I have to say. Maybe that just takes time?)

I don’t really like the leading :: either… My favor is still with Variant 1 – mostly because I like separating local and external imports. I already do this with blank lines, but having the syntactic help is even better