Relative paths in Rust 2018

[phaylon]

So, this (and the current) proposal have one big problem for me. I tend to format my code as follows:

use tendril;

pub struct MyStruct {
    value: tendril::StrTendril,
}

I only import namespaces at the top level, making the header of the file a sort of preamble of how it connects with other things.

In the current and proposed schemes however, the use tendril can fall away, since tendril::StrTendril is fully qualified due to tendril being in the preamble.

It would be nice if there were an opt-in lint I could activate that would force me to use ::tendril or use tendril or extern::tendril. Just something so I have a hint where things are coming from, instead of having a plain identifier doing a global lookup.

[gbutler]

Yes, I too am really uncomfortable with the implicitness/magic going on here. The fact that we're not even sure what kind of ambiguous imports might happen and how the compiler will have to disambiguate or give an error makes me feel that the proposal is lacking.

@nikomatsakis

(Note: If we made shadowing a hard error universally , we’d be fine.)

So, why not this? Why not have absolute paths be unambiguously absolute? That is:

• use extern::<crate name>::<rest of path>
• let x = extern::<crate name>::<rest of path>::fn_returning_something();

I just don't understand the necessity of having paths that can be interpreted as absolute vs relative depending on the context and what is already in scope. The whole premise sounds fraught with issues.

At some point, an analogy was made to file-system paths. I know of no fs path specification where paths can be interpreted as either absolute or relative depending upon what the current working directory is and what files/sub-directories exist in the current directory.

Could someone ELI5 why we need to have absolute paths not have a pre-fix that makes it 100% clear in all cases that it is an absolute path (without the compiler having to magic its way around analyzing the context). I just don't understand why that is a useful feature bug.

[gbutler]

@petrochenkov

This alternative was actually implemented too for experimentation purposes.

What was the outcome of this experiment? Why was it rejected? Why shouldn't absolute paths be 100% unambiguous?

[matklad]

@SimonSapin

This looks like a breaking change, since such paths in 2015 are equivalent to the proposed crate:: (they can refer to any kind of item at the root of the crate). Not having breaking changes was a big selling point of the previous proposals.

I think this is sort-of the crucial observation to the current discussion. We definitely rejected stuff before because it changed semantics of existing paths. If we are ready to lift this restriction, then why don't we settle for what seems to be the simplest system? Namely:

• Category of path (relative vs. absolute) is determined purely syntactically. Absolute paths look like ::crate_name::some::path, where the crate_name for current crate is keyword crate. Relative paths look like some::path, super paths look like super::some::path. self paths like self::some::path are deprecated, they are exactly equivalent some::path.
• There's no distinction between path in use items and elsewhere.
• Semantics of absolute paths require that the first component is a crate name, and this is a breaking change. Today, ::Foo may refer to the Foo struct in the root module. In the proposal, it is an error and should be spelled as ::crate::Foo. It is more verbose, but with nested imports it shouldn't be a problem.
• Semantics of relative paths is exactly as it is today.

I think the migration to this system can be done purely syntactically (modulo macros which can inspect the concrete syntax):

1. Do not remove extern crate declarations prior to path migration
2. Change all ::foo to ::crate::foo. Note that ::regex changes to ::crate::regex and still works, because extern crate mounts the name to the root module
3. Change all use foo into use ::crate::foo
4. Drop all self::

I believe that, modulo macros, this transformation is always correct. After that, we can get rid of extern crate on case by case basis: this could break in weird cases like #[cfg(foo)] extern crate x; #[cfg(not(foo))] mod x;.

[steveklabnik]

I know this has sort of been said already, but I care strongly enough that it feels worth repeating. @centril brought it up too.

While the initial modules discussion was very stressful, I really liked where we ended up. Changing everything yet again so close to the deadline feels… not great.

This scheme feels more complex in order to gain a tiny bit of unification. I much prefer what we’ve implemented, personally.

[josh]

@gbutler

Why not have absolute paths be unambiguously absolute?

Because nobody wants to write paths like extern::std::result::Result rather than std::result::Result everywhere, just so that you're allowed to have a local declaration named std. That isn't worth the substantial increase in verbosity imposed on all code.

[josh]

@matklad

Semantics of absolute paths require that the first component is a crate name, and this is a breaking change. Today, ::Foo may refer to the Foo struct in the root module. In the proposal, it is an error and should be spelled as ::crate::Foo .

We can and should fix :: to work the same as it did before, to avoid breaking backward compatibility.

[josh]

For the moment, at least, the proposal is to avoid that ambiguity by prohibiting shadowing.

In a Macros 2.0 future, this seems like an aspect of hygiene, as with any other potential name conflicts.

[petrochenkov]

@steveklabnik

I know this has sort of been said already, but I care strongly enough that it feels worth repeating. @centril brought it up too.

While the initial modules discussion was very stressful, I really liked where we ended up. Changing everything yet again so close to the deadline feels… not great.

This scheme feels more complex in order to gain a tiny bit of unification. I much prefer what we’ve implemented, personally.

+1 to this.
It's unlikely that the "use paths are same as non-use paths, but with extra errors" is implementable in the 2018 edition time frame, unless some highly qualified person is assigned to work on this full-time. This is not a trivial problem.
Perhaps we need to leave this for now and reconsider for the next edition.

[petrochenkov]

By the way, there’s a very simple approximation to the "use ident::suffix takes ident either from the current module or from the extern crate list" idea.

It’s desugaring into two “fused” imports

use ident::suffix;

=>

use extern::ident::suffix;
use self::ident::suffix;

using some tweaked error reporting (“if one import fails, but another succeeds, then it’s ok”, like with imports from several namespaces).

It was prototyped and tested before for a slightly different idea:

use ident::suffix;

=>

use crate::ident::suffix;
use self::ident::suffix;

and caused some minor but unacceptable-in-the-same-edition breakage, but we can easily adopt it as a cross-edition change.

[gbutler]

I think if there could be a model where a relative path’s first node corresponds to something currently in scope at the current scope-level and all absolute paths are designated with absolute path prefixes in all cases, things would be more clear. If it could be thought of as the prelude and external crate names being brought into top-level nested scopes/modules by the compiler implicitly, then maybe everything would just make sense.

For absolute paths we have (starting with ::):

• ::<prelude-item>::<rest-of-path>
• ::crate::<rest-of-path>
• ::extern::<create-name>::<rest-of-path>
• ::super::<rest-of-path>
• ::self::<rest-of-path> (same as a relative path/deprecated)

Relative paths start with a symbol:

• <in-scope-module>::<rest-of-path>
• <in-scope-enum>::<enum-element>

This is the same for both use and non-use paths.

Now, for what is in-scope, we first have a set of nested, implicit, top-level modules that “use” the prelude and the external crate names from the cargo.tom/–extern flag:

mod /* unnamed - prelude, created implicitly by the compiler */ 
{
    // compiler adds implicit self alias...
    use :: as self
  use ::std::prelude::*;   // this makes all of the std prelude available at the top-level
  use extern::crate; // makes the implicit "crate" module available at the root implicit module
  mod extern /* created implicitly by the compiler */
  {
    // compiler adds implicit self alias...
    use ::extern as self
    // compiler adds a super alias declaration implicitly....
    use :: as super;
    // these are added implicitly by the compiler to bring the external crate names (declared in cargo.toml/--extern flag) into scope
    use extern crate_1;
    use extern crate_2;
    ...
    use extern crate_n;
    mod crate /* created implicitly by compiler */
    {
      // compiler adds implicit self alias...
      use ::extern::crate as self
      // compiler adds a super alias declaration implicitly....
      use ::extern as super;
      // use statements and mod declarations at the top-level module in the lib.rs or main.rs
     use ...;
     use ...;

     mod ...
     {
        // compiler adds implicit self alias...
        use ::<absolute-path-to-this-mod> as self
        // compiler adds a super alias declaration implicitly to all sub-modules....
        use ::<absolute-path-to-parent-module> as super;
       ...
     } 
    }
  } 
}

Now, with this formulation of things, the keywords (no longer keywords) in path mapped as follows:

• ::<rest-of-path> is rooted at the implicit top-level prelude including implicit module
• ::<prelude-item>::<rest-of-path> would be selecting unambiguously a particular top-level prelude item and a path relative to that
• ::crate::<rest-of-path> means is rooted in the implicit “crate” module (which was aliased from ::extern::crate, see top-level)
• ::extern::<rest-of-path> means is rooted at the “extern” implicit module
• self::<rest-of-path> simply refers to the implicit self alias and is effectively as relative path always (akin to . on fs paths)
• super::<rest-of-path> simply refers to the implicit super alias and is effectively a relative path always (akin to … on fs paths), so, it would be meaningful to do a path like: super::super::super:::: (akin to “…/…/…/”)

Now, the last bit is that the root element of a relative path would be resolved as:

EDIT: Eliminated searching for symbols in parent/gp/ggp scopes except for extern and prelude scope.

• Symbol in the current module scope? Yes? Use it. No? Continue…
• Symbol in parent/super scope? Yes? Use it. No? Continue…
• Symbol in grand-parent scope? Yes? Use it. No? Continue…
• etc. until…
• Symbol in the implicit extern scope? Yes? Use it. No? Continue…
• Symbol in the implicit prelude scope? Yes? Use it. No? Compiler Error…

These rules would make the following relative paths also work correctly:

• crate::<rest-of-path>
• extern::<extern-crate-name>::<rest-of-path>
• self::<rest-of-path>
• super::<rest-of-path>
• <symbol-in-current-module-scope>::<rest-of-path>
• <symbol-in-parent-scope-not-found-in-current-scope>::<rest-of-path>
• <symbol-in-grandparent-scope-not-found-sooner>::<rest-of-path>
• etc.
• <extern-crate-name-not-otherwise-shadowed-locally>::<rest-of-path
• <prelude-symbol-not-otherwise-shadowed-locally>::<rest-of-path>

I think that covers all the cases. To me at least, thinking of it this way, clears up ambiguity. Would warning for shadowing be needed? Perhaps that could then be an optional lint?

[rpjohnst]

@matklad

why don’t we settle for what seems to be the simplest system? Namely:

• Category of path (relative vs. absolute) is determined purely syntactically. Absolute paths look like ::crate_name::some::path , where the crate_name for current crate is keyword crate .

This has the same problem that @josh points out here- it forces all use paths to start with a leading ::, and that's a regression in ergonomics. We've been around the block on this question in just about every modules thread, I think.

I'd also second @steveklabnik's point. What we already have implemented is already quite close to this---this proposal specifically solves the papercut of self:: in use statements, and nothing else, because non-use paths already have the hierarchy root "in-scope" via the prelude-like mechanism for dependencies. So the question is whether the complexity (both in implementation and human understanding), last-minute relitigation, and rush, are worth that.

@gbutler That sort of "read stuff from the parent module" approach was tried pre-1.0 and turned out to be (IIRC, according to @pcwalton) literally unimplementable. It's the king of the ambiguity we keep hitting. We definitely don't want that.

[aturon]

@petrochenkov

By the way, there’s a very simple approximation to the " use ident::suffix takes ident either from the current module or from the extern crate list" idea.

Can you say why this is an approximation rather than a full implementation of the "error on conflict" idea?

[gbutler]

@rpjohnst

That sort of “read stuff from the parent module” approach was tried pre-1.0 and turned out to be (IIRC, according to @pcwalton) literally unimplementable. It’s the king of the ambiguity we keep hitting. We definitely don’t want that.

I've edited the above to eliminate searching parent/gp/ggp/etc for symbols (other than extern and prelude scopes). Does it now align correctly?

[aturon]

@steveklabnik

While the initial modules discussion was very stressful, I really liked where we ended up. Changing everything yet again so close to the deadline feels… not great.

This scheme feels more complex in order to gain a tiny bit of unification. I much prefer what we’ve implemented, personally.

I definitely hear you! But I also think people are just now starting to really play with the existing Rust 2018 modules work (via the preview), so I feel like if we can get this implemented quickly, it's worth offering as an option.

In terms of stress/deadline: my guess is that when we start pushing toward stabilization of anything here, there's going to be a fresh round of discussion; I don't think this proposal really changes things there.

It is feasible to stick with the current setup but future-proof for this one, and that's something we should also consider, but I'm interested to explore the possibility of implementing this proposal first.

Finally, in terms of tradeoffs, I think it depends a lot on your perspective. In my view, making paths work the same everywhere is a big simplification, and I know personally any time I need to use self:: I forget to write it probably 75% of the time.

[petrochenkov]

@aturon

Can you say why this is an approximation rather than a full implementation of the “error on conflict” idea?

You still can't import from preludes (i.e. use Vec; or use u8;) or from unnamed blocks (fn f() { struct S; use S; }).

[nikomatsakis]

So, I do want to add one thing:

I’ve been finding that in my own code I tend to avoid use self:: and use super:: in code that I write. I just find that – on balance – I am happier having a big block of use statements that are all absolute.

(Interestingly, I remember coming to this same conclusion with Python, back before they changed the rules here, and hence agreeing with those rules.)

I think the factors for me are:

• No ambiguity about how to order my import list etc
• I don’t always remember exactly where I am when in a particular module, so use super::foo makes me do a bit more work
• Often I break up modules or move them around, and then in some cases absolute imports work better.
• I can scan and use grep to find imports of things in a pretty clear way
  • (there is basically only one correct path for something)

I’m not saying it’s wrong to use relative paths or anything, just giving my 2c on the pattern I find I often adopt.

[matklad]

@rpjohnst

This has the same problem that @josh points out here- it forces all use paths to start with a leading :: , and that’s a regression in ergonomics.

I don't think this applies to my proposal Only the ergonomics of paths in use items is regressed. Given that ":: in use is optional" is the reason why current system does not have 1path, it seems to be a desirable regression. This is also mitigated by the nested imports. And currently implemented crate:: is, arguably, also a regression in relation to just ::. Moreover, adding extern crates to prelude and changing ::regex to regex also not a strict win: the former is more readable, as articulated in this: Relative paths in Rust 2018 - #21 by phaylon.

Given that current implementation is not 100% backward compatible (Relative paths in Rust 2018 - #11 by petrochenkov), and we continue the search for the best system, I think we should be prepared to not included modules changes in the 2018 edition.

[aturon]

@aturon

It is feasible to stick with the current setup but future-proof for this one, and that’s something we should also consider

@petrochenkov I'm particularly curious to hear your thoughts about this point.

[phaylon]

Thinking about it more, given the following two changes:

• No more extern crate
• All extern crates are in every modules prelude

The “only import namespaces” camp (which I’m in) ends up with the following situation:

• All crate-local namespaces brought in scope are explicit, with use crate::_
• All external toplevel namespaces (which for me is most of them, due to flat hierarchies being common) are neither mentioned with extern crate nor with a use. They just show up in the code without hint or introduction.

I have to admit I find that a bit unergonomic with regards to code clarity. Certainly more than the way things currently are.

Edit: I’ll refer again to my use case with tendril. Neither the lib.rs/main.rs nor the head of a module using it would indicate that there’s a dependency on an external library in the code.