Relative paths in Rust 2018

phaylon · July 5, 2018, 12:23pm

So, this (and the current) proposal have one big problem for me. I tend to format my code as follows:

use tendril;

pub struct MyStruct {
    value: tendril::StrTendril,
}

I only import namespaces at the top level, making the header of the file a sort of preamble of how it connects with other things.

In the current and proposed schemes however, the use tendril can fall away, since tendril::StrTendril is fully qualified due to tendril being in the preamble.

It would be nice if there were an opt-in lint I could activate that would force me to use ::tendril or use tendril or extern::tendril. Just something so I have a hint where things are coming from, instead of having a plain identifier doing a global lookup.

gbutler · July 5, 2018, 1:09pm

Yes, I too am really uncomfortable with the implicitness/magic going on here. The fact that we're not even sure what kind of ambiguous imports might happen and how the compiler will have to disambiguate or give an error makes me feel that the proposal is lacking.

So, why not this? Why not have absolute paths be unambiguously absolute? That is:

use extern::<crate name>::<rest of path>
let x = extern::<crate name>::<rest of path>::fn_returning_something();

I just don't understand the necessity of having paths that can be interpreted as absolute vs relative depending on the context and what is already in scope. The whole premise sounds fraught with issues.

At some point, an analogy was made to file-system paths. I know of no fs path specification where paths can be interpreted as either absolute or relative depending upon what the current working directory is and what files/sub-directories exist in the current directory.

Could someone ELI5 why we need to have absolute paths not have a pre-fix that makes it 100% clear in all cases that it is an absolute path (without the compiler having to magic its way around analyzing the context). I just don't understand why that is a useful ~~feature~~ bug.

gbutler · July 5, 2018, 1:19pm

What was the outcome of this experiment? Why was it rejected? Why shouldn't absolute paths be 100% unambiguous?

matklad · July 5, 2018, 1:58pm

I think this is sort-of the crucial observation to the current discussion. We definitely rejected stuff before because it changed semantics of existing paths. If we are ready to lift this restriction, then why don't we settle for what seems to be the simplest system? Namely:

Category of path (relative vs. absolute) is determined purely syntactically. Absolute paths look like ::crate_name::some::path, where the crate_name for current crate is keyword crate. Relative paths look like some::path, super paths look like super::some::path. self paths like self::some::path are deprecated, they are exactly equivalent some::path.
There's no distinction between path in use items and elsewhere.
Semantics of absolute paths require that the first component is a crate name, and this is a breaking change. Today, ::Foo may refer to the Foo struct in the root module. In the proposal, it is an error and should be spelled as ::crate::Foo. It is more verbose, but with nested imports it shouldn't be a problem.
Semantics of relative paths is exactly as it is today.

I think the migration to this system can be done purely syntactically (modulo macros which can inspect the concrete syntax):

Do not remove extern crate declarations prior to path migration
Change all ::foo to ::crate::foo. Note that ::regex changes to ::crate::regex and still works, because extern crate mounts the name to the root module
Change all use foo into use ::crate::foo
Drop all self::

I believe that, modulo macros, this transformation is always correct. After that, we can get rid of extern crate on case by case basis: this could break in weird cases like #[cfg(foo)] extern crate x; #[cfg(not(foo))] mod x;.

steveklabnik · July 5, 2018, 3:53pm

I know this has sort of been said already, but I care strongly enough that it feels worth repeating. @centril brought it up too.

While the initial modules discussion was very stressful, I really liked where we ended up. Changing everything yet again so close to the deadline feels… not great.

This scheme feels more complex in order to gain a tiny bit of unification. I much prefer what we’ve implemented, personally.

josh · July 5, 2018, 3:57pm

Because nobody wants to write paths like extern::std::result::Result rather than std::result::Result everywhere, just so that you're allowed to have a local declaration named std. That isn't worth the substantial increase in verbosity imposed on all code.

josh · July 5, 2018, 3:59pm

We can and should fix :: to work the same as it did before, to avoid breaking backward compatibility.

josh · July 5, 2018, 4:05pm

For the moment, at least, the proposal is to avoid that ambiguity by prohibiting shadowing.

In a Macros 2.0 future, this seems like an aspect of hygiene, as with any other potential name conflicts.

petrochenkov · July 5, 2018, 4:15pm

+1 to this.
It's unlikely that the "use paths are same as non-use paths, but with extra errors" is implementable in the 2018 edition time frame, unless some highly qualified person is assigned to work on this full-time. This is not a trivial problem.
Perhaps we need to leave this for now and reconsider for the next edition.

petrochenkov · July 5, 2018, 4:26pm

By the way, there’s a very simple approximation to the "use ident::suffix takes ident either from the current module or from the extern crate list" idea.

It’s desugaring into two “fused” imports

use ident::suffix;

=>

use extern::ident::suffix;
use self::ident::suffix;

using some tweaked error reporting (“if one import fails, but another succeeds, then it’s ok”, like with imports from several namespaces).

It was prototyped and tested before for a slightly different idea:

use ident::suffix;

=>

use crate::ident::suffix;
use self::ident::suffix;

and caused some minor but unacceptable-in-the-same-edition breakage, but we can easily adopt it as a cross-edition change.

gbutler · July 5, 2018, 4:40pm

I think if there could be a model where a relative path’s first node corresponds to something currently in scope at the current scope-level and all absolute paths are designated with absolute path prefixes in all cases, things would be more clear. If it could be thought of as the prelude and external crate names being brought into top-level nested scopes/modules by the compiler implicitly, then maybe everything would just make sense.

For absolute paths we have (starting with ::):

::<prelude-item>::<rest-of-path>
::crate::<rest-of-path>
::extern::<create-name>::<rest-of-path>
::super::<rest-of-path>
::self::<rest-of-path> (same as a relative path/deprecated)

Relative paths start with a symbol:

<in-scope-module>::<rest-of-path>
<in-scope-enum>::<enum-element>

This is the same for both use and non-use paths.

Now, for what is in-scope, we first have a set of nested, implicit, top-level modules that “use” the prelude and the external crate names from the cargo.tom/–extern flag:

mod /* unnamed - prelude, created implicitly by the compiler */ 
{
    // compiler adds implicit self alias...
    use :: as self
  use ::std::prelude::*;   // this makes all of the std prelude available at the top-level
  use extern::crate; // makes the implicit "crate" module available at the root implicit module
  mod extern /* created implicitly by the compiler */
  {
    // compiler adds implicit self alias...
    use ::extern as self
    // compiler adds a super alias declaration implicitly....
    use :: as super;
    // these are added implicitly by the compiler to bring the external crate names (declared in cargo.toml/--extern flag) into scope
    use extern crate_1;
    use extern crate_2;
    ...
    use extern crate_n;
    mod crate /* created implicitly by compiler */
    {
      // compiler adds implicit self alias...
      use ::extern::crate as self
      // compiler adds a super alias declaration implicitly....
      use ::extern as super;
      // use statements and mod declarations at the top-level module in the lib.rs or main.rs
     use ...;
     use ...;

     mod ...
     {
        // compiler adds implicit self alias...
        use ::<absolute-path-to-this-mod> as self
        // compiler adds a super alias declaration implicitly to all sub-modules....
        use ::<absolute-path-to-parent-module> as super;
       ...
     } 
    }
  } 
}

Now, with this formulation of things, the keywords (no longer keywords) in path mapped as follows:

::<rest-of-path> is rooted at the implicit top-level prelude including implicit module
::<prelude-item>::<rest-of-path> would be selecting unambiguously a particular top-level prelude item and a path relative to that
::crate::<rest-of-path> means is rooted in the implicit “crate” module (which was aliased from ::extern::crate, see top-level)
::extern::<rest-of-path> means is rooted at the “extern” implicit module
self::<rest-of-path> simply refers to the implicit self alias and is effectively as relative path always (akin to . on fs paths)
super::<rest-of-path> simply refers to the implicit super alias and is effectively a relative path always (akin to … on fs paths), so, it would be meaningful to do a path like: super::super::super:::: (akin to “…/…/…/”)

Now, the last bit is that the root element of a relative path would be resolved as:

EDIT: Eliminated searching for symbols in parent/gp/ggp scopes except for extern and prelude scope.

Symbol in the current module scope? Yes? Use it. No? Continue…
~~Symbol in parent/super scope? Yes? Use it. No? Continue…~~
~~Symbol in grand-parent scope? Yes? Use it. No? Continue…~~
~~etc. until…~~
Symbol in the implicit extern scope? Yes? Use it. No? Continue…
Symbol in the implicit prelude scope? Yes? Use it. No? Compiler Error…

These rules would make the following relative paths also work correctly:

crate::<rest-of-path>
extern::<extern-crate-name>::<rest-of-path>
self::<rest-of-path>
super::<rest-of-path>
<symbol-in-current-module-scope>::<rest-of-path>
~~<symbol-in-parent-scope-not-found-in-current-scope>::<rest-of-path>~~
~~<symbol-in-grandparent-scope-not-found-sooner>::<rest-of-path>~~
etc.
<extern-crate-name-not-otherwise-shadowed-locally>::<rest-of-path
<prelude-symbol-not-otherwise-shadowed-locally>::<rest-of-path>

I think that covers all the cases. To me at least, thinking of it this way, clears up ambiguity. Would warning for shadowing be needed? Perhaps that could then be an optional lint?

rpjohnst · July 5, 2018, 4:43pm

This has the same problem that @josh points out here- it forces all use paths to start with a leading ::, and that's a regression in ergonomics. We've been around the block on this question in just about every modules thread, I think.

I'd also second @steveklabnik's point. What we already have implemented is already quite close to this---this proposal specifically solves the papercut of self:: in use statements, and nothing else, because non-use paths already have the hierarchy root "in-scope" via the prelude-like mechanism for dependencies. So the question is whether the complexity (both in implementation and human understanding), last-minute relitigation, and rush, are worth that.

@gbutler That sort of "read stuff from the parent module" approach was tried pre-1.0 and turned out to be (IIRC, according to @pcwalton) literally unimplementable. It's the king of the ambiguity we keep hitting. We definitely don't want that.

aturon · July 5, 2018, 4:43pm

Can you say why this is an approximation rather than a full implementation of the "error on conflict" idea?

gbutler · July 5, 2018, 4:50pm

I've edited the above to eliminate searching parent/gp/ggp/etc for symbols (other than extern and prelude scopes). Does it now align correctly?

aturon · July 5, 2018, 4:52pm

I definitely hear you! But I also think people are just now starting to really play with the existing Rust 2018 modules work (via the preview), so I feel like if we can get this implemented quickly, it's worth offering as an option.

In terms of stress/deadline: my guess is that when we start pushing toward stabilization of anything here, there's going to be a fresh round of discussion; I don't think this proposal really changes things there.

It is feasible to stick with the current setup but future-proof for this one, and that's something we should also consider, but I'm interested to explore the possibility of implementing this proposal first.

Finally, in terms of tradeoffs, I think it depends a lot on your perspective. In my view, making paths work the same everywhere is a big simplification, and I know personally any time I need to use self:: I forget to write it probably 75% of the time.

petrochenkov · July 5, 2018, 4:56pm

You still can't import from preludes (i.e. use Vec; or use u8;) or from unnamed blocks (fn f() { struct S; use S; }).

nikomatsakis · July 5, 2018, 5:00pm

So, I do want to add one thing:

I’ve been finding that in my own code I tend to avoid use self:: and use super:: in code that I write. I just find that – on balance – I am happier having a big block of use statements that are all absolute.

(Interestingly, I remember coming to this same conclusion with Python, back before they changed the rules here, and hence agreeing with those rules.)

I think the factors for me are:

No ambiguity about how to order my import list etc
I don’t always remember exactly where I am when in a particular module, so use super::foo makes me do a bit more work
Often I break up modules or move them around, and then in some cases absolute imports work better.
I can scan and use grep to find imports of things in a pretty clear way
- (there is basically only one correct path for something)

I’m not saying it’s wrong to use relative paths or anything, just giving my 2c on the pattern I find I often adopt.

matklad · July 5, 2018, 5:03pm

I don't think this applies to my proposal Only the ergonomics of paths in use items is regressed. Given that ":: in use is optional" is the reason why current system does not have 1path, it seems to be a desirable regression. This is also mitigated by the nested imports. And currently implemented crate:: is, arguably, also a regression in relation to just ::. Moreover, adding extern crates to prelude and changing ::regex to regex also not a strict win: the former is more readable, as articulated in this: Relative paths in Rust 2018 - #21 by phaylon.

Given that current implementation is not 100% backward compatible (Relative paths in Rust 2018 - #11 by petrochenkov), and we continue the search for the best system, I think we should be prepared to not included modules changes in the 2018 edition.

aturon · July 5, 2018, 5:05pm

@petrochenkov I'm particularly curious to hear your thoughts about this point.

phaylon · July 5, 2018, 5:12pm

Thinking about it more, given the following two changes:

No more extern crate
All extern crates are in every modules prelude

The “only import namespaces” camp (which I’m in) ends up with the following situation:

All crate-local namespaces brought in scope are explicit, with use crate::_
All external toplevel namespaces (which for me is most of them, due to flat hierarchies being common) are neither mentioned with extern crate nor with a use. They just show up in the code without hint or introduction.

I have to admit I find that a bit unergonomic with regards to code clarity. Certainly more than the way things currently are.

Edit: I’ll refer again to my use case with tendril. Neither the lib.rs/main.rs nor the head of a module using it would indicate that there’s a dependency on an external library in the code.

Topic		Replies	Views
Relative paths and Rust 2018 `use` statements language design	43	5093	July 5, 2018
Module paths suggestion	6	1529	March 25, 2019
`use self::super` is accepted. Is this intended? language design	2	791	February 14, 2022
Revisiting modules -- `[other_crate]::path` syntax language design	32	2794	March 25, 2019
Use statements & paths (and the epoch proposal)	13	1797	March 25, 2019

Relative paths in Rust 2018

Related Topics