Relative paths in Rust 2018

[josh]

Yes, this is now full 1path. It doesn’t require full iteration to a fixed point, though, precisely because of the restriction on module/crate name collisions. That simplifies name resolution drastically. (We could potentially support shadowing in the future, but that’d make name resolution harder.)

[matklad]

I really like this because this achieves 1path and is very close to what I’ve proposed in The Great Module Adventure Continues, + ambiguity rules !

Technical questions/points:

• Am I correct that foo and self::foo are almost equivalent? self::foo won’t work, but foo work will only if foo is in prelude/extern crate.

• Looks like an alternative formulation is “we add extern crates to prelude, overriding existing prelude items”.

• After this proposal, you’ll almost never write self::, ::foo, which feels like a nice simplifications?

• Behavior with respect to ::* imports should be specified. Currently, they shadow prelude and are shadowed by anything else.

• Ambiguity rules are an interesting solution! I think it even might help with IDE performance, because you’ll be able to stop early during transitive imports resolution. However, we must specify how they work with type/value/macro namespaces? Is it an error to use regex when you have fn regex() in scope as well as an the regex extern crate?

[josh]

Am I correct that foo and self::foo are almost equivalent? self::foo won’t work, but foo work only if foo is in prelude/extern crate.

Right. (Assuming that by your second sentence you mean “the only case where self::foo won’t work but foo will is …”)

Looks like an alternative formulation is “we add extern crates to prelude, overriding existing prelude items”.

That’s exactly how I think of it, yes.

After this proposal, you’ll almost never write self:: , ::foo , which feels like a nice simplifications?

Right.

[SimonSapin]

Starts with :: : the first name in a path is an external crate name.

This looks like a breaking change, since such paths in 2015 are equivalent to the proposed crate:: (they can refer to any kind of item at the root of the crate). Not having breaking changes was a big selling point of the previous proposals.

[josh]

As mentioned elsewhere in the proposal, “We may instead want to preserve the existing behavior of :: from Rust 2015.”. That’s the approach I would favor. In particular, it shouldn’t be particularly hard to have :: follow this resolution algorithm:

• First look at crate::
• Then look at extern crates
• Then look at the prelude

That would allow all of ::top_level_module::foo , ::some_crate::bar , and ::std::baz to work.

[petrochenkov]

This looks like a breaking change, since such paths in 2015 are equivalent to the proposed crate:: (they can refer to any kind of item at the root of the crate). Not having breaking changes was a big selling point of the previous proposals.

Huh? The previous proposal (which is currently implemented) implies a huge breaking change on edition switch (but no breakage without edition switch), look at any “2015 -> 2018” transition PR.
This proposal seems to keep the same approach.

EDIT: ::something already means extern::something in edition 2018 in the current implementation.

[petrochenkov]

Wait, so this is now full 1path? Wasn’t that previously shot down on principle for making name resolution intractable?

Exactly!

This proposal seems to make it tractable by declaring any ambiguity an error:

So instead, we can make it a hard error to write an ambiguous use statement, and instead recommend using a leading self or :: to disambiguate.

This still sounds a bit fishy, but I think we can at least to try implementing this.
E.g. select any suitable resolution during resolving imports (greedy algorithm), then proceed and perform an extra disambiguation checking post-processing stage after import resolution is complete.

The danger is that such greedy algorithm can select one of multiple possible solutions, and which one chosen exactly may depend on things like item order in a module, or expansion algorithm details.
Right now we are stopping import resolution with an error on any possibility of ambiguity so if resolution succeeds, then it’s the only possible solution.
However, If we continue stopping import resolution on any possibility of ambiguity with “full 1path model”, then I suspect we’ll very rarely be able to make any progress.

EDIT: Perhaps for imports we can consider only candidates “planted in modules” + candidates from “known in before sets” like extern crates passed with --extern, then we can make the ambiguity detection sound.
Basically, we can try to approach “1path” (which seems to be the simplest and least confusing mental model) as closely as technically possible while keeping fundamental properties like uniqueness of the found solution.
I need to think about it.

[internetionals]

Has it already been considered (a quick glance around didn’t turn up anything) to make use extern:: the prefix for importing stuff from external crates?

In that case we have self, crate, super and extern as “special prefixes” and as extern is already a keyword there should be no conflict with the current rules. It’s also very explicit about pulling in an external dependency but not nearly as verbose as the old extern crate ...; use ...;

[petrochenkov]

Has it already been considered (a quick glance around didn’t turn up anything) to make use extern:: the prefix for importing stuff from external crates?

This alternative was actually implemented too for experimentation purposes.

#![feature(extern_in_paths)]

use extern::regex::Regex;

fn main() {
    let r = Regex::new("a+");
}

http://play.rust-lang.org/?gist=6765c5d6f0950616419f6404ce50f277&version=nightly&mode=debug&edition=2015

[internetionals]

Oh great, apparently my google-fu is really lacking at the moment. Definitely have to do some experimentation around these 2018 epoch related changes.

[nikomatsakis]

The danger is that such greedy algorithm can select one of multiple possible solutions, and which one chosen exactly may depend on things like item order in a module, or expansion algorithm details.

Can you elaborate a bit more on this? You talk about “soundness” later but I’m not sure what you are trying to avoid.

One thing I could imagine is that we might “incorrectly” run procedural macros – i.e., because there is an ambiguous path, and one of the options is a procedural macro – so the choices that resolve makes are observable via side effects.

The other thing I could imagine – but I don’t think can occur – is that by making some choice, we would wind up inadvertendly ruling out the other choices. But I think that as long as we fully expand macros and things, if there was ever an ambiguity, we should always find it eventually and error out – is that not true?

[nikomatsakis]

I think we might need stronger ambiguity rules. Something feels fishy to me about making use different from other paths. Imagine we had this setup (and we are using lexical macro resolution), and that we have two external dependencies named foo and bar:

foo::macro1! { ... }
bar::macro2! { ... }

Now, we are in a bit of a bind. If we execute foo::macro1 first, it might generate a module bar – that module then shadows the external crate bar. But if we execute bar::macro2 first, it might generate a module foo which shadows the external crate foo.

Perhaps these are the soundness concerns that @petrochenkov was referring to.

We can sidestep these by making it an error to shadow from any place.

But that does raise concerns when it comes to the prelude, since we want the ability to expand that.

[matklad]

Don’t we have the same problem already? We have shadowing behavior for ::* imports, and there are use self::foo today.

That is, is this something genuinely new, or is just old behavior, present in more cases?

[nikomatsakis]

To be honest I’m not entirely sure =) but I suspect we can have this problem today in some cases.

I think what I am arguing here is that this would be really common: that is, any time that you have two relative macro invocations that come from external crates, you are stuck and cannot proceed – unless I suppose you use an explicit form or pull those paths into use statements, at which point you’re fine, because if executing (say) foo::macro1 created something that shadows bar, it’d be an error anyway. (Note: If we made shadowing a hard error universally, we’d be fine.)

Maybe this is ok – sometimes you get weird errors from the name resolution system saying “you need to disambiguate this path for me to proceed” and then you change to ::foo::macro1 or something. Seems par for the course with things like T: 'a annotations – but it is something we generally try to avoid =)

[nikomatsakis]

(Note that I am talking about the Macros 2.0 future here – #[macro_rules] stuff is still expanded in a weird “pre-name-resolution” pass, as far as I know, that doesn’t really respect lexical name resolution in the usual sense.)

[phaylon]

So, this (and the current) proposal have one big problem for me. I tend to format my code as follows:

use tendril;

pub struct MyStruct {
    value: tendril::StrTendril,
}

I only import namespaces at the top level, making the header of the file a sort of preamble of how it connects with other things.

In the current and proposed schemes however, the use tendril can fall away, since tendril::StrTendril is fully qualified due to tendril being in the preamble.

It would be nice if there were an opt-in lint I could activate that would force me to use ::tendril or use tendril or extern::tendril. Just something so I have a hint where things are coming from, instead of having a plain identifier doing a global lookup.

[gbutler]

Yes, I too am really uncomfortable with the implicitness/magic going on here. The fact that we’re not even sure what kind of ambiguous imports might happen and how the compiler will have to disambiguate or give an error makes me feel that the proposal is lacking.

(Note: If we made shadowing a hard error universally , we’d be fine.)

So, why not this? Why not have absolute paths be unambiguously absolute? That is:

• use extern::<crate name>::<rest of path>
• let x = extern::<crate name>::<rest of path>::fn_returning_something();

I just don’t understand the necessity of having paths that can be interpreted as absolute vs relative depending on the context and what is already in scope. The whole premise sounds fraught with issues.

At some point, an analogy was made to file-system paths. I know of no fs path specification where paths can be interpreted as either absolute or relative depending upon what the current working directory is and what files/sub-directories exist in the current directory.

Could someone ELI5 why we need to have absolute paths not have a pre-fix that makes it 100% clear in all cases that it is an absolute path (without the compiler having to magic its way around analyzing the context). I just don’t understand why that is a useful feature bug.

[gbutler]

This alternative was actually implemented too for experimentation purposes.

What was the outcome of this experiment? Why was it rejected? Why shouldn’t absolute paths be 100% unambiguous?

[matklad]

This looks like a breaking change, since such paths in 2015 are equivalent to the proposed crate:: (they can refer to any kind of item at the root of the crate). Not having breaking changes was a big selling point of the previous proposals.

I think this is sort-of the crucial observation to the current discussion. We definitely rejected stuff before because it changed semantics of existing paths. If we are ready to lift this restriction, then why don’t we settle for what seems to be the simplest system? Namely:

• Category of path (relative vs. absolute) is determined purely syntactically. Absolute paths look like ::crate_name::some::path, where the crate_name for current crate is keyword crate. Relative paths look like some::path, super paths look like super::some::path. self paths like self::some::path are deprecated, they are exactly equivalent some::path.
• There’s no distinction between path in use items and elsewhere.
• Semantics of absolute paths require that the first component is a crate name, and this is a breaking change. Today, ::Foo may refer to the Foo struct in the root module. In the proposal, it is an error and should be spelled as ::crate::Foo. It is more verbose, but with nested imports it shouldn’t be a problem.
• Semantics of relative paths is exactly as it is today.

I think the migration to this system can be done purely syntactically (modulo macros which can inspect the concrete syntax):

1. Do not remove extern crate declarations prior to path migration
2. Change all ::foo to ::crate::foo. Note that ::regex changes to ::crate::regex and still works, because extern crate mounts the name to the root module
3. Change all use foo into use ::crate::foo
4. Drop all self::

I believe that, modulo macros, this transformation is always correct. After that, we can get rid of extern crate on case by case basis: this could break in weird cases like #[cfg(foo)] extern crate x; #[cfg(not(foo))] mod x;.

[steveklabnik]

I know this has sort of been said already, but I care strongly enough that it feels worth repeating. @centril brought it up too.

While the initial modules discussion was very stressful, I really liked where we ended up. Changing everything yet again so close to the deadline feels… not great.

This scheme feels more complex in order to gain a tiny bit of unification. I much prefer what we’ve implemented, personally.