Relative paths in Rust 2018

[gbutler]

One possibly surprising thing that follows from uniform paths is that this works:

use std::collections;
use collections::HashMap;

This makes me uncomfortable. It feels fragile. For example, if I have an external crate called “collections” that has it’s own HashMap module, then, both of these compile, but, mean two different things:

use std::collections;
use collections::HashMap;

and

use collections::HashMap;
use std::collections;

So, when re-factoring (and re-factoring tools like in IDE’s) would need to analyze use statements and such before re-ordering them. This might be unavoidable, but, if we required a specific keyword/pre-fix for external crate usages in all cases, might that prevent this situation entirely? Would the downside of that be worse or better than this sort of ambiguity?

In other words, the above, to get the “collections” crate HashMap would have to be:

use ::collections::HashMap;
use std::collections;

or

use std::collections;
use ::collections::HashMap;

which would both be equivalent to each other, but, would be different from:

use std::collections;
use collections::HashMap;

and

use collections::HashMap;
use std::collections;

would be a compiler error along the lines of, “Unable to resolve collections::HashMap”.

Now, this would break non-use prelude paths like:

let x = std::time::SystemTime.now();

which would have to be:

let x = ::std::time::SystemTime.now();

instead. If that is too much of a backwards compatibility issue, then, what if the rule was relaxed to allow only standard prelude items to be resolved without the leading :: instead of prelude AND external crate items. This would mean your rule:

Starts with an identifier :

• If the identifier is in scope (e.g. declared or use d within the current module), resolve it to the corresponding declaration
• If the identifier is the same as an external crate name, resolve it to that crate
• If the identifier is a name in the prelude, resolve it to the corresponding item

Would instead become:

Starts with an identifier :

• If the identifier is in scope (e.g. declared or use d within the current module), resolve it to the corresponding declaration
• If the identifier is the same as an external crate name, resolve it to that crate
• If the identifier is a name in the prelude, resolve it to the corresponding item

Would that not get rid of the ambiguous cases? Would it be backwards compatible? Is there some big ergonomic issue that this brings up that outweighs the benefits to a lack of ambiguity?

[rpjohnst]

both of these compile, but, mean two different things:

No, this is not what’s being proposed- order does not and will not matter. The proposal would make that situation into a hard error, and you would have to disambiguate your use collection::HashMap statement to either use ::collections::HashMap (for the external crate) or use self::collections::HashMap (for the relative use). This is kind of in between the “oops order matters” and "you always have to use ::" that you describe.

[gbutler]

@rpjohnst So you are saying that this:

Let’s say a path is ambiguous if it starts with a leading identifier, and that identifier could be two or more of: a local declaration, a crate name, or a prelude item.

along with this:

While it may ultimately be possible to apply the same disambiguation order for use , the implementation is much more challenging, and it’s not obvious that it’s desirable. So instead, we can make it a hard error to write an ambiguous use statement, and instead recommend using a leading self or :: to disambiguate.

means this:

This is an unavoidable consequence of having a uniform notion of paths. But I’d propose that we include a warn-by-default style lint, suggesting rewriting the above as:

use std::collections::{self, HashMap};

Doesn’t apply? So, if “collections” were an external crate in the cargo.toml, then, this would be a hard error, but, if no such crate were referenced in the cargo.toml it would be a lint warning?

I guess that’s better, but, it still feels fragile (and difficult for the compiler to get right??? @nikomatsakis ??? )

[ExpHP]

Wait, so this is now full 1path? Wasn’t that previously shot down on principle for making name resolution intractable?

(I mean, even just on its own it needs to be iterated to a fixpoint, but when you bring macros into the mix…)

[josh]

Yes, this is now full 1path. It doesn’t require full iteration to a fixed point, though, precisely because of the restriction on module/crate name collisions. That simplifies name resolution drastically. (We could potentially support shadowing in the future, but that’d make name resolution harder.)

[matklad]

I really like this because this achieves 1path and is very close to what I’ve proposed in The Great Module Adventure Continues, + ambiguity rules !

Technical questions/points:

• Am I correct that foo and self::foo are almost equivalent? self::foo won’t work, but foo work will only if foo is in prelude/extern crate.

• Looks like an alternative formulation is “we add extern crates to prelude, overriding existing prelude items”.

• After this proposal, you’ll almost never write self::, ::foo, which feels like a nice simplifications?

• Behavior with respect to ::* imports should be specified. Currently, they shadow prelude and are shadowed by anything else.

• Ambiguity rules are an interesting solution! I think it even might help with IDE performance, because you’ll be able to stop early during transitive imports resolution. However, we must specify how they work with type/value/macro namespaces? Is it an error to use regex when you have fn regex() in scope as well as an the regex extern crate?

[josh]

Am I correct that foo and self::foo are almost equivalent? self::foo won’t work, but foo work only if foo is in prelude/extern crate.

Right. (Assuming that by your second sentence you mean “the only case where self::foo won’t work but foo will is …”)

Looks like an alternative formulation is “we add extern crates to prelude, overriding existing prelude items”.

That’s exactly how I think of it, yes.

After this proposal, you’ll almost never write self:: , ::foo , which feels like a nice simplifications?

Right.

[SimonSapin]

Starts with :: : the first name in a path is an external crate name.

This looks like a breaking change, since such paths in 2015 are equivalent to the proposed crate:: (they can refer to any kind of item at the root of the crate). Not having breaking changes was a big selling point of the previous proposals.

[josh]

As mentioned elsewhere in the proposal, “We may instead want to preserve the existing behavior of :: from Rust 2015.”. That’s the approach I would favor. In particular, it shouldn’t be particularly hard to have :: follow this resolution algorithm:

• First look at crate::
• Then look at extern crates
• Then look at the prelude

That would allow all of ::top_level_module::foo , ::some_crate::bar , and ::std::baz to work.

[petrochenkov]

This looks like a breaking change, since such paths in 2015 are equivalent to the proposed crate:: (they can refer to any kind of item at the root of the crate). Not having breaking changes was a big selling point of the previous proposals.

Huh? The previous proposal (which is currently implemented) implies a huge breaking change on edition switch (but no breakage without edition switch), look at any “2015 -> 2018” transition PR.
This proposal seems to keep the same approach.

EDIT: ::something already means extern::something in edition 2018 in the current implementation.

[petrochenkov]

Wait, so this is now full 1path? Wasn’t that previously shot down on principle for making name resolution intractable?

Exactly!

This proposal seems to make it tractable by declaring any ambiguity an error:

So instead, we can make it a hard error to write an ambiguous use statement, and instead recommend using a leading self or :: to disambiguate.

This still sounds a bit fishy, but I think we can at least to try implementing this.
E.g. select any suitable resolution during resolving imports (greedy algorithm), then proceed and perform an extra disambiguation checking post-processing stage after import resolution is complete.

The danger is that such greedy algorithm can select one of multiple possible solutions, and which one chosen exactly may depend on things like item order in a module, or expansion algorithm details.
Right now we are stopping import resolution with an error on any possibility of ambiguity so if resolution succeeds, then it’s the only possible solution.
However, If we continue stopping import resolution on any possibility of ambiguity with “full 1path model”, then I suspect we’ll very rarely be able to make any progress.

EDIT: Perhaps for imports we can consider only candidates “planted in modules” + candidates from “known in before sets” like extern crates passed with --extern, then we can make the ambiguity detection sound.
Basically, we can try to approach “1path” (which seems to be the simplest and least confusing mental model) as closely as technically possible while keeping fundamental properties like uniqueness of the found solution.
I need to think about it.

[internetionals]

Has it already been considered (a quick glance around didn’t turn up anything) to make use extern:: the prefix for importing stuff from external crates?

In that case we have self, crate, super and extern as “special prefixes” and as extern is already a keyword there should be no conflict with the current rules. It’s also very explicit about pulling in an external dependency but not nearly as verbose as the old extern crate ...; use ...;

[petrochenkov]

Has it already been considered (a quick glance around didn’t turn up anything) to make use extern:: the prefix for importing stuff from external crates?

This alternative was actually implemented too for experimentation purposes.

#![feature(extern_in_paths)]

use extern::regex::Regex;

fn main() {
    let r = Regex::new("a+");
}

http://play.rust-lang.org/?gist=6765c5d6f0950616419f6404ce50f277&version=nightly&mode=debug&edition=2015

[internetionals]

Oh great, apparently my google-fu is really lacking at the moment. Definitely have to do some experimentation around these 2018 epoch related changes.

[nikomatsakis]

The danger is that such greedy algorithm can select one of multiple possible solutions, and which one chosen exactly may depend on things like item order in a module, or expansion algorithm details.

Can you elaborate a bit more on this? You talk about “soundness” later but I’m not sure what you are trying to avoid.

One thing I could imagine is that we might “incorrectly” run procedural macros – i.e., because there is an ambiguous path, and one of the options is a procedural macro – so the choices that resolve makes are observable via side effects.

The other thing I could imagine – but I don’t think can occur – is that by making some choice, we would wind up inadvertendly ruling out the other choices. But I think that as long as we fully expand macros and things, if there was ever an ambiguity, we should always find it eventually and error out – is that not true?

[nikomatsakis]

I think we might need stronger ambiguity rules. Something feels fishy to me about making use different from other paths. Imagine we had this setup (and we are using lexical macro resolution), and that we have two external dependencies named foo and bar:

foo::macro1! { ... }
bar::macro2! { ... }

Now, we are in a bit of a bind. If we execute foo::macro1 first, it might generate a module bar – that module then shadows the external crate bar. But if we execute bar::macro2 first, it might generate a module foo which shadows the external crate foo.

Perhaps these are the soundness concerns that @petrochenkov was referring to.

We can sidestep these by making it an error to shadow from any place.

But that does raise concerns when it comes to the prelude, since we want the ability to expand that.

[matklad]

Don’t we have the same problem already? We have shadowing behavior for ::* imports, and there are use self::foo today.

That is, is this something genuinely new, or is just old behavior, present in more cases?

[nikomatsakis]

To be honest I’m not entirely sure =) but I suspect we can have this problem today in some cases.

I think what I am arguing here is that this would be really common: that is, any time that you have two relative macro invocations that come from external crates, you are stuck and cannot proceed – unless I suppose you use an explicit form or pull those paths into use statements, at which point you’re fine, because if executing (say) foo::macro1 created something that shadows bar, it’d be an error anyway. (Note: If we made shadowing a hard error universally, we’d be fine.)

Maybe this is ok – sometimes you get weird errors from the name resolution system saying “you need to disambiguate this path for me to proceed” and then you change to ::foo::macro1 or something. Seems par for the course with things like T: 'a annotations – but it is something we generally try to avoid =)

[nikomatsakis]

(Note that I am talking about the Macros 2.0 future here – #[macro_rules] stuff is still expanded in a weird “pre-name-resolution” pass, as far as I know, that doesn’t really respect lexical name resolution in the usual sense.)

[phaylon]

So, this (and the current) proposal have one big problem for me. I tend to format my code as follows:

use tendril;

pub struct MyStruct {
    value: tendril::StrTendril,
}

I only import namespaces at the top level, making the header of the file a sort of preamble of how it connects with other things.

In the current and proposed schemes however, the use tendril can fall away, since tendril::StrTendril is fully qualified due to tendril being in the preamble.

It would be nice if there were an opt-in lint I could activate that would force me to use ::tendril or use tendril or extern::tendril. Just something so I have a hint where things are coming from, instead of having a plain identifier doing a global lookup.