The Great Module Adventure Continues

Originally I though that we can allow relative paths there as well, which should give us 1-path.

That is, I think the following could work

/*
Something like

extern crate future;

is injected by the prelude
*/

use future::Future; // using the future crate, "absolute" path
use my_futures::NiceFuture; // importing a name from a child module, relative path

mod my_future;

fn foo() {
    let f: futures::Future /* "absolute" path */ = my_future::MyFuture::new() /* relative path */ ;
}

One point there is that, while Java doesn’t allow relative paths in import statements, that’s possibly because they wouldn’t even make sense because Java packages don’t nest. They do have hierarchical-looking names, but that’s just a convention- import a.* only imports classes from package a, not other packages. a.b isn’t actually part of a.

I suspect this also has a large impact on the level of confusion, and probably in a good way. It translates to Rust as an inability to write mod foo; fn f() { foo::bar() }, which might lessen the instinct to write relative paths in use statements altogether.

I don’t really see a way to get there backwards compatibly, though, and it would probably be rather annoying with current idiomatic module structures. Just an aspect of Java’s design that makes for an interesting comparison.

5 Likes

What does it mean "all extern crates"? This is an open set.

Thus any reasonable system should work with extern crates in "on demand" way - if we have a name foo in the source code we 1) search this name in the source code somehow (that may include compiler's commandline) and if that search failed, then 2) we go into filesystem and search for a crate file named foo.rlib or something.

Btw, if all relative paths can fallback to extern crates, then we will have to search filesystem on every new name FIX: on every new unresolved name, that's better.

Also, scope-relative paths (this includes prelude and primitive types like u8, but also e.g. local variables) and module-relative paths (to the current module, or some other module) are very different things.

So far scope relative resolution didn't interact with use resolution in any way, use is module-relative "by definition" so far and only names "rooted" in some module may interact with import resolution.

(To clarify, prelude names are *not* rooted in every module and thus can't be used, so we can't reuse this existing mechanism.)

Ah, I've completely forgotten that rustc has a "search path" concept :angry: ! Is it really required for anything besides the sysroot stuff though :slight_smile: ?

Can we inject only crates, passed via compiler flags, the standard library facade crates and remove the search path concept from the public api of rustc?

Certainly! This is simple and this is what Automatically Usable External Crates by cramertj · Pull Request #2088 · rust-lang/rfcs · GitHub did.
The drawback is that it creates a mismatch between crates passed via --extern and (at least) crates from the standard distribution (including std and core).

1 Like

This sounds super bad, how do I import things with a relative path with that?

With self like today. I don't believe @aturon intended to exclude that. I certainly didn't.

1 Like

This also interacts with people not using cargo... I'm not really sure about that side of the story. (That is, how onerous is it to expect them to supply crates by name on the command line?)

1 Like

No more onerous than building without cargo in the first place, I think.

4 Likes

So I was talking to @jseyfried today about fallback, because indeed it seems like fallback is one of the "lynchpins" about which this decision rests. It's worth highlighting that there are two kinds of fallback. Let's say we're resolving ::foo::bar in the new epoch:

  • "Fallback to new semantics": If there is a plausible foo resolution in the root module, use that but issue a warning/error. If not, look for a crate.
  • "Fallback to old semantics": If there is a foo crate available, use that. Otherwise, check in the root module for something. If found, issue a warning/error.

The second version is quite clean: the set of available crates is fixed, either by the Cargo.toml or the file system. This makes the fallback simple to implement, because the first test can be done anytime.

However, simply implementing "fallback to new semantics" is not compatible with the "hard constraints" section of the epoch RFC, which states:

There are only two things a new epoch can do that a normal release cannot:

  • Change an existing deprecation into a hard error
    • This option is only available when the deprecation is expected to hit a relatively small percentage of code.
  • Change an existing deprecation to deny by default, and leverage the corresponding lint setting to produce error messages as if the feature were removed entirely.

There might though be some way to make a variant that complies, wherein pre-epoch we resolve through the root module but also check for an available crate. If a crate is available, but we found something in the root module that is not the corresponding extern crate, then we report a warning. This means we warn only if the path would change semantics.

In the newer epoch, then, we would resolve to the crate, but make it an error if there is any item in the root module (i.e., any plausible resolution) and there is a crate by the same name, unless that root item is an extern crate. This avoids some of the concerns that @josh had about ambiguity, since there is only one possible meaning of ::foo. It does mean that -- at least initially -- you cannot have a root module named foo and an external crate named foo.

I don't know though that this is strictly compatible. I think there could be e.g. a macro expansion in the root module that used to resolve to something from within the crate but which now resolves to something from an external crate. In the older semantics, that macro might have generated items at the root level which would have shadowed an external crate -- but now that the macro is resolving via an external crate, it does not. (But, I guess for that to happen, the path to the macro itself must be ambiguous in the new semantics, so maybe there is an induction argument here?)

Sorry if I'm retreading ground here. It's hard to remember all the territory we covered. Maybe we need to make a kind of "fallback summary" post as well, covering the various ways we can do fallback, and the conditions to be concerned about? (I remember @rpjohnst in particular making a comment earlier referencing the detailed examination of fallback cases from the original RFC thread.)

1 Like

This alternative is bad because it steps into build system territory and makes untrackable changes somewhere in the filesystem to be able to change results of name resolution (https://github.com/rust-lang/rfcs/pull/2126#issuecomment-328079126).

1 Like

There is absolutely nothing about a fallback system that can be “clean”. The whole “let’s change the module system” is supposed to simplify things, having a fallback to some different set of rules is the exact opposite of this.

1 Like

Wait, if 'old semantics' means 'current semantics' then I think I'm missing something. If I have the following setup right now:

mod cc {
    pub extern crate cc;
    //  ^^^^^^^^^^^^^^^
    // Needs to be here, not at root, otherwise we get 'cc defined twice'
    // errors.
}

mod foo {
    use ::cc::Build;
}

I won't see a successful lookup for Build in the cc crate - instead I get an error that there's no item named Build in the cc mod. Where would I see an absolute ::foo::bar path resolve to an extern crate instead of a root module in current Rust?

Also, testing that snippet above is how I learned that extern crate declarations need privacy annotations, but I haven't seen any discussion about how that interacts with the "insert all extern crates into the prelude or at the path root" ideas. Are we mostly just dropping it?

Yeah, good point. To avoid touching the filesystem, we could limit it to cases where the crates are supplied on rustc command line (the "normal case") and expect extern crate for the "search directories" case.

(Ok, I've definitely got to make a kind of summary issue tracking the pros/cons of various fallback options -- will tackle later.)

1 Like

Just to clarify, the fallback system exists to ease the transition to the new system. In the new epoch, it would (by default, at least) only give errors to deprecated usage.

The new system would be either leading-:: or leading-crate, meaning that absolute paths have the form ::<crate>::foo::bar always (and you do ::crate::foo::bar to select from the current crate).

That said, a downside of the system I described is that it would affect you even if you were just using the new system, because you would be forbidden from having a module in the current crate that has the same name as an extern crate, even though -- using the new-style paths -- there is no ambiguity there.

Anyway, I gotta run, so no more comments from me for now -- this is the hazard of leaving comments on the weekends!

I think this is the correct behavior regardless of what syntax is chosen. I'm surprised the RFC didn't specify this; I don't think it would be good behavior to scan the library path for a name unless the code specifically says to do so somehow (e.g. with an extern crate declaration).

I am in general excited about something along the lines of this 'Java-style' approach to the syntax.

@matklad’s idea to always put external crates in scope seems great, but shadowing seems problematic. If in the new epoch we make it an error for any type name or module name to conflict with an external crate name and we’re ok with stabilizing this only in the next epoch, then I think it would get us 1path without fallback. Edit: Actually migrating to something that allows relative paths in use seems difficult, so probably no 1path.

Name conflicts caused by introducing an external crate sound rare but annoying, we would need an ergonomic way of renaming external crates, for example in Cargo.toml itself since it would the canonical source of external crates.

This is already available in nightly Cargo: https://github.com/rust-lang/cargo/pull/4953

1 Like

Actually I’m now satisfied simply with what’s written in the RFC. Transposing paths to and from use isn’t that big of an use case considering we’ll eventually have the IDE writes use statements for us.

Regarding fallback, we can deprecate name conflicts between crates and root-level types and modules so there won’t be anything to fallback to. If we measure that this has a low impact we make it a hard error in the next epoch, we may then stabilize the RFC without fallback.