The Great Module Adventure Continues

To get to the bottom of this syntax discussion, the Lang Team has decided to convene a small group (with representative opinions) to discuss at high bandwidth: myself, @nikomatsakis, @josh, @withoutboats, and @rpjohnst. We’ll post summaries of the discussions here.

Can each of you please add yourself to this Doodle poll and I’ll send an invite?

1 Like

The informal modules WG met today and successfully narrowed down to a single core proposal, with two variants! The lang team and stakeholders on the WG are all on board with this overall direction (the first time we’ve reached total consensus in this group.)

Proposal

The basic proposal is “Java-style imports”:

  • Fully qualified paths begin with one of:
    • a crate name
    • crate (for the current crate)
    • self (for the current module)
    • super (for the parent module)
  • use statements require fully qualified paths (in Rust 2018)
  • All crates provided by Cargo.toml, --extern, or the standard distribution are treated as part of the prelude for the crate.
    • That means that within items you can freely reference e.g. std::mem::transmute without a leading :: or a use std.
    • As with other names in the prelude, these names can be shadowed by local definitions; we will lint such shadowing.
  • In items, you can use a leading :: to signify a fully qualified path, though it’s almost never necessary.

Example

Here’s some code taken from the wild. With this proposal, I was able to remove several of the use statements in favor of direct references.

use crate::{
    reactor::Handle,
    PollEvented,   
};
use std::{
    io::{self, Read, Write},
    net::{self, SocketAddr, Shutdown},
    time::Duration,
}
use bytes::{Buf, BufMut};
use futures::{Future, Poll, Async};
use iovec::IoVec;
use tokio_io::{AsyncRead, AsyncWrite};

pub struct TcpStream {
    // note: reference to `mio` which is in Cargo.toml
    io: PollEvented<mio::net::TcpStream>,
}

// no need to import `fmt`
impl std::fmt::Debug for TcpStream {
    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
        self.io.get_ref().fmt(f)
    }
}

Leading-:: variant

An alternative is to shorten the syntax for crate-local qualified paths from leading crate:: to just leading ::, i.e.:

use ::{
    reactor::Handle,
    PollEvented,   
};
use std::{
    io::{self, Read, Write},
    net::{self, SocketAddr, Shutdown},
    time::Duration,
}
use bytes::{Buf, BufMut};
use futures::{Future, Poll, Async};
use iovec::IoVec;
use tokio_io::{AsyncRead, AsyncWrite};

In this variant, a leading :: signifies “local crate root” rather than “fully qualified path”.

Assessment/rationale

  • Uniform behavior in top-level and children modules. The way that you bring items into scope, no matter where they come from, is the same in all modules of a crate. The “cliff” between top-level and children modules is eliminated: std is available everywhere, and there’s no confusion between absolute and relative paths for use in the top level.
  • Unambiguous use paths. The paths you write in use are always fully-qualified: by looking at the path, with no other information, you know immediately where the “root” is (whether in this crate or external).
  • Partial “1path” property. All paths that work in a use also work in items and have the same meaning (modulo shadowing, which is linted). The reverse is true for all crate-rooted paths, but not for relative paths.
    • This is a very common setup in programming languages, where “import” statement take fully qualified paths, actual code refers to “anything in scope”, and the former is a subset of the latter.
  • Transition. This approach does not involve fallback logic (aside from what we already have with the prelude). Migrating to Rust 2018 will involve adding a leading crate:: to internal paths, dropping leading :: in items, and (optionally) dropping extern crate declarations. All of which will be facilitated by rustfix.
    • Existing code snippets from blog posts and StackOverflow are very likely to work without modification, because most examples import only from external crates, where the syntax remains unchanged.
  • Ergonomics. Referencing items from the current crate in absolute paths is slightly more verbose. Referencing items from external crates in code is slightly less verbose. Fewer use statements are needed overall (since crate names are automatically available).
  • Aesthetics. Neither variant involves adding a new sigil or dramatically altering the way paths look.

Next steps

At this point, the relevant teams feel positive about this proposal in abstract; it’s the first time we’ve reached full agreement across this set of stakeholders.

We would like to implement this proposal (including the variant as an option) and to start gaining experience with it as a community ASAP, together with a couple other Rust 2018 features that haven’t gotten sufficient testing yet. @Manishearth has been working on setting up infrastructure to make it easier to try out Rust 2018 as a whole; expect further announcements here soon!

30 Likes

To me, this is a strong argument against the “Leading-:: variant.”

In the main (“Leading-crate”) variant, then absolute paths like ::std::io basically vanish everywhere, except for rare cases. (I assume the case where it's still necessary is to access an absolute path shadowed by a local item.) But in the Leading-:: variant, these paths are still a common occurrence in imports, and worse, they have a different meaning in imports than in other items (where I assume they would still be needed as fully-qualified paths to shadowed items).

5 Likes

So let me clarify a bit.

First, there is always the option of using use-as to get access to a shadowed item (since the use path is fully qualified). So neither variant actually needs a mechanism to do this at the item level. That means that the leading-crate version could drop leading :: entirely, which would in turn help resolve a number of parsing ambiguities we have or are facing.

For the leading-:: variant, then, the idea is that a leading :: has the same meaning everywhere, which is "path starting from this crate's root". We just don't provide a separate mechanism for writing fully qualified paths in items; you have to do it through a use if you need it.

2 Likes

There are a few other ways too. Presuming you are shadowing the crate pickle:

use pickle as pickle_crate;

or by aliasing in Cargo.toml.

To my eyes, this proposal just feels right. At a glance, I can see what is happening. I’ve read over this proposal a few times, and like that this optimises for code reading.

Will also throw a vote for “crate::” as it seems the simpler of the two.

4 Likes

Sorry to disappoint, but this is not implementable as written.
Existing prelude mechanism doesn’t work like this post assumes it work (imports cannot look into the prelude).
Some other mechanism need to be invented to make it work, and it’s going to look either like the fallback from “Clarify and streamline …” RFC or like extern crate *; in the root from “Automatically Usable External Crates” RFC.

1 Like

Also, this is... bold!
So far the library team was very careful about putting things into the prelude, but here we are opening the floodgate and fill lexical scope (available in all modules) with numerous names at once.

2 Likes

As written here, the first two bullet points mean the prelude doesn’t interact with use statements- Rust 2018 use statements just always treat their first segment as a crate name. The prelude is there to provide the partial 1path property that paths outside of use statements may also begin with a crate name.

It would probably be good to clarify this some more, though- is that what we intend this proposal to mean? It’s basically “flag day” in that use top_level_module will stop working in Rust 2018.

Ah, ok, I looked at the code examples, but didn't notice that "(in Rust 2018)".

It’s basically “flag day”

...

(By “flag day” I’m referring to the term originally described here, as an alternative approach to fallback.)

I like the end goal (except for putting everything into the prelude), but still think it would be nice to provide fallback to cushion the effect of the “f*** day”.

1 Like

This was the intention. My belief is that this is OK, for a few reasons:

  • References to external crates continue to work as is, so code from stackoverflow etc will mostly "just work" (that tends not to have internal module structure)
  • For references to crate-local things, we can give a nice error after the fact
    • "to reference a module in this crate, try use crate::foo"
1 Like

I think that a (deprecated) fallback would be ok too, if we find it works out all right.

Question: does "the standard distribution" mean everything in the sysroot including unstable crates with compiler internals, or a few whitelisted crates, e.g. std and core?

(In the latter case we can even do without language changes and just put the names in libstd and libcore preludes.)

2 Likes

I was imagining just std and core; your prelude idea makes perfect sense.

Late to the party, but for a vaguely similar system to Java’s and the proposal above, see C# using syntax. (global ~= prelude I think). C# namespaces are not directly tied to file structure. IIRC external packages are “global”.

On the proposal itself, here’s a confusing looking little snippet:

impl X {
    fn foo(&self, z: self::Z) -> Self {
        self::bar(self, z)
    }
}

As a beginner it might be misleading that the two selfs are totally unrelated. But it’s probably not that bad since path-self should probably always be followed by ::.

self::foo paths are also fairly uncommon, particularly outside of use statements, so I'm not so worried about that.

(And I don't expect them to become more common with this proposal in particular, unless I'm missing something.)

oh, and it should be automatable with rustfix too

SGTM. I also vaguely prefer the "leading crate" variant. Leading :: is a bit punctuationy, and if we can remove it entirely [from R2018] that’s nice (and potentially fewer parsing headaches is just gravy).

15 Likes