Revisiting Rust’s modules, part 2

[dpc]

Clarifing questions: If we have a syntax to import externally, will we be able to use an imported name in other places/modules?

Will that work:

use ::somecrate; // or use extern somecrate or whatever, just "from crate namespace"
use somecrate::{symbol_a, symbol_b};

?

If so, will I be able to use somecrate (as opposed to ::somecrate/extern somecrate in submodules)?

[ahmedcharles]

I think this proposal is well reasoned and more approachable than the previous one.

What I like:

1. Relative use statements. I think this is a huge problem with understanding Rust currently and I’d prefer it be considered for an RFC independently of any other features listed in the proposal.
2. I mostly like from / use. I’d like for it to support from foo use self; which would allow it to mean exactly what extern crate foo; means currently. Though, hopefully, this would replace the current extern crate proposal, entirely.

What I don’t like:

1. use potentially meaning something other than binding a new name to an existing item.
2. Using the filesystem as the sole source of which files get built. I’m actually not sure why this is a feature of every proposal so far.

I’d like to see:

1. An independent RFC for relative use statements, with no other added features.
2. An RFC for from / use, which harmonizes with 1 and deprecates extern crate.
3. An inline mod RFC to reduce pub use boilerplate.

Then let those bake and revisit how people view the module system after a time.

I’m curious, how controversial would those three RFC’s be, independently, as targeted improvements?

[rpjohnst]

I like this! I especially appreciate the details under the “knobs” sections.

Determining module structure from the file system is fine with me, even making modules themselves pub(crate). The visibility of modules themselves doesn’t seem to add much, and this neatly sidesteps the issue of specifying it without mod declarations. (I had suggested keeping pub mod declarations to specify visibility in the last thread but this feels nicer.)

I will admit the git stash scenario could become an issue. Along the same lines, it may make external build system integration harder- one of the pitfalls of Makefiles that determine which files to include automatically is that \a file deletion doesn’t trigger a rebuild- explicitly listing source files in some way fixes that.

---

I’m also relatively positive on from/import. I like that it keeps the same ordering while making it obvious that something is from another crate. It also solves the difference between the root module and submodules by removing crates from the root scope. (For reference, re @nikomatsakis’s coment, I always tend to go straight for use std::cmp; rather than trying to write ::std::cmp::foo.)

---

I am still not happy about relative paths in use statements, though. I use a lot of absolute paths to refer to things in the same crate. Perhaps I’m weird- this should be possible to collect some data on? I also suspect that the confusion about differences in use vs paths will disappear given the other changes- it’s quite common in other languages (Python, Java, C#, Javascript, etc.).

Perhaps it would be helpful to go in the other direction? Without mod declarations, submodules don’t need to be in scope just by existing. So maybe we should change everything to absolute paths, modulo any use statements. This seems to have precedent in other languages as well.

---

I am also still concerned about backwards compatibility. I’d prefer a bit more caution before doing anything that would require a new checkpoint. The more we can make work on current Rust the easier the transition will be if we need one.

[rpjohnst]

I find browsing Rust projects to be annoying due to the large number of subdirectories under src/ and the large number of files named “mod.rs”. I also think the current situation where we have to move x.rs to x/mod.rs when we add submodule x::y in x/y.rs to be unfortunate; I think we should be able to keep x.rs as-is while adding x/y.rs.

I agree on the mod.rs naming- that’s annoyed me in several contexts (other languages, etc.) But I also like having all the files of a module in the same directory. Could we perhaps allow the “main” module of a directory to keep its name (i.e. x/x.rs), or would this be too confusing? It makes the transition easier and makes it easier to tell what file you’re in just by its name.

[sanxiyn]

I am strongly against relative use. I think “paths are absolute in use, otherwise relative” is simple enough, and we should solve confusion by emphasizing this in the documentation.

I think Python PEP 328 is relevant here, in that Python switched from relative import to absolute import in Python 3.

[djc]

I like this proposal a lot! It focuses on the right problems, and makes great headway in solving them. I do also have some suggestions:

• It does feel like the from .. use form is relatively verbose (and requires a new keyword). Can’t we just reuse the leading :: notation to start meaning the global scope? So from petgraph use prelude::* becomes use ::petgraph::prelude::*. This seems more similar to what we have to do, does away with the extra keyword and verbosity, but maintains the invariant that useing items from other crates should look different from useing items from the same crate. useing items on a path from the root of the current crate could then just be ::<crate>::.

IMO this is nice in part because I think it mimics how actual code references symbols. Thinking about it more, use statement symbol resolution must be as similar as possible to how symbols get resolved when you’re just writing code; I think that explains why absolute use feels so jarring today: while I can simplify reference the State enum written above this function by calling it State, if I want to use something from it I suddenly have to go through the absolute path from the crate root. Since ::std::cmp::min already works in code today (IIRC), it makes a lot of sense to reuse that syntax for use.

• I don’t think explicit pub should mean pub(crate) in some contexts; that seems like it’s going to cause more confusion. On the other hand, implicit visibility can vary, so making modules crate-visibile by default makes a lot of sense to me.

I didn’t see anything in the proposals to address the biggest problem I’ve encountered with the module system: If some module a does impl b::T for S, and I want to call a method on a::S declared in b::T, I have to use b::T, even if I never actually reference the name b::T. I find this to be a much more significant ergonomic issue than what’s been discussed so far, because it isn’t just a stumbling block for learning Rust, but rather a persistent problem. (See Inconvenience of using functions defined in traits, an ugly workaround, and proposed solution for previous discussion of this issue.)

While I agree that the ergonomics on that are suboptimal, I guess this is largely orthogonal to the other problems discussed in these module system threads.

Could we perhaps allow the “main” module of a directory to keep its name (i.e. x/x.rs), or would this be too confusing?

I like that proposal, though it does introduce a paper cut of having to rename both file and directory at the same time, otherwise leading to possibly weird failure modes.

I think Python PEP 328 is relevant here, in that Python swithced from relative import to absolute import in Python 3.

Python doesn’t have multiple modules in a single file, though, and symbols defined in an outer scope are typically visible in all the inner scopes by default. Also, in Python the absolute imports are more explicit, in the sense that they require from . import foo rather than Rust’s use foo;. I guess I have to agree that making the “absolute” aspect of use statements more explicit is also a viable direction!

[sanxiyn]

Also, in Python the absolute imports are more explicit, in the sense that they require from . import foo rather than Rust’s use foo;.

No, in Python 3, default import foo is absolute, from . import foo is relative. Python 3 is exactly like Rust in that paths are absolute in import and relative everywhere else.

[dan_t]

Isn’t the correct transformation of:

extern crate petgraph; use petgraph::prelude::*;

this:

from petgraph use petgraph::prelude::*;

Or does the proposal expect that crate and root module names match?

I don’t know how I feel about the even stronger destinction of intern and external modules in this proposal.

But I’m not a big fan of the ‘::’ prefix for accessing the crate root, because for bigger crates you might see a lot of them.

[jimmycuadra]

Perhaps it’s just hard for me to remember exactly what it was like learning Rust’s module system, but most of the proposal doesn’t seem like it’s any easier than what we have now. It’s still a bunch of stuff to remember, except now you’ll have to remember two systems (with the original being less and less common as time passes after the checkpoint that would enable the new version.)

I like this proposal better than the first one, only in that it doesn’t try to encode module privacy into the filesystem with that nasty leading underscore thing. Personally, there is one and only one annoyance I have with the current module system, which is that items in the root of a module with children are in path/to/foo/mod.rs instead of path/to/foo.rs. This is a minor quibble, and is only really annoying when expanding what was once a single module into a module with children, at which point you have to move foo.rs to foo/mod.rs, as @briansmith mentioned. This occurs rarely enough that it’s just an annoyance—if it was never changed, I wouldn’t be unhappy.

The things proposed both here and in the previous post do absolutely nothing to improve the module experience for me today:

1. Once you learn the rule that use statements work with absolute paths and paths are otherwise relative, imports are not a problem. I agree that this was a stumbling block when first learning the module system, but I think it was a documentation problem. I don’t remember it being mentioned in the docs when I was first learning, and certainly not emphasized. I think good documentation is sufficient in learning that rule.

2. I don’t need or even want a syntax that distinguishes local modules from external crates. I don’t understand the motivation for this at all. Do you really not know the names of your own dependencies to the point that you need a special syntax for it? Even with this special syntax, it wouldn’t prevent the desire (at least for me) to separate my imports into three sections (stdlib, external crate, local), so having a separate syntax for one of these three sections buys me nothing.

3. Writing mod foo; is not hard or confusing. Implicitly promoting the existence of a file to a pub(crate) module breaks module encapsulation with no recourse, and for what? This is strictly worse than what we have now for me.

In the end, any sort of breaking change to the module system, which would require the use of epochs/checkpoints, had better be a massive improvement for everyone. I think any troubles with the module system as it is today should be approached with pedagogy, and that the cost of any sort of breaking change is not worth it for this part of Rust.

[arielb1]

Sure. I don’t think writing mod declarations is an annoyance when you’re used to it, and I think the main thing confusing new users is that there are so many ways to do things in the root module, that don’t work when you move out.

I think relative use by default might also add to the “things that work in the root module” problem - “old-style” imports (use within::krate::path;) work in the root module, and not otherwise (you have to write use ::within_krate_path or w/e).

[dobkeratops]

I liked what I was reading in the previous proposal about inferring structure from the directory structure. Fixing stale file hazards other ways would be preferable, I want a clean directory structure for recursive grep anyway.

i do like the ‘from use’ use.

I’ll repeat my idea about ‘automatic promotion of key items’, facades often involve many files in a directory, each one focussing on a single item whose name matches the filename. vec::Vec, option::Option, window::Window etc. a way of promoting a key item one level up might streamline a lot of the need for specific using? … and if the name had to match the filename it would eliminate the possibility of clash

pub use solve::Solver; // note use of relative path

… just call it solver.rs and automatically give this_module::Solver.

[arielb1]

Are the single-item reexports for facades actually annoying people? They don’t even involve “type tetris”. I don’t think we should add more edge cases to the module system - if we want to overhaul it to make it “directory per module” C#-style that’s one option, and if we want it to stay as “file per module” that’s another option, but “sometimes file per module, sometimes directory per module” would just be confusing.

Plus, vec and option actually expose other types, like vec::IntoIter and vec::Drain (and no, I don’t think typeof(Vec::into_iter)::Output is an acceptable solution).

[EpicatSupercell]

Knob: include on use

Sound good, works great with proposals to make the compiler pull-based.

sometimes modules contain nothing but impl blocks, in which case they are not naturally referenced elsewhere

Can they really? When using impl Rust only allows it if you either declare the trait or struct in that same module. And unless you reference that trait/struct, that impl will not have any effect. So having an important file not being referenced should be imposs…

Because impl blocks don’t have to be in the same file as the type they are for, this is very possible (the coherence module in chalk is actually an example of this).

That is, it actually looks like basically like this:

use ir::Program;

impl Program {
    pub(crate) fn check_coherence(&mut self) -> Result<()> { ... }
}

… That’s allowed?! WTF? That’s completely breaking the consistency with impl Trait for Struct. Who thought it’s a good idea to allow this? If my assumption is correct and it’s a solution for impl u8 and the rest of the build in types, then it’s the wrong workaround.

This probably should be changed, and I would write an RFC if I had the time, but I already have another RFC I pay too little love for :(. Also it would meet lots of resistance from the same people who hate the current limitation on impl Trait for Struct, and I’m not a fan of internet arguments, so I’m staying out this time.

But if it would be changed/fixed, include on use could be both convenient to use, allow .rs files to be in a folder without being referenced (useful with multiple build targets), and work in the spirit of a pull-based compiler (which has some basic talk going on but is really far in the future, if ever).

[dobkeratops]

Are the single-item reexports for facades actually annoying people?

if you could automate the most common use case, that would be a win

Plus, vec and option actually expose other types, like vec::IntoIter and vec::Drain (and no, I don’t think typeof(Vec::into_iter)::Output is an acceptable solution).

r.e. Drain, IntoIter - these are temporaries returned by methods of ‘Vec’, as such you should not need to refer to them manually; inference should get them

Drain - A draining iterator for Vec. “This struct is created by the drain method on Vec.”

Struct std::vec::IntoIter1.0.0 [−] [src]

pub struct IntoIter { /* fields omitted */ } [−] An iterator that moves out of a vector.

This struct is created by the into_iter method on Vec (provided by the IntoIterator trait).

What I’m seeing here is use cases that remind me of what we can do with nested types in C++. Once you get the ‘main one’, the ‘internals’ follow naturally from context (you don’t have to refer to them individually)

[arielb1]

r.e. Drain, IntoIter - these are temporaries returned by methods of ‘Vec’, as such you should not need to refer to them manually; inference should get them

Except when you want to store them in a struct.

[dobkeratops]

“Except when you want to store them in a struct”

well in that case go and use them, write the whole facade , whatever.

Or better still, ask for them to unleash the full power of the type inference engine. (“this value in this struct is inferred from it’s creation via acquisition by .drain() etc”).

But I think those things you point at are more like ‘expression templates’ that you would very rarely cache manually. They’re temporaries that just flow through expressions.

Most of the time using these you wouldn’t even know what they’re called; you just see logical behaviour from methods.

[dobkeratops]

Plus, vec and option actually expose other types, like vec::IntoIter and vec::Drain (and no, I don’t think typeof(Vec::into_iter)::Output is an acceptable solution).

Also, surely in the context I’m talking about, you wouldn’t want to promote those into the higher namespace. you use ‘Vec’ all the time, but ‘Drain’, ‘IntoIter’ might clash with other collections that also have a ‘Drain’,‘IntoIter’. in C++ use you’d really be thinking about something like Vec<T>, Vec<T>::Drain, Vec<T>::IntoIter as equivalents (if you built a c++11 library workalike)

In rust we just happen to have a seperation between module and type, but I really see ‘vec::’ as entering on Vec. the whole pattern is so similar.

I think they’re even recovering the ‘privaleged acess’ aspect that you’d achieve with nested classes. (things in the same module can see each other details?)

I don’t think the way C++ works is perfect (I have ran into frustration with asymmetry between namespaces and class internals) but we have the opportunity here refining a newish module system to achieve full symmetry or devise other means of achieving the same result.

I think in C++ all I really wanted was to extend what namespaces and classes can both do to the extent that they really are the same (e.g. extending classes elsewhere… extention methods, and ‘stuff inside a namespace’ is more like class statics , etc)

[DanielFath]

@aturon Overall I like this proposal better than previous one. However I have a question:

Is it possible to define submodules? Meaning stuff like:

File: model.rs

  fn  foo(x: i32) -> bool {
  ...
  }

  mod test {
       #[test]
       fn test_foo() {...}
  }

If it is not allowed, how will achieve similar effects?

[le-jzr]

Putting aside the rest of the proposal, I actually rather like from-use. Much more so than the variants with use extern or use crate::crate_name. from-use feels much less like an overloaded hack. Additionally, this part of the proposal can be hashed out and RFCd independently from the rest, since from-use explicitly states path root, regardless of the default.

OTOH, @crate_name is getting less ugly the more I think about it.

[phaylon]

I can see @ being an issue with match, since

match foo { a@b::C => {} }

and

match foo { @b::C => {} }

would be completely unrelated.