Revisiting modules, take 3


#42

I’m loving use [std]::fmt; and combined with :: for absolute paths everywhere aka relative paths everywhere, doesn’t that completely solve “path confusion” aka “works in main/lib.rs but not elsewhere?”

struct Type;
mod foo {
  use ::Type;  // must use absolute path or super::Type

  fn bar(t: Type) { ... }
}

The more I look at ::, the less ugly it looks and I think after a while it would be “normal.” And it’s a small price to pay for consistent paths in use, export and regular code.


#43

[crate] even seems rather evocative: with square brackets, it looks like a long box, or in other words, a crate. :slight_smile:


#44

I think export would be worth it if leading :: stays as the root of current crate. Then the migration story is very simple: deprecate absolute paths in use that don’t start with :: (using a simple lint), and add export with relative paths.

That way, the semantics of use paths don’t need to change at all – the confusing part just gets phased out over time as people fix the warning, or just run updated cargo fmt on their crates. The cost of two characters per import is well outweighed by the uniformity of paths everywhere.


#45

I also prefer [crate]:: to extern::crate::. It feels like a lighter-weight change to the current system, as it is much less invasive into how paths work. I especially appreciate the symmetry between [some_other_crate]::item and ::some_item_in_this_crate, as the :: syntax is already familiar to C++ programmers.


#46

I also prefer the solution of keeping use and pub use. One of the complaints about the module system is how many different types of statements are involved. For this and for ease of transitioning (including people reading old code), I would prefer to avoid export.

One solution for your second bullet point here is that use statements could simply have an even-more-private default than other items, that doesn’t leave the current module. I don’t think that would be surprising, as people wouldn’t expect or attempt to use non-public uses from other modules- indeed, accidentally being able to do so is the source of the current confusion.

This leaves the first bullet point of forcing people to use self:: for most of their pub uses, but I’m not convinced this is either confusing or hard. The consistency with crate-relative uses should help quite a bit.


#47

I’ve caught up on this thread now, and want to make a few scattered observations. In general I feel like we’re converging on a solid set of problems to focus on and ingredients to use, but haven’t quite found the ideal mix.


About relative paths: there’s a subtlety about use-as-items at play here. Unqualified paths can be understood in two ways:

  • Based on what is in scope, which you might take to be a combination of items used in the module and those defined by the module (including submodules).

  • As a relative path starting from the current module, which means the start of the path must be an item the module defines.

These two views are the same in the current module system, precisely because use declarations actually define items within the module. This is what makes self:: paths in use match the behavior of unqualified paths within the module. Take the following example:

use std::io;
use self::io::Read; // possible because the above is an item

fn mention_write<T>() 
    where T: io::Write {} // possible due to either of the two views.

If use ceases to define an item, we’ll lose this alignment, which is possibly worrisome.

This issue also has implications for the “use-universally” (i.e. “part 2”) proposal, because it’d let you do things like:

from std use io;
use io::Read;

which seems potentially pretty confusing. But if we don’t retain this use-as-item behavior, then there would still be a difference between paths you can write in use and those you can write in module bodies.

I suspect these considerations are at least part of why most languages employ absolute paths for import statements.

I wonder if there’s a way to (1) keep paths for use starting at crate root, (2) keep use as items, but (3) avoid the surprise when certain things work at crate root but not deeper within the crate.


Regarding export. At first, I felt pretty resistent to adding this on top of our existing module system setup. And some others have voiced concerns about it providing a different interpretation of unqualified paths than use statements. But I wanted to point out that it woud have the same interpretation as e.g. paths when writing a type in a function signature within a module. In other words, only use would employ paths starting from crate root, and everything else would be based on what’s in scope.

I worry, though, that export will end up being a point of significant confusion. As others have mentioned, it doesn’t actually guarantee that an item is visible outside of the crate. And it puts us back in a situation where it needs to be used to export certain things – submodules and used items – but not others.


One bit of hesitance about collecting data from existing crates: it seems likely that usage is influenced by our current defaults. I know I tend to avoid using self:: because it just feels weird. That said, some of the numbers on this thread are quite strong :slight_smile:


I’d like to toss in my support for [std] as a way of referencing external crates in paths in general. While I agree with @withoutboats that this isn’t as immediately obvious as the from syntax, I think it’s easy to learn and will be encountered/taught very early on. I think it’s very helpful to have the same unambiguous path syntax work in all locations, and the [std] syntax, like from, helps the crate name stand visibly apart. I think we could also allow omitting a :: when using braces to list multiple items:

use [std] {
    io::{Read, Write},
    collections::HashMap,
}

I also like the combination of this syntax with :: for going to the current crate root; it feels very natural to then say that [std]:: takes you to the root of std.


#48

The example here is interesting, combined with the problem of private uses being visible in submodules. Perhaps use items’ default visibility could be reduced from the current “private” visibility to “visible in the current module; but not submodules or other use statements.” The current visibility could be recovered with pub(self) use.

This could potentially be the only backwards-incompatible change we need, and it shouldn’t have much of an impact. (This would also be something nice to collect data on.) Combined with deprecating extern crate foo; and mod foo; for [crate]:: and file/dep-inclusion-by-use, which should be backwards compatible, this looks to me like a minimal solution to the learnability problems from the original post.

As an aside, one concern about [crate]:: would be disambiguating it from an array expression. The :: should be enough (?), but that does make parsing a little weird.


#49

I don’t think that this improves readability in any way compared to

use [std]::{
    io::{Read, Write},
    collections::HashMap,
}

Just having two kinds of brackets besides each other separated only by whitespace (or nothing at all?) looks weird to me, while [std]:: makes it clear that this is the beginning of a path, as you say yourself:

it feels very natural to then say that [std]:: takes you to the root of std.

— Apart from that, I like the direction that this is going!


#50
use [std]::{
    io::{Read, Write},
    collections::HashMap,
}

I don’t think tree-like imports with nested subpaths should be tied to [crate] in any way.
It’s an orthogonal feature applicable to all imports and useful regardless of the fate of other proposed module reforms.
There’s an RFC issue about it - https://github.com/rust-lang/rfcs/issues/1400.

[std] is already a path to crate root, no need in the weird trailing ::.

use [std]; // Equivalent to `extern crate std;`

Module names don’t make sense in any other contexts beside use and pub(restricted), so there are no ambiguities with arrays/slices in expression, patterns and types.


#51

Oh, I didn’t mean to suggest they would be! I think we should allow nesting generally.


#52

I don’t think anyone is proposing [std]:: on its own- just leaving out what comes after by analogy to writing ::.

Module names are used all the time outside of use and pub(restricted):

use module;
module::function();

It’s also always nice to avoid having the parse tree depend on what kind of item a name is. C has that problem with x * y and C++ further has it with x y();.


#53

I mean, module names as full paths, not path prefixes.
let x = mod_name;, match x { mod_name => ... }, let _: mod_name; are always errors.
So, let x = [ident];, macth x { [ident] => ... }, let _: [ident]; can be unconditionally parsed as arrays/slices and not paths to a crate named ident.


#54

Can we please avoid deprecating mod? It doesn’t just force you to open an often very inconveniently long tree view of the file system, it also comes with disadvantages like not being able to make all contents of a module private except you do a reexport.


#55

Deprecating mod doesn’t have to mean either of those things. Python doesn’t have it, and people can find modules just fine based on their path. And, implicit modules can be private by default- that’s an independent change.

Personally I would prefer modules be private by default, included in the build based on use, and made visible outside via pub use.


#56

I think navigation inside large codebases is not one of the the virtues of Python, while Rust specifically targets large codebases with its features like static typing and the ownership system, telling you many things about a function’s behaviour statically. Getting closer to python should be no goal.

Right now you can see when looking at a mod declaration whether a module is private or not (IMO reexports don’t change privacy of the reexported thing, as much as wrappers around functions don’t change it). If you go with a model where the privacy of a module is the privacy of its most public member, you won’t have this feature any more. You simply won’t know, is this module part of the public API? Is it some util thing?

If you go with a model like you suggest where you force modules to be private, and only allow public modules through doing pub use, you reintroduce pub mod with a new name.


#57

That is hardly because of its module system, which is far less confusing than Rust’s. A Python module’s position in the namespace is the same as its position in the filesystem, so when you see a path you can go straight to its definition.

Yes, that’s the point. One of the confusing parts of the module system is the redundancy between all the various module-related declaration keywords, and how to gain access to a new file in the first place. If we get rid of private mod foo; declarations and replace them with use foo;, that problem goes away.


#58

I think most of the surprise comes from the external crate problem, so most of it will be fixed if you don’t mount external crates at the root, but manage them through [cratename] instead.

Especially, if you disallow stuff like fn mention_write<T>() where T: std::io::Write {} and require it to be written like in submodules to the root, aka with [std]::io::Write, I think most of the confusion will be gone.

AFAIK, currently, the only thing that is technically different in the crate root from other places of the crate is that std or core get auto-mounted at the root. In theory, you can declare extern crate in submodules as well so its the same except for std. However, popular practice is to put the extern crate declarations most times into the root, making the root practically different in this regard as well.

So the only difference you could think that remains where the crate root is different from one of its submodules is that it can use its modules with seemingly absolute paths (no need for :: prefixing) but that property is IMO already the case for any module and its submodules, and there is also no practical difference arising from this; if you want to split up the crate root, maybe you want to split up the first level of modules into a second level as well at a later point.


Revisiting modules -- `[other_crate]::path` syntax
#59

As was discussed above, there’s also the problem that new users might form a model where paths in use behave like paths in expressions, and get confused "why can’t use find my local use". I think we can fix this by making paths in use work like paths in expressions (requiring use ::local::thing;).


#60

This has also been mentioned before, but that may not be the whole/actual problem, for a couple of reasons. Plenty of other languages use absolute paths in imports but relative paths elsewhere, without causing confusion. Instead the problem seems to be that, in Rust, this difference only exists in submodules. Further, a large portion of uses are absolute, so requiring a :: prefix would add a lot of tedium and extra syntax for (given the above) not much gain.

Taking crates out of the root namespace does only partially resolve the difference between the use in the root module and use in submodules. Using submodules is still different between the two. However: this difference is also present in these same other languages. Perhaps we simply need to teach use similarly- by analogy to include paths, $PATH, etc.


#61

Python’s import statement is an interesting example that I think is pretty good, but hard to get to for us.

Imports are from an absolute root, which contains this package under its name:

import serde.Serialize
import thispackage.Item

extern and crate dependencies are distinguished then by you knowing the name of the crate you’re looking at.

Relative imports also work, using a leading period. However, in most modules these would be the equivalent of super::, whereas for __init__.py (the equivalent of mod.rs) they are equivalent to self::. This gives . the very understandable semantics of “from this directory.”:

import .sibling.Item # imports from $PWD/sibling.py

Problems with migrating to this model for us:

  • Our crates can depend on a crate of the same name.
  • Unclear how to move from self and super to a “this directory” signifier.