Revisiting Rust’s modules, part 2

I think I figured out how to articulate the biggest thing that’s bugging me about the from syntax.

Right now, the path syntax used in use precisely matches the path syntax used to name a module without using use. That makes it obvious that use of anything other than a top-level module is primarily a way of shortening names. For instance, you can write std::foo::bar, or use std::foo; and foo::bar, or use std::foo::bar; and bar. That seems extremely intuitive and compelling.

If we introduce a new syntax like from std use foo;, or use extern::std::foo;, or similar, then suddenly, the two syntaxes don’t match, breaking that compelling intuitive model.

I don’t have a fundamental objection to the idea of putting external crates and internal modules in different namespaces somehow. I do have a vehement objection to introducing inconsistency between use (or equivalent) and direct usage of a full path. Let’s preserve that parallel and intuition.

Does that seem reasonable?

6 Likes

Everything is great in this version except that files are not automatically part of directory module. This we still have to write all those use statements.

Sometime I write ::std::sync in macros. How will that change?

The only thing I kind-a dislike in Rust modules, is that one has to go to main.rs or lib.rs to see the origin of the crate (external crate). What I want is to be able to have visual differentiation between module-local [1], crate-local [2] and external include [3]. The exact syntax does not really matter, as long as I can quickly tell the origin.

  1. use foo::bar;
  2. use ::foo::bar;
  3. from ext use foo::bar; (or use external foo::bar;)

If something is automagically imported, or you just use use for every import, I have to make multiple go-to-declaration jumps with my IDE and then look at the location of the file, that I landed in. Please, no.

Writing as a guy, who mostly looks at somebody else’s code…

1 Like

I think the proposed from syntax is consistent. If you takeout the from part, use part is same as if using root use from the crate internally.

If I am writing foo crate with bar module. With the new syntax I’ll use ::bar from with in foo crate. External user will use from foo use bar

But you can’t write std::foo::bar, unless you’re in the root module, and that’s precisely the confusion that from/use clears up. Remove std (and other crates) from the namespace of the root module, and that distinction goes away- all modules have to from std use foo; and beginners no longer form an incorrect mental model when working just in lib.rs/main.rs.

How else would you move crates out of the internal module namespace? The other ideas seems to be

  • to prefix all internal paths with crate or :: or something, which IMO puts the extra verbosity in the wrong place, or
  • to put a sigil on all crate names, like @std::foo;, which is IMO too sigil-y (but does let you write it outside of use).

And this is probably slightly controversial, but I appreciate that from/use doesn’t work in non-use paths- it encourages people to use it up front, which cuts down on long paths in code and makes it easier to see which crates are being referenced.

3 Likes

I'd love to see that fixed as part of this proposal; I do think the whole relative/absolute paths issue is a major part of the confusion here.

If there's no longer a way to use a name from an external crate directly without first importing it, that seems like a serious regression. I'd love to have a consistent way to do so, from anywhere in a crate.

2 Likes

I personally don’t share the sentiment, but several people have even expressed rather determined opposition to extern crates bring accessible without upfront declaration.

The way I see it, the only objective drawback of from is inability to import crate root itself.

Subjectively, I also don’t like how it interacts with visibility modifiers. Is it pub from crate use symbol or from crate pub use symbol? Both is ugly. Reversing it to use symbol from crate seems better in that regard.

On your first point - I’ve been writing Rust for two years, and I still get confused about imports being absolute. I cannot tell you how many times I’ve written use submod::Doop; and had to go back and change it to use self::submod::Doop;

1 Like

The issue Aaron pointed out about the syntax interacting with IDEs is correct. The JS syntax is more ackward to get completions for imports for. We ran into this working on TypeScript.

1 Like

I like most of this proposal a lot, much more than the previous one.

With a big but:

Relative paths

I really don’t like that paths are relative by default.

Almost all languages have absolute paths by default, and optional relative paths.

  • Python: from . import y (otherwise absolute, with Python 3)
  • Js: import "./relativeModule, otherwise absolute
  • Go: `import “./x”, otherwise absolute
  • Java: no relative imports at all
  • Haskell: also no relative imports (as far as I can remember)

So this would be a big break from the convention of other langauges.

What I would like to see:

Paths are absolute by default

All external crates also available as a regular path.

  • use std::collections::HashMap
  • type x = serde_json::Value

Referencing the current crate root is done by:

  • use cratename::mymodule
  • use ::mymodule (like it is now)

Relative paths

Since the . used by other languages is way too obtuse, I’d recommend an underscore.

So a leading underscore means a realtive path:

  • use _::submodule

  • type X = _::submodule::SomeType

  • type y = _::super::Whatever

    (just super without the leading underscore is NOT allowed, to make it unambiguous that relative paths start with an underscore)

This alleviates the concerns regarding origin ( crate-internal or external).

Everything is assumed external, unless the path starts with an underscore or with :: / the crate’s name.

It also stops the horrible confusion about paths (::x vs self::x vs x vs …).

3 Likes

So I was thinking about what @theduke is proposing, and it occurs to me that a less radical version may be a simple way to solve the path confusion problem.

So far, every solution to the problem was either about changing the default and dealing with the fallout that ensues, or ignoring the confusion altogether. But the thing is, we don’t actually need a default. We can already write use ::absolute::path; as well as use self::relative::path;, so what we should do is disallow implicit paths in use altogether.

use absolute::path; in current code would become a lint, easily solved by two extra characters, and all proper paths inside use would start with one of ::, self::, super::. Anything outside use would remain unaffected. Poof, path confusion gone. Any confusing path would invoke the lint, telling the programmer to make it explicit.

I admit this solution is more suitable for the directories-as-modules proposal, but it may actually be much more digestible than trying to find a way to flip the default.

4 Likes

Sigh, I wanted to comment on the whole post today, but didn’t have enough time again.

Regarding relative/absolute paths in imports, the problem may be potentially solved by importing from both modules. More precisely,

use a::b::c;

can be desugared into two “fused” imports

use ::a::b::c;
use self::a::b::c;

By “fused” I mean the same thing that is currently done by desugaring

use a::b::c;

into

use a::b::c::{type};
use a::b::c::{value};
use a::b::c::{macro};

I.e. the error is reported only if all imported entities are non-existent or private, but individual imports can silently fail.
I suspect this should be simpler to integrate into the current algorithm than similar alternatives like fall-back from global paths to local paths.
If this behavior is undesirable, the import can always be disambiguated by adding :: or self::.

This of course needs implementation experience, crater run, etc, but it may be a viable solution.

3 Likes

I guess it’s too little too late, but what I liked best about the “directories-as-modules” proposal was the way it would have allowed implementations to be be organized among as many or as few files as needed, without interfering with the organization of the interface across modules.

It’s like walking with a stone in my shoe, every time I have to manually specify what goes in a public API by writing

pub use module_created_for_organization_and_full_of_private_implementation::ApiElement

whereas (if I read the original proposal correctly) I could just indicate public visibility right on the appropriate item by writing

pub ApiElement { /* blah blah blah /* }
2 Likes

In principle, use crate::a::b::c; can be thrown into the mix too, lol (not sure if that's a good idea or not).
I don't think all the names collected this way will conflict often.

EDIT: But in this case for every import (!) we'll have to go to the filesystem and search for matching crates in library search directories (unless use crate::xxx; searches only for crates passed with --export and can't import good old std and friends from the sysroot).
EDIT2: But the search can be performed once on the first import and results can be cached though.

Yes, but only because this doesn't work, and I imagined it would:

extern crate foo;
use foo;

or this:

mod foo;
use foo;

I've made that mistake a lot. That's because my mental model was that the first one declares the thing exists in the project (controls availability, like deps in Cargo.toml, or -lfoo linker flag in C) and the second brings it to the namespace (controls symbols, like #include in C).

So what confused me that use and mod/extern are not orthogonal, and functionality of the latter partially overlaps and conflicts with the former. That's the only odd thing here.

I'm fine with crates and modules living in the same namespace. I know crate names. I don't want to create modules with same name as an external crate (now that would be confusing, even if Rust could handle it).

2 Likes

I am not 100% sure what you find odd about getting an error with mod foo; use foo;. I see two possible things:

  • Paths in uses are always absolute, and hence you would have to do use self::foo (outside of the root module)
    • This, of course, is in part what this proposal is all about.
  • You can't import something into a namespace when you have another declaration
    • This is actually mildly surprising to me, since I thought we permitted you to have multiple imports of the same name, as long as they all resolved to the same item. But it doesn't seem to be the case, and I admit I sort of forget the final details of the name resolution algorithm that we ended up with.

But let's go back to the premise: I've never noticed this as an error, because it never occurred to me to and use something which I've already declared. For example, I would never expect to write code like this (which also gets an error, for the same reasons):

struct Foo;
use Foo;

So, if I am understanding correctly, that suggests that you view modules and crates as being different than other items in terms of their visibility. I think you are not alone in this! The current module system, however, draws no distinction.

This is in part why we've been pushing on removing the "declarations" of creates and modules and instead just focusing on the use part. Those declarations are fairly unique to Rust, and -- for a certain set of people -- they seem to be a constant source of confusion and misplaced expectations. For others (including myself), they feel quite natural -- but I also feel that they can be easily forgotten, so I wouldn't sad if we find a less redundant way (e.g. the file system).

(I am, however, quite wary of making the rule of what code exists be based on which modules is imported, for the reasons we've previously enumerated, as well as the complexity of implementations.)

1 Like

That's an interesting idea that had never occurred to me...I sort of like it, in that it will force the user to clarify in the case of ambiguity.

Yes. I think somebody's comment that mod is like fn was the key for me to make the sense of the whole system. In a way it's very logical and I appreciate how it makes all items uniform. It's just no other languages does it like this.

Originally in my mind it was something roughly like:

extern crate foo; === gcc -lfoo
use foo; === #include "foo.h"

so it was bizarre to get an error "can't #include "foo", because -lfoo is already using it"

In most languages modules are imperative "go and find this thing and make it usable here" (Ruby, PHP and JS literally execute code synchronously that does that).

In Rust use is just "make it usable here", but if you treat it as the imperative loading command, it becomes an absurd "go and find this thing and make it usable here, …but only if it's been declared before, but just not if it was declared in this file"

I’ll amend my previous post to add a second minor gripe about the current module system: I’ve had the same confusion as @kornel in thinking that both extern crate foo and use foo would be required to use an item from an external crate. Only now, reading the last few comments, do I finally understand why it’s an error to have both. It has never been clear to me that extern crate foo was anything more than a declaration of linkage. I haven’t had the same confusion with mod and use because the former is only ever used for declaring modules and the latter is only ever used for importing items.

The only real reason I can think of that modules need to be explicitly declared in Rust, whereas many other popular languages infer module structure based on the file system, is that you can have modules of different visibility. If it was decided that in a new module system, modules were private unless the first line of the file was pub mod; that would probably work well enough. I still don’t see that as being a better system than what we have now, especially if it required a language checkpoint that otherwise wouldn’t be needed.

1 Like