Revisiting Rust's modules

[est31]

The IDE argument works both ways. You can also say that and IDE can just automatically add any mod foo; and extern crate; declarations.

[turnage]

That rust’s modules are limited to files that have to be explicitly referenced (i.e. the directory doesn’t collectively define the module as in this proposal) is tedious when writing but so so pleasant when reading. I know exactly where to look to find where something is defined.

We can see the alternative in Go for example and I think it’s a strictly inferior experience having to rely on grep to determine what file defines what.

[kornel]

While the inline mod proposal solves the problems of re-exports, it adds yet another feature to the module system, and removes nothing, so it’s not an improvement on the learnability front.

[quodlibetor]

I like this proposal. I dislike the automatic includes of sibling modules, but, how about something like use self:: being required to actually use a name from a sibling file?

given:

src/
  mything/
    a.rs
    b.rs

with:

// file a.rs
pub struct A;
struct Aprime;

// file b
use self::A; // required to be able to use A in this scope
use self::Aprime; // error

[dpc]

Just to leave some opinions:

• It is generally useful to explore the space, possibilities and have a discussion around it
• “Rust way”, IMO, is to prefer generally reliability over convenience; saving some keystrokes here and there is a minor gain
• I don’t think public API not corresponding to a directory structure is a bad thing. When investigating someone’s code, I don’t browse files, I just jump in the code with Vim using racer/rusty-tags plugins, and I expect the directory structure organization to have a different meaning than API origanization (which I view in a Web Browser anyway); as an author I consider it a valuable freedom to express my code with a directory structure that fits my needs, while exposing API that makes sense for the user
• I have a bad experiences with languages that implicitily pull in all the files in a dir to be a part of a module (leftover files, stale files left after autogeneration because building system is imperfect etc.) ; I prefer explicitly list them and have a confidence is that what I get is what I intended

[withoutboats]

@dhardy

What? You know git (probably all VCSs) track files, not directories, right? Git won’t delete an unknown file in a tracked subdirectory (for good reasons), but you definitely don’t want files outside version control affecting your build.

How is this not already the case?

[withoutboats]

@lambda

I also feel like this is optimizing for the more complex cases, at the expense of simpler cases. In particular, for a small project that I want to split into a couple of small modules

But why do you need them to be modules, if you can split them into separate files & privacy scopes without making them modules?

[dobkeratops]

"saving some keystrokes here and there is a minor gain"

it's not just keystrokes, its the number of mental steps in figuring out how to set it up - the learning curve. how similar it is to other languages that you already have to keep in your head (c++ and whatever else)

I have a bad experiences with languages that implicitily pull in all the files in a dir to be a part of a module (leftover files, stale files left after autogeneration because building system is imperfect etc.)

IMO given the number of other tools that can use a directory structure out of the box (recursive grep, 'git' telling you whats unversioned, etc..), I'd guess it's worth putting up with /or fixing those hazards (i.e. clean up the stale files, etc).

My main means of navigating Rust is recursive grep (it's great that it's is so grep-able), so I want the directory structure to be clean.

[rpjohnst]

Throwing in my support for explicit mod declarations- their explicitness is my absolute favorite thing about the module system and I strongly oppose getting rid of them. I also strongly disagree with the article’s assessment that “the file system arrangement itself is a perfectly “explicit” way of providing information.” Explicit declarations provide both detail (particularly intent; also an easy solution to the sibling file question) and flexibility (single file modules vs directory modules vs mod { .. }; no accidental files) that the file system does not and cannot provide- especially with the inline mod proposal.

I like @ahmedcharles’s inline mod proposal as an alternative- in a way it is actually more explicit than the current system, by elevating the facade pattern into a language feature, and yet it is still more succinct. It clearly differentiates the case of organizational files from publicly visible submodules, without enforcing a directory structure. I do not agree that it hinders learnability, which I see as entirely a problem of discovery- facades are not a solution to the problems beginners have, so they don’t need to deal with it right away.

The new user scenario is a perfect example of why the problem with the current system is discovery. The solution, IMO, is absolutely not guessing what people expect, in the form of assuming that they’ll intuit the directory-based system. The solution is simply to tell them what they did wrong, via better error messages! This stands out to me as a better solution because it also applies to every other problem listed in the post:

• “Too many declaration forms”: We can’t remove any, so the file system proposal itself deprecates some- another error message. We could alternatively work on improving what we do now, as @xfix suggests.

• “Path confusion”: The file system proposal doesn’t address this at all. I suggest we could start warning on path usage in the top-level file that would break in submodules. For example, writing extern crate regex; and then regex::Regex could warn that regex is not used. (This could even become a hard error with implicit extern crate, which I’m somewhat less opposed to.)

• “Filesystem organization”: This plays into the same scenarios as the first bullet point. When someone just wants to add a new file to their project, and they try useing it or directly referencing it, just suggest mod the_file; in the error message.

• “Privacy”: With inline mod, we could even start warning on the facade pattern to suggest it instead. This could be adjusted depending on how precisely the code matches the behavior of inline mod.

• “Who can see this item?”: Taking a different tack here, since it’s about discovery while reading instead of writing- improve rustdoc! Add information about where things are defined and exported from. Maybe just a collapsed-by-default “internal path” would be enough, maybe something more involved. This is certainly better than making things more implicit.

• "pub use abuse": The file system proposal doesn’t solve this problem, either, IMO. Instead it just gives you even more files you have to check. inline mod and rustdoc improvements can both help the discovery problem here, in cases when you’re not in an IDE.

• “Repetition”: inline mod and implicit extern crate help here, without sacrificing readability.

In the end, even if mod foo; gets deprecated and modules are implicitly declared by directories, I will continue to use the current system and go out of my way to avoid the new one. It brings too many of the downsides of C++'s linking model, and its touted upsides are dubious.

[rpjohnst]

“separate files & privacy scopes” are modules.

[withoutboats]

@turnage

That rust’s modules are limited to files that have to be explicitly referenced (i.e. the directory doesn’t collectively define the module as in this proposal) is tedious when writing but so so pleasant when reading. I know exactly where to look to find where something is defined.

We can see the alternative in Go for example and I think it’s a strictly inferior experience having to rely on grep to determine what file defines what.

This is very different from my experience. Because of re-exports, I'm as likely to find a chain of hops between files if I follow a use statement as I am to find the actual definition. That is, if it wasn't glob imported at some point in the chain. And if what I'm looking for is a method (most commonly), it might not even be the file were the type was defined, because there's no requirement that impl blocks be co-located with type definitions. There's no way to follow use statements to find the impl block.

In other words, I already use ripgrep to search for everything. I don't find this to be a bad experience, but I imagine other people might prefer the IDE "jump to definition" solution instead. Either way, you can't reliably find definitions by reading the import system today.

[ahmedcharles]

The only reason you think Aaron’s proposal removes things is because it ignores backwards compatibility. I appreciate why he did that but one can’t judge how easy it will be to learn his proposal in isolation.

The primary features of the proposal are:

1. Implicitness in more places (which some number of people will always view as being a bad thing, myself included)
2. Collapsing modules which are declared as files.
3. Some new privacy primitives.

Like many other people, I don’t see any reasonable transition to this proposal resulting in this making Rust easier to learn.

But anyways, my goal isn’t to make Rust easier to learn, because I largely think that’s futile in a situation where you can’t break compatibility. My goal was to present a proposal that does a lot of what the original proposal does, but without losing the part of the module system I like, the explicitness of it.

[ahmedcharles]

No proposal is ever going to fix the problem that programmers write code that’s hard to read. Aaron’s proposal won’t make it easier to find the code. (In fact, it makes it harder than today, since you can’t tell which file something is in based on it’s module, if there are multiple files in a directory, unlike today, if you avoid pub use.)

If @turnage successfully uses the module system without reexports and therefore, it allows him to easily find code in his code base, that seems like reasonable feedback and this proposal would remove his ability to do that.

[kornel]

@ahmedcharles

The only reason you think Aaron’s proposal removes things is because it ignores backwards compatibility.

No. I mean new users could learn only the new, simpler way, and wouldn't have to learn the old way. Even if old way would have to remain in the language for backwards compatibility, for new Rust learners we could pretend it doesn't exist.

[kornel]

@withoutboats

How is this not already the case?

It is already the case, but only if committed code is broken (i.e. accidentally references non-tracked files). The proposal creates a new failure scenario where even a perfectly correct and sensible commits may be broken by stray files.

With current explicit mod declarations a stray file will be ignored after commands such as git checkout or git reset --hard, because they will erase any uncommitted mod declarations and the project will compile cleanly, even if there are extra files on disk.

With the proposed solution compilation after git reset --hard may still not give the same result as a clean checkout.

[vadimcn]

If you’d like to split a module into several files, why not just include!() them? Granted, this will not give you file-level privacy, but is that really so important?

IMO, the only thing worth fixing about Rust module system would be the difference between name lookup in use and the rest of the module (“Path confusion”). I would just make them both absolute, with an option to use self:: if one would like a relative path.

…okay, another small thing: it is a bit annoying that in alphabetic directory listing mod.rs, sorts in the middle of submodule files. I kinda like how Python’s __init__.py stands out among other files. Perhaps we could consider allowing _mod_.rs in addition to mod.rs?

[bennofs]

I do like this proposal, but I have three concerns:

a) this will make the layout for “simple” projects more complex. As an example, right now I have project with this layout:

src/
 hydra.rs
 nixpkgs.rs
 package.rs
 ...
 mod.rs
 bin/
    foo.rs
    bar.rs

With this proposal, I’d have to change that to

src/
 hydra/
  mod.rs
 nixpkgs/
   mod.rs
 package/
   mod.rs
 ...
 mod.rs # this is perhaps unnecessary
 bin/
    foo.rs
    bar.rs

Perhaps there could be the convention that automatic merging is only done if a mod.rs (or another name) file is present, so you can still use simple files for simple modules? Of course, that introduces another special case which hurts learnability…

b) Is there a particular reason why this proposal only considers pub(crate) and pub modules, but not pricate ones?

c) I fear that learnability may not be solved by adding to the language. Adding makes the language more complex, it cannot really reduce complexity (just hide it, and this is always leaky) as can be seen with C++ for example.

[phaylon]

I’d like to add one downside to non-explicit module discovery, that I believe also came up in previous discussions but I haven’t found here yet: The explicit version makes it really easy to exclude a whole module tree during development, by adding // in front of the mod statement.

Not being able to simply do that would make things less ergonomic for me.

[withoutboats]

You ‘just’ have to /* and */ (one more comment line at the end of the file), but we could also have a #[ignore] attribute or something.

[rpjohnst]

No, you’d have to do that to every single file in the directory.