My Preferred Module System (a fusion of earlier proposals)

language design
1

There are currently a couple of module system ideas floating around, and most of them have some aspects that I really like, but also some downsides.

I tried to amalgamate the presented ideas and see how they fit into my own vision of what I would find most useful. This is just a draft, and I tried to keep myself from getting too wordy.

I should again say that most of these ideas aren’t my own. The rough structure is very, very similar to @withoutboats proposal. The glob specifications and visibility considerations come from earlier discussions. There is probably nothing in here that I came up with myself.

I’m also not sure if I have left any problematic holes that came up before.

Summary

  • extern::.. paths for items in external crates.
  • crate::.. paths for absolute items in the current crate.
  • foo paths are now relative.
  • extern crate drives external crate availability.
  • mod drives crate structure.
  • use brings external items into the local scope.
  • pub specifies global availability.
  • mod * and extern crate * globs for externally driven structure.

Paths

  • Relative: foo, foo::other
  • External absolute: extern::cratename, extern::cratename::other
  • Internal absolute: crate::module, crate::module::other
  • Explicit relative (for macros): self::child, self::child::other
  • Parental relative: super::sibling, super::sibling::other

Local and global availability

  • Locally available items can only be accessed from the module itself. Submodules cannot access only locally bound items.
  • Globally available items can be used from elsewhere, according to their visibility.

External crate availability

External crates are made resolvable in the current module and all submodules with one of

<vis?> extern crate <ident>;
<vis?> extern crate *;

Where <vis?> is an optional visibility specifier and <ident> is an external crate identifier. The glob specifier * will make all externally specified extern crates available.

Without a visibility modifier, the crate can only be accessed locally from the current module or via an external absolute path (extern::cratename::item) from the module or any submodules. The visibility modifier binds the crate globally as item in the local module with the specified visibility and makes it globally referable to via <module-path>::<item>.

A single glob variant can be combined with any number of non-glob variants, where the glob variant provides the default visibility.

Examples:

// Make all external crates available to this module and submodules
extern crate *;

// Make the `tendril` crate available to this module and submodules
// and make it globally available as `<mod-path>::tendril`
pub extern crate tendril;

// This will make `tendril` available in the outer module, but
// only the `inner` module will be able to access `serde`
extern crate tendril;
mod inner {
    extern crate serde;
    ..
}

Module structure declaration

Submodules are declared with one of

<vis?> mod <ident>;
<vis?> mod <ident> { .. };
<vis?> mod *;

Where <vis?> is an optional visibility specifier and <ident> is a module name identifier. The glob specifier * will declare all externally provided submodules, as specified by the filesystem.

Without a visibility modifier, the submodule can only be accessed locally from the current module. A visibility modifier will make the submodule available as item of the current module with the specified visibility and makes it globally referable to via <module-path>::<submodule-ident>.

A single glob variant can be combined with any number of non-glob variants, where the glob variant provides the default visibility.

Examples:

// Load all submodules, but only make them available to this module
mod *;

// Load all submodules, and make them available to the whole crate
pub(crate) mod *;

// Declare submodule `text` to be available to the whole crate
pub(crate) mod text;

// Declare submodule `api` to be available to everyone
pub mod api;

Importing items into the local module

An item can be imported into the local module with

<vis?> use <import-spec>;

Where <vis?> is an optional visibility specifier and <import-spec> is either a single item path like foo::Bar or an item group like foo::{ Bar, Baz } or a glob like foo::*.

Without a visibility modifier, the imported items can only be accessed locally from the current module. A visibility modifier will make the items available as items of the current module with the specified visibility and makes them globally referable to via <module-path>::<item-ident>.

Examples:

// Make `io` locally available, cannot be referred to globally
use std::io;

// Make a submodule item publicly available as child item of the
// current module
pub use submodule::Item;

Restricting item visibility

An items global visibility can also be specified with

<vis> <relative-item-spec>;

That is: A visibility specifier followed by a relative item path like foo or foo::Bar or a relative item specification like foo::{ Bar, Baz }.

The visibility adjustment can also be specified for items in child modules, but they can only be restricted further than specified by the submodule itself. In other words: A submodule item that the submodule considers part of its public interface can be for example restricted with pub(crate) submodule::Item.

The specified visibility of a submodule is the default maximum visibility for its items. Further relaxations of the visibility of items of the submodule will also adjust the visibility of the submodule itself, but not of the items that aren’t specifically mentioned.

Examples:

// set visibility of a local item
pub MyType;

// `submodule`s item are by default only available via local access
// from the current module, but `submodule` and ?submodule::Item`
// can be referred to globally.
mod submodule;
pub submodule::Item;

Provided solutions

Used paths are now relative or explicitly absolute

Paths that aren’t fully qualified can only be resolved through locally available items.

The crate root is no different than any other module

An explicit pub is required at any level to make items globally available, and paths are relative by default, so referals to the crate root have to be explicit.

Crates and modules don’t have to be strictly specified

Only requires specifying that submodules actually exist with the glob syntax. Allows for consistent, explicit visibility specification.

External items are identifiable

Due to the extern keyword at the beginning of the path.

Adjusting submodule item visibility does not require reexporting

Example:

// Autoload all submodules locally, make image available to the
// whole crate, and provide image::Png, image::Jpeg and text::Text
// as part of the current module's API while keeping them in their
// original place.
mod *;
pub(crate) mod image;
pub image::{ Png, Jpeg };
pub text::Text;

Appendix

Requiring some specification of external crates and modules

  • Allows specifying default visibility.
  • Documents in the module that submodules are autodiscovered.
  • Documents that external crates are autodiscovered.
  • extern crate is useful for restrictions.
  • extern crate * is only one statement and can be inserted by cargo new.

Possible compatibility with facade pattern solutions

Since this is being discussed elsewhere, I thought I’d note that this should be perfectly compatible:

inline mod *;
pub(crate) inline mod submodule;
1 Like
2

There’s no need for extern crate anymore. Everything external is ::extern::smth so compiler knows which crates to take, and one can grep for it to get a list (if Cargo.toml is not good enough). I do prefer explicitness, but how many redudndant lists of everything we have to maintain to make things explicit enough? :smiley:

crate:: can be just :: for absolute paths.

There is no need for mod. Since mod m does bring a module name m into the namespace, it is the same as use m. With relative paths by default, if the compiler sees use foo::bar and there is no foo symbol in the current module, it can just check for foo module and consider it an implicit mod foo

In other words, if there is no point in repeating yourself because of ambiguity, reliabilit, etc I don’t want to be repeating myself.

1 Like
3

I do believe that there is value in those specifications.

In my proposal above, both mod and extern crate have glob variants:

extern crate *;
mod *;

It documents that there are dependencies and modules being autoloaded. It also solves the problem of default visibilities, since there is an obvious place to put them:

pub extern crate *;
pub(crate) mod *;

It also has good symmetry when specific crates or modules should have different visibilities:

mod *;
pub mod api;
pub(crate) mod buffer;

And binding under a different name follows the same symmetry:

mod *;
mod text as encoded_text;

Using the extern crate directive to drive availability of external crates to modules and their submodules makes it easy to contain where certain dependencies are handled.

Considering the above (renaming, visibility) and other things in a similar vein (attributes) I would want these to be handled in code, and not defined externally. And I would find it inconsistent to have to specify some modules, but not others. In other proposals without the glob specifier, I would have to explicitly list all modules with (for example) visibility deviating from the default. The proposal above just requires adjusting the visibility given to the glob declaration.

As for ::foo for absolute paths, I wouldn’t be opposed to it, but it is nice that absolute paths are all rooted in something recognizable.

4

I like the paths grammar, and use. :slight_smile: (I assume the word "external" in the description of use does not mean external crate. Do you mean something like "use brings paths into local scope"?

I find this pretty confusing. It took me a little while to realize what you mean by "available" here -- you mean items that can be mentioned with use, but are not actually brought into scope yet. So I have to first make an external crate available, and then I have to use it, except in the module that did the extern crate; this module does not need an additional use extern::. I don't think it is intuitive that use extern:: somewhere depends on extern crate somewhere else. Absolute paths are all about not depending on the relative context.

Thinking about this made me finally understand why people propose "automatic extern crate" and "automatic mod based on the file system". And I have to say, I sympathize with them. The alternative presented here requires an entire additional concept (rather than just imported or not, items can now be unavailable, available and imported), and it makes absolute paths depend on the local context. So I think I am with @withoutboats now as far as this is concerned -- and I think this reasoning should be presented in the module system proposals that want to get rid of extern crate and mod; either everyone else wants this for different reasons or I somehow entirely missed people explaining why that's a good idea.

So use sometimes is an item visible from submodules and sometimes not? That's not pretty. It should be consistent.

1 Like
5

Yes, "external" here means outside.

I'd be fine with having extern crate function just at the crate root, but I'd still like not dropping it, because of visibility modifiers, renames, attributes (as mentioned above).

It's not required. But since extern crate * and mod * are useful (see above), I thought it would make sense for them to mean something.

I'm not sure what part gave you that idea. My plan was:

// assuming we have
mod foo;

use foo; // no longer needed, mod foo or mod * do this
use foo::Item; // can now be referred to locally
pub use foo; // would just make the submodule public, same as pub mod foo;
pub use foo::Item; // Item can now be referred to locally, and it's public

The difference to the current system is that without any visibility specified, the submodule would only be available in the current module.

6

Update: I’ve made a new proposal with a much smaller scope and impact.

7

What do you mean, not required? Your proposal has a concept that is entirely absent in some of the others: external crates that are "available" but not imported, vs. those that are not "available". This distinction doesn't have to exist. While you provide an option to conveniently make everything available, that doesn't solve the problem that this distinction keeps existing, and that everyone will have to learn about it.

If I read your text correctly, non-pub use cannot be used from submodules:

use foo::Item;
mod bar {
  use super::Item; // fails
}

So, use is not an item. However, pub use can be used like that, so pub use is an item.

8

I basically meant that even without that functionality, there is a possible use. With the distinction in tact you're right that it would be required.

Ah, yes. Personally I'd still call that an item just with a different visibility, but my terminology might be off.

As noted higher up, I have made a new, much smaller proposal that only takes the necessary parts from this one, and actually allows not having extern crate, and leaving all other visibility functionality as it is. I'd love your opinion on that one as well!

9

A "hidden even from submodules" visibility that does not exist anywhere else? That's quite a stretch.

10

Well, it’s local-only visibility. After all, the module itself would still be able to see it. It is a visibility I have missed in the past, when submodules were supposed to use something from their parent module, but that something had internal implementation details I didn’t want to allow leaking into the submodules (and their APIs).

Like I said, the new proposal doesn’t have that, since it is kind of a separate issue.

11

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.