The Great Module Adventure Continues

[nikomatsakis]

@rpjohnst

Another aspect of the “amount of transition” axis is the effect on existing documentation. There is a substantial amount of example code out there on StackOverflow, in blog posts, etc. where it won’t be fixed by rustfix. My guess would be that this kind of stuff primarily uses external paths, both in uses and in expressions, so preserving those would be the least likely to break that example code.

Yes, a very good point. It seems like "co-existence" is key here.

[petrochenkov]

I don’t like the : syntaxes.
Even ignoring type ascription ambiguities, path being just a homogeneous list of segments is such a nice property.
You can combine paths with segments, paths with paths (see https://github.com/rust-lang/rust/issues/48067 and nested imports) and you’ll still get a path resolved in the same way regardless of how the segments were glued together.

[nikomatsakis]

@petrochenkov

Even ignoring type ascription ambiguities, path being just a homogeneous list of segments is such a nice property.

Is there really a "deep" distinction here? For example, we have ::std::foo and std::foo. That first segment (::) is kind of a "special case" of sorts, perhaps no different from :std. That said, I personally lean towards either the leading-:: or leading-crate options, as much for their familiarity as for any other reason, so perhaps it's a moot point.

@petrochenkov, how hard would it be to implement the "fallback" semantics for absolute paths, so we can test the impact? Would it be reasonable to assess said impact by doing a crater run? (i.e., to ensure that all existing code still works)

[rpjohnst]

@nikomatsakis

It seems like “co-existence” is key here.

I don't think coexistence in the sense of your summary post is quite that important for preserving existing documentation, actually. Presumably we don't want people to actually use the old syntax, so "just" making that documentation work is insufficient. There may be lints and later error messages that guide the transition, but it would be better just to make the delta smaller so the documentation is already using the new form as much as possible.

This goes along with a lot of the anxiety around having epochs at all- the easier it is to read code from a different epoch, because you're spelunking in some transitive dependency nobody's touched in years, the better. The fewer subtle differences to trip on when an epoch isn't your main focus, the better. This is part of why the RFC went the way it did, and this is what we lose by going for properties like 1path, which is part of why I don't think it's that helpful of a property.

[nikomatsakis]

@rpjohnst

I don’t think coexistence in the sense of your summary post is quite that important for preserving existing documentation, actually.

I think we can distinguish four cases:

• Older syntax is still the right syntax.
• Older syntax still works as it ever did, but is deprecated (you get a warning).
• Older syntax errors out, but you get some good advice on how to fix it.
• Older syntax has been repurposed, and hence now does something different.

All other things being equal (which of course they are not), these are in order of preference. The second one is what "co-existence" gives us -- it still seems like a useful distinction.

I definitely think we should not "jump" to permitting use foo::bar as a 'relative path', even if we lay the steps that permit us to do so later. But laying those steps would mean that -- for such syntax -- we would expect an error, but hopefully with clear directions for how to fix. So we'd be at the 3rd rung.

(There is even a slower path, of course, where we don't make anything an error, but we reserve right to do so in the next epoch, if we so choose. This may be wise, and moves everything up to the second rung.)

[petrochenkov]

@nikomatsakis

how hard would it be to implement the “fallback” semantics for absolute paths, so we can test the impact?

I don't know until I try. Should be simpler than "macro modularization" (subset of Tracking issue for "macro naming and modularisation" (RFC #1561) · Issue #35896 · rust-lang/rust · GitHub), which probably laid necessary groundwork.
Certainly not hard for @jseyfried from the parallel universe in which he has time

[troplin]

I didn’t follow all the discussions since last time this came up but is this syntax completely off the table?

from crate use path

What I like about this is that it cannot be used “inline”, i.e. importing an external crate sticks out. Still, I’d say that the 1path property applies somewhat, if only the path is considered, not the crate name.

But if this has already been ruled out, I don’t want to revive the discussion.

[petrochenkov]

@nikomatsakis

Is there really a “deep” distinction here? For example, we have ::std::foo and std::foo. That first segment (: is kind of a “special case” of sorts, perhaps no different from :std.

Yeah, any sigil at the path start (:a::b) is probably not worse than existing ::a::b in this sense, it will turn into {{root}}::a::b anyway.
(Note: representing absolute paths using an extra segment turned out to be such a great idea in the hindsight).

[djc]

Really like the table and structured approach to evaluating goals here!

Personally I think leading-:: is the most attractive in the long term, followed by leading-crate. I guess a lot might depend on whether the fallback thing works out.

[arielb1]

@nikomatsakis

This table summarizes the proposals in terms of the properties above. It’s hard to judge aesthetics, of course, but I’ll note some of the specific concerns that have been raised.

Was the [krate]::foo::bar syntax decided against? It's semantically equivalent to the rest of the use extern::krate::foo::bar syntaxes and feels more natural than the already-overloaded single colon while being less heavy.

Having use foo:bar; (or use :foo::bar;) and use foo::bar; refer to 2 similar but different things feels like it would raise "when do I use a single colon and when do I use a double colon" confusion.

Of course, if we can have a good enough fallback story (which I am not that sure there is - IIRC there was a problem because use foo; can either crawl in the filesystem to find foo or import a module foo) then using a crate::-style syntax might be a better choice to reduce the impact on examples.

[dan_t]

I somehow like ‘leading-crate’, but the difference between 'External use’ and ‘External in fn body’ is really irritating.

The beauty of ‘leading-::’ is, that there’s no such difference, but ’::crate’ feels redundant.

I don’t like the variants with ‘:’, the visual difference between ’:’ and ‘::’ is IMHO too low.

There’s something about the ‘leading-extern’ variant. Yes, it’s a bit more noisy, but it’s pretty clear and consistent.

[nikomatsakis]

@arielb1

Having use foo:bar; (or use :foo::bar;) and use foo::bar; refer to 2 similar but different things feels like it would raise “when do I use a single colon and when do I use a double colon” confusion.

Very good point.

[nikomatsakis]

@arielb1

Was the [krate]::foo::bar syntax decided against? It’s semantically equivalent to the rest of the use extern::krate::foo::bar syntaxes and feels more natural than the already-overloaded single colon while being less heavy.

Not definitively. I will extend the table.

[nikomatsakis]

Updated the summary to include [crate]; also included a note about the conversation that @rpjohnst and I had regarding the cost of transition.

[nrc]

Recording my preferences:

• I think the 1-path property is very nice to have. While it doesn’t have to be an essential goal of this work, I fear that the cost of any churn is probably not justified if we miss out on it.
• I find the : vs :: distinction very subtle, I’ve been stumbling on it in this thread, never mind in a real code. I’m pretty strongly opposed to any scheme which relies on a semantic difference based on : vs :: especially inside a path (as opposed to as a prefix)
• I use :: in my code more frequently than many, and I don’t find it odd at all, it feels like a natural way to distinguish between absolute and relative paths which is coherent with the rest of the path syntax and fairly lightweight.
• I would prefer a scheme which encourages one block of imports per crate, rather than one block for the local crate and one block for all external crates (thinking about imports in Python and JS, one block of imports per crate feels like it would be easier to read quickly).

Given all the above, my preference is for leading-:: I would also like to allow relative paths in use statements, eventually.

[withoutboats]

@nrc

I find the : vs :: distinction very subtle, I’ve been stumbling on it in this thread, never mind in a real code. I’m pretty strongly opposed to any scheme which relies on a semantic difference based on : vs :: especially inside a path (as opposed to as a prefix)

Important to remember that your syntax highlighter will hopefully help you in real code.

---

I also prefer :: in the abstract - it seems like the obvious solution. But I'm worried about the fallback & transition story. I think its not that uncommon to have an extern crate and a module at the top level with the same name - usually because the extern crate is an optional dependency and that module contains all the code relating to it. In fact, failure has just such a module. So if the story around this isn't smooth, its not hypothetical - people will actually be inconvenienced.

I'm generally open to any 1-path solution except for extern, just because it is far too long. This isn't only because inline use is important, but also because I don't want to see a single nested import become necessary or the idiomatic default because the alternative is too noisy. The fact that imports/mod statements have a very different shape from code is very useful to me when I'm visually parsing a file of code - it tells me where to start reading. If imports have a braced and highly nested structure, they'll start to look like types and functions and that will frustrate me. (I'm not against nested imports as an option when they're suitable, I just think the idea of always importing everything from std with 1 use statement is actually a loss, not a gain).

[djc]

@dan_t

The beauty of ‘leading-::’ is, that there’s no such difference, but ’::crate’ feels redundant.

I actually like ::crate, because it means all use statements start from the same root level (of the crate registry universe).

[dan_t]

I actually like ::crate, because it means all use statements start from the same root level (of the crate registry universe).

The crate might not be part of the registry universe (crates.io).

The prefix '::' is a stronger visual marker if it only references to external dependencies.

Having a separation of internal and external modules by the prefix - '::' and 'crate' - seems IMHO a lot more clear than having '::' and '::crate'.

[rpjohnst]

@djc

I actually like ::crate, because it means all use statements start from the same root level (of the crate registry universe).

Leading-crate has this property as well- both use crate::foo and use std::foo start from the same root level, and this was why the RFC wound up with that design to begin with.

@nrc

I think the 1-path property is very nice to have. While it doesn’t have to be an essential goal of this work, I fear that the cost of any churn is probably not justified if we miss out on it.

I am not a fan of leading-::, or really the concept of 1path in general, for similar reasons to @withoutboats's point against idiomatic nested use. use statements are different from expressions in practice: the vast majority of the time they pull from dependencies and across the module hierarchy within a crate, not from submodules or anything else in scope. Thus:

• Forcing extra syntax to import from the root is extra noise for the common case, without solving the problem that kicked off this discussion to begin with- confusion around where use paths start, caused by extern crates being in-scope in the root module. This is solved in the RFC by moving dependencies up one level in the hierarchy, implying they're not relative as in the common case, but leading-:: primarily seems to imply that they are relative and that you're opting out of that.
• I like the super::/self:: syntax to explicitly call out when something different is going on. Previous discussion even proposed other ways to specify reexports, though I think self:: + nested use might be enough there. This is both a) a little extra syntax for the uncommon case, and b) familiar from other languages like Python or Haskell that keep an explicit list of reexports.
• Further, 1path is not a very common property in other languages. The few that have it only have it by virtue of shadowing-style scoping across namespaces, not by forcing imports to use the syntax from expressions. Those that don't do this (i.e. most of the ones I looked at) require an import statement to reference anything at all. It's also notable that Python intentionally removed the ability to do relative imports without their equivalent of self/super.

To me, the smaller churn of leading-crate is justified because it directly and more thoroughly solves the "path confusion" problem, puts use more in line with other languages that people may be familiar with, and avoids introducing any more line noise into the language.

I'll also say again that it feels to me like a step back from the RFC to re-introduce a bunch of alternatives after all the discussion it took to eliminate them.

@withoutboats

I think its not that uncommon to have an extern crate and a module at the top level with the same name

I'm not sure what the difference would be between the proposals, here. In all of them you'd replace the extern crate in the module with a use, and add a crate::/::crate::/[crate]::/[]:: before references to the module. Before doing so, all of them would give you a deprecation warning on use backtrace:: because it refers to a top-level item without going through crate::. I suppose the flag day/fallback variants might additionally require an extern crate backtrace as something_else (or the equivalent Cargo.toml change)?

[nikomatsakis]

Since there hasn’t been a lot more iteration since that last survey post, I brought up this thread with the lang team in our most recent meeting. We had quite the epic discussion, discussing this thread, the proposals, and some of the tradeoffs between them. Somewhat surprisingly, we made some actual progress: we achieved a measure of consensus on some of the constraints we ought to be aiming for, which in turn helps to narrow down the ‘search space’ considerably.

First, there was a general consensus on the team that we really want to ensure co-existence and to avoid fallback. In particular, co-existence ensures a “graceful” introduction of the epoch (e.g., old code from stack overflow continues to have its unaltered meaning, though it may yield deprecation warnings). Fallback as a means to ensure co-existence is suboptimal, since it implies the potential for confusion (not so clear what a path means, particularly if e.g. you are moving back and forth between projects in distinct epochs). (There are also technical concerns about its feasibility.) This conclusion effectively rules out the leading-crate and leading-:: proposals, as both of those proposals require fallback to ensure co-existence.

There was also a general consensus that the 1path property would be valuable. As @joshtripplet put it, it seems a shame to come this far on the design, and yet not achieve the ability for a path to have uniform meaning. Not coincidentally, 1path and coexistence are naturally aligned, since both are achieved by introducing some new, unambiguous syntax for absolute paths.

Proceeding from there, the remaining proposals can basically be grouped into two camps:

• the leading-extern camp, which use keywords to identify an absolute path (in particular, crate or extern).
• the leading-sigil camp, which use some form of sigil to identify an absolute path.

There are various known characteristics to both styles:

• leading-extern:
  • pro:
    • very readable, clear meaning
    • extern crate foo ==> use extern::foo is a kind of natural transition
    • use extern::{..} blocks can be seen as a clean, elegant syntax
  • con:
    • the “current crate” is no longer “parallel” with the extern crates
    • extern::foo is annoyingly long for inline use, pressing towards use statements
    • similarly, if you don’t want to use a use extern::{..} block, then very repetitive
• leading-sigil:
  • pro:
    • compact, suitable for use “inline”
    • current crate is parallel with the extern crates
  • con:
    • sigils are always a learning hazard – they are hard to google, etc
    • use statements are one of the first things you see when opening a crate
      • what will be the visual impression from a “wall of sigils”?

Thus it’s probably best to try and focus on deciding between these two variants, if we can (without getting too stuck on what the sigil should be). At this point, it makes sense then to do some experimentation with the two styles. Try out different sigils. See how you feel and report back.