[pre-rfc] Stable features for breaking changes

[nagisa]

• Feature Name: (fill me in with a unique ident, my_awesome_feature)
• Start Date: (fill me in with today’s date, YYYY-MM-DD)
• RFC PR: (leave this empty)
• Rust Issue: (leave this empty)

Summary

This RFC proposes a concept of stable #[feature]s, which allow implementing backwards incompatible changes.

Motivation

Rust wants to be able to introduce breaking changes without necessarily releasing the 2.0 version of the language (yet). We already use a system of deprecation warnings which eventually become hard errors to remove features we want.

This approach, however, is only appropriate for breaking changes where the “old way” becomes invalid and an alternative approach is introduced. It is not appropriate if it is desired to change semantics of some code altogether.

Notable examples of such code could be:

• Introducing new keywords (e.g. union, catch);
• Changing semantics of some syntax (e.g. impl Trait versus Trait);

Detailed design

Before proposing something concrete lets investigate what the other languages do to enable breaking language changes.

Case Study

Python

Python has a mechanism to “import” new features (from __future__ import ...). These were used extensively to implement libraries in a way that’s compatible with both Python 2 and Python 3 at the same time. In Python 3 most of these imports became a no-op.

It wasn’t widely used before the release of 3-series of Python.

Haskell

{#- LANGUAGE -#} pragma is very similar to Rust’s #[feature]. Due to this pragma, Haskell code can take advantage of advanced features such as GADTs, type families and template haskell without necessarily having these features specified in the base Haskell language (The Haskell Language Reports). Note, that LANGUAGE pragma is used more as a way to enable extra super-user-like features, rather than as a way to introduce breaking changes to the language.

Across major revisions of language specification (from Haskell 98 to Haskell 2010) only a very small number of the LANGUAGE features got enabled by default in the standard.

Stable opt-in semantic-changing features

The primary component of this RFC is introducing stable #![feature(name)] annotations. These annotations would have behaviour very similar to our current unstable feature attributes; the only real difference is being able to use these in stable compilers. The lifetime of such stable attribute is expected to be as follows:

• A semantic-changing feature gets implemented, is available behind unstable #![feature(name)] (same as currently);
• The feature gets “stabilised” and follows all the usual stability guarantees. The alternative behaviour must be enabled with #![feature(name)] before use;
• At eventual release of the next major language version (see Releasing Rust 2.0) these features are considered for being enabled by default.

This approach has both the benefit of allowing early adopters to use nice, but breaking features and allows people to write forward-compatible code as well (i.e. code which would work on both Rust 1.x and Rust 2.0).

Incompatible library changes

There’s a question of the language changes which impact the libraries. Most notable of these changes is introduction of the new keywords.

Making union/catch a keyword (an immediate use-case described above) would involve changes to the libraries as well. The libraries could use these words in their public APIs: methods, free standing functions and so on. In fact the standard library itself exports a few methods with the name union.

Presumably, as a part of an RFC introducing a new keyword renamings for these methods would be proposed. However, in that case the feature cannot be implemented before the public API is fully renamed (i.e. the public API with the old name is entirely gone).

This demonstrates that the libraries need an ability to react to the presence of the feature somehow. It is a problem with many tricky corner cases, however. For example:

Crate ROOT depends on A and B, B depends on A as well. A exports a function with name catch. Both ROOT and B use this function. A implements some scheme which renames catch to _catch, when catch_keyword feature is enabled. ROOT enables the feature and replaces all calls with _catch. B, however, hasn’t enabled this feature yet and still uses the old catch function. This should work – which implies two different A crates in the dependency graph(?).

To solve this something along the lines of #[cfg(feature(something))] that gets evaluated on import of a library would have to be implemented.

[Author note: no concrete implementation ideas currently]

Guidelines

The stable features shall not be abused to introduce unnecessary breaking changes. In short, the stable features introduced by this RFC should only be used as a last effort – when a strongly desired design or syntax cannot be introduced or changed in a backwards compatible way. A good indicator of stable features being misused would be a lack of (heated) discussion exploring alternative syntaxes for some feature.

(Sidenote: technically if a feature could be implemented with a contextual keyword, then the current wording suggests the contextual keyword approach ought to be preferred, but to me personally an option to get rid of these contextual keywords is a strong motivation for this RFC, as I find them to be the nastiest hack ever invented)

Some immediate use cases for this could be:

• Trait now has the behaviour of current impl Trait proposal, old unsized behaviour is accessible via dyn Trait (new keyword);

  Current behaviour of Trait is seldom used and hides the costs of dynamic dispatch, while the 0-cost sugar cannot use the most sensible syntax for it. The feature would be inaccessible without enabling trait_shorthand feature first.

• catch keyword. Introducing catch blocks ran into syntax ambiguities with struct initializers. Making catch a keyword is the only fix (not even contextual keywords would help), but it cannot be done backward-compatibly.

Releasing Rust 2.0

With the scheme as above in place, it becomes significantly easier to implement and release a breaking version of the Rust language. The #[feature] opt-ins allow users to test the feature for major warts, helps teams to probe the desirability for a particular feature, and in would signal the general direction the 2.0 version should go towards.

Ideally, releasing 2.0 would then become switching a number of these features to enabled by default and changing the version number (maybe also adding a flag similar to --language=rust1, which would switch these features back off).

Forward compatibility

When considering a breaking release, the forward compatibility is not to be underestimated. Python 3 example shows clearly that people will be using both the old and the new version of the language thus fragmenting the ecosystem and some libraries will not be ported to the new version of the language. It is important to make it easy to port these libraries.

While python has a tool to do the automatic conversion, it turned out to not be the ideal solution, primarily because it couldn’t handle converting all the code bases correctly. It also generates code that is only valid on python 3 and not both versions, which prompted people to do the “porting” manually anyway.

If the ideal scheme outlined in Releasing Rust 2.0 is followed, then porting the codebase would simply involve adding all the features which got enabled by default to the top of the library file and fixing up the code (could be done automatically to at least some degree). Not only libraries ported this way would be forward compatible, but the library authors would have an easy way to track the porting progress as well.

How We Teach This

…snip…

Drawbacks

It is possible that eventually wanting to use a full subset of Rust would involve adding a large number of #[feature(...)] to the lib.rs. (counter argument: it is not a big problem in Haskell)

People learning would have to learn the base language and then in addition each of the “features” they encounter along the way. This is both good and bad. In haskellandia stuff like GADTs are not trivial to understand, and so on one hand people avoid learning what they most likely will not need when starting out. On the other hand, it is much harder to go back to understand what GADTs are when you encounter them in the wild while working on your own stuff with possible time constraints.

Alternatives

No alternatives have been investigated yet.

Unresolved questions

How to handle libraries? (see section Incompatible Library Changes)

end of rfc text

---

Note that this design is not as fleshed out as I would like it to be, but some recent discussion on #rust-lang IRC channel prompted me to write these ideas down and start a more public and lasting discussion this way.

cc @aturon @nikomatsakis @withoutboats

[rkruppe]

As you describe, Python and Haskell use a similar mechanism for very different purposes; __future__ imports are supposed to become the default eventually, while {#- LANGUAGE -#} pragmas just as often describe experimental features that may never be standardized (at least in the current form). This means their respective drawbacks are different as well:

It is possible that eventually wanting to use a full subset of Rust would involve adding a large number of #[feature(…)] to the lib.rs. (counter argument: it is not a big problem in Haskell)

Our use of “stable features” would be closer to Python than to Haskell, so the counterpoint is not very convincing to me. It’s not just about using “the full subset”, it’s making your code forwards-compatible with the “true” or “intended” Rust language. If we want to transition towards all Rust programs using these features, then enabling the corresponding feature needs to be the default at least for the transition period (which may easily span a year or two). This is particularly severe with far-reaching changes such as union being a keyword or dyn Trait, which directly affect a large portion of programs.

On top of that, there’s issues like being able to move code from one crate to another, or introducing code that is affected by the feature into a crate that previously wasn’t. Because of these issues, even crates which wouldn’t be affected by a feature should enable them, especially as the feature becomes the overwhelming default for code that is affected. In other words, to actually achieve a painless transition, we’d need an ecosystem where almost every program larger than a few lines enables these features without prompting. I have doubts whether this is possible, even if we take lessons from Python to heart as you sketched above.

[kornel]

This sounds like a very good approach. If rustc supports both Rust 1 and Rust 2 at the same time, it’ll avoid fragmentation on packaging level. If each crate (or file?) can switch between v1/v2, it’ll allow free mixing and gradual migration.

Adding #[feature()]s sounds a little bit like use strict in JS or Perl, but it’s a tolerable opt-in, especially if Rust promised to eventually make it unnecessary (or disable per project with a flag).

[jntrnr]

Seems like the “How we teach this” shouldn’t be snipped here. We’re recommending a way for people to think about the future and backward breaking changes. There’s some socialising/education aspect to this proposal we should explore.

[retep998]

A implements some scheme which renames catch to _catch, when catch_keyword feature is enabled.

This is incorrect usage of cargo features. Cargo features must only be used for additive API additions. They should never change or remove anything. The only practical way to deal with this situation at the moment is to release a new major version of A with the rename.

EDIT: Unless you actually meant that A isn’t providing a cargo feature, but rather it is a Rust language feature which ROOT is enabling which affects its dependencies which is typically what cargo features do, which is why I’m so confused. This really needs to get properly fleshed out with an actual concrete implementation idea that makes sense within the cargo ecosystem.

[nagisa]

Seems like the “How we teach this” shouldn’t be snipped here. We’re recommending a way for people to think about the future and backward breaking changes. There’s some socialising/education aspect to this proposal we should explore.

This is snipped primarily because I have no idea what challenges would be. There’s a reason I wrote “pre-RFC” and “Note that this design is not as fleshed out as I would like it to be”.

---

This is incorrect usage of cargo features. Cargo features must only be used for additive API additions. They should never change or remove anything. The only practical way to deal with this situation at the moment is to release a new major version of A with the rename.

This pre-RFC does not deal with the “cargo” features, as explained on the IRC. Thanks for the insight about parallel-ity with the cargo features, though. It may be helpful to flesh out the design further.

[est31]

The #[feature] opt-ins allow users to test the feature for major warts

Isn’t what stable #[feature] opt ins are about that their behaviour doesn’t change (being stable)? So shouldn’t testing for warts rather happen in the pre-stabilisation phase?

releasing 2.0 would then become switching a number of these features to enabled by default and changing the version number

Why not switch all to enabled? I mean, once a feature gets added to the stable language, it should be clear that its presence in Rust is wanted, no? Or should they be used for something else than for breaking changes (like this thread’s title suggests).

maybe also adding a flag similar to --language=rust1, which would switch these features back off

I think that maintaining support for the old rust would be not beneficial, like the long time that the old python was maintained wasn’t beneficial.

It might also make the rustc codebase grow in complexity over time.

About the general idea of the RFC, while I do like it, it can of course be misused. I think before starting to use it, we should gain consensus about how a Rust 2.0 (and succeeding breaking changes) should look like. Should such changes be able to introduce radical new concepts, like rewriting the entire standard library to use named args? Should they be able to remove language features, like the macros 1.0 system? Should they be some kind of bug-fix release with no more radical breaking changes than additions of keywords and making a better entry api for hash maps? Should they be used for something else than to avoid to make new features feel bolted on?

You kind of touch that topic when saying that stable features shouldn’t be used light heartedly. But I’d like to have more rules about it, maybe even in a new RFC.

[matklad]

Some thoughts:

• I think contextual keyword is not the most compelling example: I’d rather prefer union to always be contextual, because I use HashSet::union more often than the untagged union (but the catch problem is indeed more sever).

• Instead of several #[feature] flags we can have one #[language_version] flag: that way, there won’t be an exponential number of dialects, and the users will be encouraged to actively update their code (“if you want this shiny new feature, please fix all old deprecations”).

• In the case study section, I think it’s important to mention that although Python is a prime example of bad incompatible changes, in Rust they presumably may be less devastating, because Rust is a compiled language (i.e, you can’t load Python2 and Python3 modules together, but you can link Rust 1.0 and Rust 2.0).

• I’ve heard that Swift manages to do some breaking changes with the help of automatic code update tool. It should be mentioned in the case study if I’ve heard right.

• <rant> Upgrade tool is hugely important. Moreover, I think we need a proper library for manipulating Rust AST. libsyntax is to tied to the compiler internals last time I’ve checked (it’s been a while, maybe somethings changed). It’s insane that RLS does not have access to syntaxt data structure (or does it?), and that rustmft is build on top of syntex_syntax https://github.com/rust-lang-nursery/rls/issues/118 </rant>

• I think that if we want to do breaking changes in theory, we should not postpone it forever, but to actively experiment with ultra-minor breakages. That is, I’d release Rust 2.0 with just std::thread::scoped removed just to uncover potential problems with larger changes. This RFC is a very good step towards embracing the need for breaking changes,

[aturon]

Thanks, @nagisa, for getting this conversation started! As I mentioned in IRC, the core team has also been thinking some about these questions, in large part because of the interaction with language design questions that are arising with the ergonomics initiative.

I want to make clear that these are early thoughts, and we are not yet to the proposal stage, let alone to any kind of decision. The core team has mostly been working to uncover constraints and otherwise explore the design space around versioning questions.

That said, let me spell out a few key points that have emerged in our discussions.

The primary goal should continue to be stability without stagnation. In detail:

• “Stability” means you should never dread upgrading the compiler. It means that that code written for different versions of the Rust compiler and standard library should work seamlessly together; there should never be an ecosystem split.

• “Without stagnation” means that the language can grow and evolve over time, with idioms shifting accordingly. Ideally, it also means that we can produce versions of Rust that, for example, essentially remove (or change) deprecated features.

Of course, there are some obvious tensions here! For one, stability cannot mean that Rust code must forever look and feel the same way; after all, even new features like ? have already shifted idioms, so that try! often looks a bit dated—despite it not being a breaking change.

But sometimes we want to make deeper changes to Rust, which on their face are backwards incompatible, like changing fn foo(Trait) to mean fn foo(impl Trait) instead of a trait object. We can start by deprecating the existing syntax, that’s easy. The harder question is how to do things like “remove” deprecated features, while retaining stability as stated above.

Epochs

The idea we’ve been tossing around in recent core team discussions for threading this needle has two components:

• The Rust compiler stays forever on the 1.X series. From a semver perspective, the compiler never breaks existing code (module the usual caveats about changes to inference etc. which we’ve covered elsewhere, and hope to separately improve the story for). The basic experience for existing code is much like today, with painless upgrades and ecosystem-wide interoperation.

• Separately, we introduce the notion of epochs, which correspond to calendar years like Rust 2018 (though not literally every year; see below). Somewhat like proposed in the pre-RFC of the original post, these epochs are basically compiler switches to opt in to a “newer version” of the language—not unlike with C++ compilers today. Crucially, cargo new would set up a project on the latest epoch, so that new code always defaults to the latest version. But you can also “upgrade” the epoch of existing projects if you want to keep up.

For the most part, what changes when moving to a new epoch is that previous deprecation warnings become hard errors, or otherwise change behavior. Thus, for example, we could warn on the current epoch that an identifier in use will become a keyword in the next epoch (and start with a contextual keyword), then make the actual switch in an opt-in fashion in the next epoch.

To put it more sharply, ideally we can satisfy the constraint that, when a new epoch is released, code that compiled without warnings on the previous epoch will compile without errors, and with the same meaning, when opting into the new epoch. That’s a very strong stability story.

Now, an immediate implication of the above is that the compiler, and likely other tooling, must continue to support arbitrarily old versions of Rust. So even if we introduced a new keyword in a new epoch, when compiling under old epochs the keyword would still need to parse as an identifier. That additional maintenance burden is the cost of achieving stability without stagnation.

Epoch releases would not come every year, but rather when enough changes to Rust (including new features) have stabilized that we want to put it all together and declare a… new epoch of Rust! That would likely happen through the roadmap process.

So, if we still have to keep old code paths around, what exactly is the benefit here?

There are two main benefits:

• While deprecated styles are still permitted on old epochs, newcomers are immediately introduced to the new style, and the documentation can reflect the current epoch only. In this way, we can “clean up” aspects of the language, without breaking code.

• Grouping changes into epochs gives people an easy way to talk about the chapters in Rust development. If you think about a version of Rust with stable impl Trait, ATCs, changes to match and so on—that feels like a major step from what Rust was at 1.0. Grouping these changes into a single epoch allows us to think about these changes in coherent units, to polish up and “release” a batch of changes to the world, and to lay out a more compelling story about how Rust is evolving.

I believe that the above model would also support changes like introducing dyn and repurposing bare Trait syntax for today’s impl Trait, by doing things gradually:

• Introduce an stabilize dyn
• Deprecate bare Trait in favor of dyn
• In a new epoch, make bare Trait an error
• Later in that epoch (or a later epoch), parse bare Trait as if it was impl Trait, and deprecate impl Trait.

Relation to this pre-RFC

I think the epoch idea shares many of the goals of the original pre-RFC post, but there are a couple of key differences:

• Avoiding a major semver bump, emphasizing the fact that all of the changes are opt in (by changing your project’s epoch).

• Avoiding feature flags on the stable compiler. While that means that certain kinds of changes can only happen in an epoch shift, it simplifies the mental model of what the stable language is at any point. The pre-RFC cites Haskell as an example, but (at least in my experience) feature flags in GHC Haskell do not provide a great experience in practice; most every project ends up with a different set of flags, some of which make substantial changes to the language, making it feel like there are a lot of different languages at play.

Note that opting, on stable Rust, into a future epoch is also not a solution, because over time the changes included in an upcoming epoch may shift—which would then break code on stable Rust.

There’s a lot more detail to cover here, but I’m out of time.

Wrapup

Ok, that’s all I have time to write down just at the moment, but hopefully others from the core team can help add in some of the pieces I didn’t have time to get to. I’m hoping @nikomatsakis, in particular, can talk a little bit about the interaction with possible match changes. This is a great discussion to have now, as I said, because the evolution model will impact how we design features this year.

[yigal100]

This sounds very promising and I like the concept of a Rust epoch.

I have a couple of additional points I’d like to raise:

1. Clearly every project would need to know the epoch it belongs to and so we’d need to add a field to Cargo.toml so that cargo can pass this to rustc. But, should we have an additional finer grained resolution as well? I’m thinking about code snippets in stackoverflow, forums, blogs, etc… We can formalize this with an annotation that the compiler recognizes and so, copy-pasting examples from the web will at least compile (perhaps with warnings). We can keep it informal as it is today where people just mention the compiler version used to compile the code.
2. Should the epoch version be SemVer or not (e.g. Rust2018). I think it makes sense to decouple the language spec from the compiler version and if we use SemVer we could also be able to mark minor versions (e.g. additions of new backwards-compatible features). For example, we added the question mark operator but it won’t compile on compilers that predate the feature. If the epoch is not semver we’d need to keep two fields, the minimal compiler version that accepts the new syntax and the epoch version. it would be better to just have a Rust version so that other compilers could just support the exact spec version.

[glaebhoerl]

@aturon That sounds really good!

I just want to note that it seems like an answer to a different (though much more important!) question than the one which sparked this thread, however; questions like “how can we add new keywords (e.g. catch) or otherwise evolve the language?”. Whereas this thread started out asking, “how can we get more testing and evaluation of unstable features, especially minor ones (e.g. loop-break-value) before committing to stabilizing them and setting their behavior in stone?”. It doesn’t seem like epochs have much to offer for this latter question, though maybe I missed something?

(Incidentally, it seems like epochs also offer a really nice and clean answer to what a “semver-major” bump would mean, should we ever want to do one: simply dropping support completely for one or more “really old” epochs.)

[aturon]

Whereas this thread started out asking, “how can we get more testing and evaluation of unstable features, especially minor ones (e.g. loop-break-value) before committing to stabilizing them and setting their behavior in stone?”

It looks like some wires might’ve gotten crossed? This thread is on a pre-RFC for breaking changes/stable feature flags, not the one about unstable features.

In any case, I’m glad the proposal resonates for you! And I agree about semver-major bumps.

[glaebhoerl]

You’re right, sorry! The wires which got crossed were that allowing some form of #[feature] on stable was a proposed answer to, apparently, both questions.

[comex]

Somewhat like proposed in the pre-RFC of the original post, these epochs are basically compiler switches to opt in to a “newer version” of the language—not unlike with C++ compilers today. Crucially, cargo new would set up a project on the latest epoch, so that new code always defaults to the latest version. But you can also “upgrade” the epoch of existing projects if you want to keep up.

I feel slightly vindicated, as someone who’s wanted this for years… For example, if an “epoch” or language versioning mechanism had existed in the past, I think there would’ve been a strong case to use it for the change to default object bounds shortly after 1.0, rather than a hard break. It’s not that breaking changes aren’t disruptive even with language versioning, but if you’re going to make them anyway, it’s much nicer if all existing crates continue to compile.

I agree with @matklad that an automated upgrade tool is essential.

Also, a reminder that a similar approach could theoretically be used for library breakage as well, including (especially) trivial breakage caused by ambiguity after adding new names or impls. The standard library already marks the minimum stable version of all public named things, and the compiler could hide things with a version later than the current target version (or just disfavor them in the case of ambiguity, with a warning). However, this would obviously require more fine-grained versions than yearly epochs and more frequent upgrades, which would be annoying and is a big downside. (It’s also tricky to implement for impls, afaik.)

This would be an alternative to the commonly tossed-around suggestion of ‘elaborated source’, where before submitting a crate to crates.io (or another repository) the compiler could automatically transform the source by filling in types, trait names, and the like, making it truly stable, resilient to the aforementioned trivial breakage. Elaborated source could work, but it has the downside of not working with source control systems or other situations where you only have the original code.

I also think people underestimate the pitfalls it would have. rustc can’t type check #[cfg]-ed out blocks, so for elaboration you’d have to compile with every possible outcome of all cfg expressions in the crate, including every supported target OS and every combination of features (though theoretically the compiler could be smart and notice when features are used orthogonally in the code). Even then, there may be permutations of target cfg options that don’t currently correspond to any supported OS but might in the future. And there’s no rule that a certain bit of code has to have the same types on every target. There are typedefs like c_long which expand to different types on different OSes: for the elaborated source to be portable, the compiler would have to figure out that the ‘fully elaborated’ form is the typedef rather than the underlying type. That doesn’t sound too hard when there are explicit type annotations, but could get weird-to-impossible when type inference gets involved, or when the type determines trait selection…

It’s not impossible to deal with all this and make it work in 99% of cases, but you’d have to accept the possibility of spurious failures, and it’d probably be pretty slow. Compared to that, I don’t think explicit library versioning is all that unattractive.

[est31]

Great post!

I think it should point out though that this process should not be abused for pointless churn, like the reverting the meaning of pub proposal.

Also, what about people staying on old epochs. Will they still be able to enjoy new features? If yes, will it be a more limited set of features?

[carols10cents]

So, if we still have to keep old code paths around, what exactly is the benefit here?

This sounds like the worst of both worlds to me :-/ I’m worried about the perception that we’re somehow trying to get around our stability guarantees, and instead of doing something like staying at 1.0 with epochs, either we should keep the old code paths around forever and stay at 1.0, or we should bump the major version. There’s already a perception that lingers that we don’t really guarantee stability, and I think this proposal would reinforce those perceptions.

“Stability” means you should never dread upgrading the compiler.

I’m unclear on how this is different from what C/C++/Java do, where people do fear upgrading (and stay on, say, C89 forever because of that fear). This proposal sounds a lot like what those languages do, so how do we communicate clearly that this is different? That is, aren’t we just moving the problem from “you should never dread upgrading a 1.x version of the compiler” to needing to ensure that “you should never dread upgrading to a new epoch” if we don’t want stagnation?

Now, an immediate implication of the above is that the compiler, and likely other tooling, must continue to support arbitrarily old versions of Rust.

Another thing I’m concerned about is how this plan interacts with cargo and crates.io… how could changes in a new epoch affect those tools? How would we keep them compatible with arbitrarily old versions of Rust?

there should never be an ecosystem split.

Perhaps this is one of the things you didn’t have time to get into, but I don’t quite see how I’d be able to install and use libraries that have opted in to a newer epoch if my project is still targeting an older epoch, even if Rust supports compiling those crates together. If the response is “well this happens today already” (trying to use a crate created with Rust 1.16.0 in a project using Rust 1.0.0 won’t necessarily work), then don’t we already have a split ecosystem?

For example, how would I implement a trait from a library that is using the newer syntax and semantics around traits? Even if I can compile the crate alongside my project, what I would do currently to learn about a trait the crate provides is read the crate’s docs. The crate’s docs would be using the newer trait syntax. I would not be able to use the new syntax as a direct reference for implementing the trait in my crate using the old syntax… Would everyone staying with an older epoch have to know how to manually translate the newer epoch’s code into the older epoch’s?

[matklad]

Are there any fundamental differences between “we release new major versions of the language, but the rustc compiler will also be able to compile all older major versions and to “link” older crates into newer ones” and the epoch proposal?

[steveklabnik]

Are there any fundamental differences between “we release new major versions of the language, but the rustc compiler will also be able to compile all older major versions and to “link” older crates into newer ones” and the epoch proposal?

So, this is a good place to list my thoughts, I think

I will admit that initially, I was extremely against this proposal. I actually said to @aturon “This is my worst nightmare.” However, after working through some details, I’m coming around to the idea. And the reason why is the answer to @matklad’s question.

Rust has a reputation of being a language that changes all the time. Yes, it’s been two years of stability, but it was a lot more than that that we were breaking all the time, and people outside of Rust-world still have that impression. So, even talking about this topic introduces a degree of risk. I don’t like it.

However.

When you actually look at how languages like C, C++, and Java implement their “we don’t break ever” policies, it’s actually different than that perception. That is, there are small incompatibilities that don’t break the world, and even if they are, they offer flags so that you can get the exact semantics you want.

So, @matklad, this is the significant difference to me. This actually can be presented as an improvement of our stability story. Technically, the door for a Rust 2.0 is still open. I’d like to close that. This lets us say definitively: Rust will never have a 2.0. It’s compatible with SemVer because your old code keeps on building and working the exact same way it always did. If you don’t want to program with the latest epoch, fine. It should still all Just Work. That’s a big deal.

On a technical level, I don’t think there’s a lot of significant difference, but this is just as much, if not more, of a social problem than a technical one.

I’m worried about the perception that we’re somehow trying to get around our stability guarantees, and instead of doing something like staying at 1.0 with epochs, either we should keep the old code paths around forever and stay at 1.0, or we should bump the major version.

+1000000

For example, how would I implement a trait from a library that is using the newer syntax and semantics around traits?

This is forward compatibility, not backward. Which still is a certain kind of problem, but one we already have. If you’re on Rust 1.2 and an example uses ?, you still have to know how to use try! instead, and there are still some features you just couldn’t use until you upgrade.

[nagisa]

Okay, there have been enough of insightful comments that its plausible to begin composing this mega reply .

I would like to reiterate that this is a pre-RFC (or call-to action) and not an actual RFC. As @aturon said:

I want to make clear that these are early thoughts, and we are not yet to the proposal stage, let alone to any kind of decision.

---

@aturon

I think the epoch idea shares many of the goals of the original pre-RFC post, but there are a couple of key differences:

Okay, so to me the epochs idea sounds very similar to the idea of stable features. The major difference, in addition to the differences noted by yourself, between the two is the granularity. Whereas with epochs code’d specify an absolute point in time (or version) of the language the code targets and with the stable features proposal code’d specify the features it uses instead.

---

@glaebhoerl and @est31

Isn’t what stable #[feature] opt ins are about that their behaviour doesn’t change (being stable)? So shouldn’t testing for warts rather happen in the pre-stabilisation phase?

Whereas this thread started out asking, “how can we get more testing and evaluation of unstable features, especially minor ones (e.g. loop-break-value) before committing to stabilizing them and setting their behavior in stone?”.

You’re right, sorry! The wires which got crossed were that allowing some form of #[feature] on stable was a proposed answer to, apparently, both questions.

As @aturon already stated testing features more is not a focus of this pre-RFC. It proposes an idea on how to deal with breaking changes in a way that could be done in a compatible way. Once a change is available via stable-#[feature], it retains all stability guarantees the same way regular code does.

I fully understand where such confusion may arise, though, and will blame it on the fact that I wrote the text in a distracting environment in a day and possibly didn’t give it enough proofreading.

---

So, if we still have to keep old code paths around, what exactly is the benefit here?

But we do not need to maintain the old code around either! Instead, it is entirely plausible to just ship multiple binaries/rustcs, and the driver would invoke the pipeline for requested “epoch”/“version”/etc. Sure, this approach would increase size of the binaries, but if balanced well with keeping some old code paths around, it can make the horrors of maintaining old versions a non-problem (i.e. whenever it starts becoming a problem, fork out a new “pipeline”).

---

That is, aren’t we just moving the problem from “you should never dread upgrading a 1.x version of the compiler” to needing to ensure that “you should never dread upgrading to a new epoch” if we don’t want stagnation?

This is a fair point and I feel like as opposed to the epochs idea, the stable-features idea is better in this respect. IME the worst part of upgrading in C-like scheme is the fact that you have to port the whole code unit at once for all the features (a problem much amplified in rust, because code unit in C is a single file, unlike Rust). It is not enough to just fix compilation errors, because some code that was valid and changed could still be just as valid too! I feel like “epochs” would most likely share the same problem. Stable-features due to their granularity, would enable porting the codebase piecemeal, feature by feature, and likely on-demand.

Obviously making a tool that does it all automatically would make the point moot, but I fear that it might be plain impossible to handle all the cases (as the 2to3 (python) example shows)

[carols10cents]

This is forward compatibility, not backward. Which still is a certain kind of problem, but one we already have. If you’re on Rust 1.2 and an example uses ?, you still have to know how to use try! instead, and there are still some features you just couldn’t use until you upgrade.

Right, which was kind of my point. We already have a split ecosystem in terms of forward compatibility, so it’s too late to say “there should never be an ecosystem split”; the ecosystem is already split.

If we want to avoid splitting the ecosystem (further), we should make it as easy as possible to upgrade to the latest stable version and do the most we can to help everyone stay current, IMO, which means “never dreading an upgrade” should take priority over enabling a way for people to choose not to upgrade.