[pre-rfc] Stable features for breaking changes

[leonardo]

@nikomatsakis

Where things get a bit tricky is are things like bits of public API. In that case, there could be Epoch 1 crates that (e.g.) have a public type named catch that is not typeable in Epoch 2. This could be circumvented by introducing some form of "escape" that allows identifiers with more creative names (e.g., I think that scala uses backticks for this purpose).

And C# uses "@keword".

[nikomatsakis]

@anon19897381

Another concern I have is - how do we teach this to new users? How do I explain to a friend how match works? Do I say "hey, if you're using epoch 2017, you need this ref to borrow the object rather than take ownership, but if you're using epoch 2018 then ref is implicit"? What if they copy some code from StackOverflow and it doesn't compile?

This is definitely a key concern, and it's one of the reason that I favor something like epochs -- that is, a kind of "big release" that groups together a number of changes. This makes it easier to categorize code samples and things as "older code". It also means you can label stackoverflow answers with a simple epoch number to give context and so forth.

I think another big factor here of course is error messages. That is, if the problem is the code not compiling, we can presumably recognize patterns from older epochs and give a very helpful error (much like we used to do when removing obsolete syntax etc).

[colin_kiegel]

Can we have epoch code names (like android/ubuntu)?

This feature will become available in the bronze epoch.

[Ixrec]

We could name the epochs after different crustaceans, then have different logos (also like Android), so there’d be a Crayfish Ferris and a Lobster Ferris and a Fiddler Crab Ferris and a…anyway.

Some serious points:

I believe a large portion of the perceived instability of Rust simply comes from the amount of Rust code that’s still stuck on unstable, and the unavoidable fact that no one’s had Rust code in production for several years yet. I’m not sure there’s a clear action item there, other than maybe trying to reduce the number of features in “unstable limbo”, or trying not to design and implement features that we aren’t going to be pushing to stabilize in the relatively near future (or are needed to make progress on stabilizing other stuff).

I completely agree that languages like C++ do make plenty of minor breaking changes in practice, and that “epochs” as described here are pretty much exactly what every other serious language seems to be doing. I’m totally on board with the idea of doing something similar for Rust, especially if the “breaking changes” are as minor and/or desirable as the examples given so far.

But I did have one concern about the impl/bare/dyn Trait syntax example (which I would love to see actually happen btw). For features which have a significant impact on the public APIs that libraries present to their clients, I think it’s desirable for there to be one syntax that libraries can use such that none of their clients, on any past or future epoch, get warnings, errors or bugs. In this particular example, I think achieving that is as simple as not deprecating the “impl Trait” syntax, which seems fine to me since the plan is merely to make the “impl” keyword redundant, not to repurpose it after we’ve redefined bare Trait.

[milesrout]

I’m not sure how much experience people here have writing Haskell, but a problem that has been growing and growing and growing lately is this at the start of every file:

{-# LANGUAGE GADTs #-}
{-# LANGUAGE RecursiveDo #-}
{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE ViewPatterns #-}
{-# LANGUAGE PatternSynonyms #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE ExistentialQuantification #-}
{-# LANGUAGE PartialTypeSignatures #-}
{-# LANGUAGE DeriveGeneric, DeriveFoldable, DeriveTraversable #-}
{-# LANGUAGE FunctionalDependencies, MultiParamTypeClasses #-}
{-# LANGUAGE TemplateHaskell #-}

Now that’s not to say that language extensions and #[feature] are a bad idea. But they do post the risk of ending up, in a couple of years, with every non-trivial project starting with a list of a dozen features and changes etc. etc.

Has Perl-style use v5.20 been considered?

[glaebhoerl]

@milesrout

Has Perl-style use v5.20 been considered?

This is the epochs proposal.

@withoutboats

Whatever we call them, I think we can't support making changes as deep as the changes to the string type. But all of the changes we've seriously considered are, in my opinion, in line with the kinds of changes that are made in upgrades to the C standard.

However, our current compatibility guarantee is much firmer than what C++ version upgrades provide; we can't even introduce new non-contextual reserved words. This makes sense - C++ upgrades every 3 years & we upgrade every 6 weeks. But its quite difficult when there's this misalignment in perception, where our 1.X upgrades are treated as equivalent to C++ upgrades, and what we've been talking about as "breaking changes" are treated as equivalent to Python 3, when they're really equivalent to C++11.

I think this point is quite valid for things like adding new keywords (catch etc.), and probably the change to match semantics as well, but... less so for changing the meaning of &Trait (or banning it or whatever). Not sure what difference this makes to your argument, just wanted to point it out.

[withoutboats]

I don’t agree. We can change the meaning of a syntax by doing a ‘pivot’ - first we ban it and replace it with a more explicit form (‘dyn Trait’ in this case). This is a mechanical change.

Then at some point after the epoch has started, when most code is moved over, we introduce a new meaning for the syntax.

Even if we didn’t wait, running rustfix would still be a mechanical change (because rustfix knows what epoch you’re in, so it knows what that syntax meant). There’s just a risk if you try to do it manually.

[glaebhoerl]

I… wasn’t disputing any of that. My point was that it’s not the sort of thing they tend to do in new versions of the C or C++ standards.

[withoutboats]

That might be true, but to me the difference isn’t really “what C++ has done” vs “what Python has done” but “will people dread upgrading across this?” If its a purely mechanical change, the answer to me seems to be no.

[FraGag]

There are a few things I’d like to have clarified to make sure we’re all on the same page.

1. Will each crate specify its own epoch, or will the root project specify the epoch and its dependencies inherit it?

So far, I’m assuming the former, because the latter would imply that libraries would have to be written in such a way that they are compatible with every epoch that they decide to support. If it’s the former, then…

2. What kind of changes can and cannot be made in a epoch?

or the collorary:

2a. If a library updates its epoch, is it (allowed to be) a breaking change for the library (i.e. does the library need to bump its major version)?

If the answer is yes, then library authors need to be aware of this. Bumping the major version means that applications on an older epoch will be able to continue using the older major version of a library without fearing that cargo update will update the library to a version that is incompatible with the application’s epoch.

I think we already have this issue today with Rust releases: if a library starts using Rust 1.y features without a major version bump, and an application that uses that library is stuck on Rust 1.x, where x < y, then cargo update will update the library to a version that is not compatible with Rust 1.x and it will not compile. If we eventually logic to cargo update to not upgrade a library to a version that requires (based on an attribute added in Cargo.toml, perhaps) a newer version of Rust than the currently installed version, then we could do something similar for epochs to avoid imposing major version bumps on library.

The answer to this question will have an impact on the answer to the next question (specifically scenario 3.2):

3. What is the interaction between crates using different epochs?

There are two scenarios. Let’s suppose application A uses library B.

3.1. Application A uses a newer epoch than library B. 3.2. Application A uses an older epoch than library B.

In scenario 3.1, library B might be using features or syntax that has become deprecated or that changed semantics in the epoch that application A uses. If these changes don’t affect the library’s API (e.g. the proposed change to match), there’s no issue. But if they do, then in order for this scenario to work, there needs to be an alternate feature or syntax that application A can use in order to use library B.

For example, let’s consider the semantic change to &Trait. Suppose that library B defines these traits:

pub trait Foo {
    pub fn foo(self, x: &Bar);
}

pub trait Bar {}

If application A wants to implement Foo for one of its types, it will be able to, but it will have to use the new syntax:

impl Foo for MyFoo {
    pub fn foo(self, x: &dyn Bar);
}

(If it used &Bar, it would fail to compile because the declaration wouldn’t be compatible with the trait’s.)

This might be a problem if library B exports a macro that expands to an impl Foo for $x: presumably, the macro would be written with the old syntax. If you use that macro in application A, which epoch should the expanded code be compiled under? In order to maintain compatibility, we’d have to compile the expanded code with library B’s epoch. I don’t know if that’s easy of even feasible.

Scenario 3.2 is essentially scenario 3.1 backwards, but now instead of worrying about forward compatibility, we’d be worrying about backward compatibility. If new epochs are allowed to introduce features that code using an older epoch cannot consume, then it may force applications to upgrade their epoch in order to keep using the library. Coming back to question 2, if a library updates its epoch and has the consequence of forcing downstream crates to update their epoch, then that’s a breaking change. Now, if the library doesn’t use any of the features that are backwards-incompatible, then it’s not a breaking change. If the library then evolves and adds a new feature that depends on a feature only available in the new epoch, it will force downstream crates to upgrade only if they want to use that feature.

[ubsan]

Random thoughts from IRC:

I would like our current epoch, the default epoch for code without epoch markers, to keep updating at a reasonable pace. I propose that every LTS is a good pace; perhaps we switch the epoch over on LTS releases, so that people can use a stable compiler with the latest standard.

[carols10cents]

@seanmonstar

Coming from a server development position, I can say that in many places that I've worked at (even at Mozilla, as forward reaching with technology as we are), it's very common to not keep up with the latest version of a language. We actually specifically do not want to run the latest release in production. We want all horrible bugs and vulnerabilities to be patched out of new features before we ship that stuff. So, it's actually quite common for us to upgrade huge versions at a time, often times from one LTS to the next. It'd be sad if someone in our position felt we couldn't upgrade because then we'd need to do all this busywork fixing up our code for the newest compiler.

@ubsan

I propose that every LTS is a good pace; perhaps we switch the epoch over on LTS releases, so that people can use a stable compiler with the latest standard.

So Rust doesn't currently have a Long Term Support version planned, and from what I understand of this proposal, epochs are not the same as LTS releases either. Is that correct? Would the rust compiler (or anything in the ecosystem) ever get patches affecting an older epoch? (I would assume yes because bugs happen, but how would this work and be communicated?)

If we're supporting every 1.x version on every epoch on every platform, what does that do to the CI build matrix over time? Or rather, obviously it adds another dimension, but is that sustainable?

Is having an LTS version under consideration currently as well, and if so, how do yinz envision LTS and epochs interacting?

[cuviper]

I think LTS and epochs are separate ideas. LTS is about maintaining older version of the compiler; epochs are about making breaking transitions in new compilers without losing compatibility with older code. The LTS compiler doesn’t need to worry about any features/epochs that came after its release, and the new compilers should continue supporting old code regardless of the existence of LTS.

[steveklabnik]

@carols10cents

Is that correct?

Yeah, this says nothing about LTS. I'm still very interested in an LTS release.

@carols10cents

Would the rust compiler (or anything in the ecosystem) ever get patches affecting an older epoch? (I would assume yes because bugs happen, but how would this work and be communicated?)

So, with the "epochs == flags" idea, older epochs would still be getting patches, that is, there's still only one compiler, and it knows how to build all epochs. So any fixes for stuff not specific to epoch-related flags is gonna get improved on all epochs. Stuff that new flags replace, on the other hand...

@carols10cents

Is having an LTS version under consideration currently as well, and if so, how do yinz envision LTS and epochs interacting?

I think this is a good, and very important question.

[steveklabnik]

https://isocpp.org/files/papers/p0636r0.html

Here’s what’s different from C++14 -> C++17. Note removed or deprecated features, for example.

[petrochenkov]

C++ standard explicitly permits implementations to support removed library features (and language features too in some cases), which the implementations are surely going to do for years. Rust, unlike C++, is an implementation, so the strategy taken by C++ is not of much help to it.

[Ixrec]

Part of the proposal is that newer Rust compilers would continue to support older epochs, otherwise “epochs” would be no different from a major version bump (it’s even been suggested that Rust would never need a major version bump if epochs happened), so C++'s strategy is very relevant.

[JeffBurdges]

There are uses for stable features besides breaking changes too. You might want to disable non-lexical lifetimes for teaching purposes, for example.

[Ericson2314]

Ah, finally found the epochs thread. Overall, I’m very glad this is happening—breaking changes to the surface langauge are inevitable. The use of years not versions strikes me a bit as a marketing gimmick, but I always avoided python and thus don’t have scars from it’s 2 to 3 transition.

One detail, that I assume will be fleshed out but would like to kick off discussing now, is the pipeline of unstable features into epochs. Currently all our unstable features are non-breaking, but with epochs we can start have breaking unstable features. I’m thinking the nightly should have a “next epoch” to queue up breaking unstable features we are considering stabilizing.

Less clear to me is how this works with the “all errors were warnings” rule. Do we need two next epochs? Separate warn and err versions of breaking unstable features?

Put differently, maybe it’s better to keep things simple on stable, but nightly will have to do much of what this RFC proposes either way.

[Ixrec]

One other question worth bringing up soon-ish is how libraries are supposed to support multiple epochs, if at all. I can’t think of any option besides the simplistic #[cfg(epoch=...)] answer, but we clearly need to decide if this is the sort of code we want people to write when impl Trait finally stabilizes:

trait MyTrait {
    // speaking of marketing gimmicks...
    #![cfg(epoch = "hermit-crab")]
    fn foo() -> MyIterType {}

    #![cfg(epoch = "mantis-shrimp")]
    fn foo() -> impl Iterator {}
}

We should probably make an actual epochs thread at this point.