Currently, Rust has three release channels: stable, beta, and nightly. Stable's role is obvious, but beta and nightly each serve distinct use-cases:
- Beta: Give the next stable release a try before it become stable to catch stable regressions.
- Nightly: 1) Expose nightly features so users can test them out if they need them — this allows gathering evidence for an eventual stabilization. 2) Expose the latest compiler changes, whether those are new stabilizations, compiler improvements, standard library changes, etc. — this allows changes to bake with a small subset of users (those who use nightly) before they land for a wider audience (like on beta).
I believe there is an important use-case that is not addressed by the existing channels, and which would benefit both Rust itself and its users: wide-scale testing of specific unstable features. By way of example, consider a company that wishes to contribute an
aardvark feature to Rust. The company's engineers submit an implementation PR (and maybe an RFC) which lands under the
aardvark feature flag. At this point, the Rust maintainers (rightly) want evidence for the feature's correctness, as well as experience data for how well this implementation of the feature works in practice. They want to know if the feature is broken, or if the API is janky, restrictive, or too hard to use, so that they can make an evidence-backed decision as to whether the feature should eventually be stabilized in its current form. The company engineers want to help by testing the new feature internally, which would provide both a large and diversified sample of real-world use.
Here, a problem arises. The developers face a problem: how do they test this feature internally? They essentially have three options:
- Make the nightly compiler available internally.
- Wait for the next beta release and set
RUSTC_BOOTSTRAP=1in their build environment.
- Hope that the feature will stabilize without their involvement.
These options are all bad. The last option (3) means little evidence will be provided for stabilization, which means either the stabilization goes ahead with scant supporting data, or the stabilization falls through due to lack of evidence, which may mean Rust misses out on an important feature. The middle option (2) takes advantage of an obscure hack that is really not intended for public use, and which the Rust maintainers discourage the use of. In particular,
RUSTC_BOOTSTRAP violates the expectation that the stable compiler (and beta since it'll become stable) is stable and thus does not allow unstable features, and is mostly a historical relic that should arguably be removed. The first option (1) would mean also opting into very recent code-changes that haven't undergone as much battle-testing as a
stable release, and which may have as-yet undiscovered issues that can only be fixed by upgrading to the next
nightly. Which may in turn have similar issues that require another upgrade, etc. While Rust's
nightly channel is fairly stable in practice, it's likely not going to fly with the company's internal security team, or with the engineers' desire to not be regularly paged for nightly breakages.
The issue that arises here is that
nightly includes both "new code" and "unstable features" into a single release channel. There is no way to get the latter without the former, which is really what the company engineers would need in order to test out the unstable feature they contributed in a reasonable way (likely combined with
-Zallow-features=aardvark to ensure only that feature is used).
And so we get to my suggestion for a new release channel. Let's call it
testing for the sake of exposition. I propose
testing should have the following properties:
- It should allow unstable features. Without this, this release channel wouldn't serve much of a purpose.
It should be identical to the current
beta. This ensures that the code that lands in the release is fairly stable, while not forcing the company engineers to wait ~9 weeks to be able to test their new feature. It also means that fixes will be backported into this release channel when they are backported into the beta, which is likely to make large users more willing to adopt it. Hooking this off of beta also reduces the cost of maintaining the channel, since no separate release train is needed.
It should require
-Zallow-features. This is probably somewhat controversial. I believe this channel should be specifically for testing particular unstable features, not a general replacement for
nightly. Users who are actively working on a feature, or who are relying on a large number of features, should continue using
nightly— that way, we keep bug reports and such about nightly features current in the majority of cases. It also encourages "good use" of
testingfor adopters who may not already have been aware of
-Zallow-features— it is unlikely that they intended to blanket allow all current unstable features, whatever they are.
The list of available features should be curated and time-limited. That is, there should be an explicit process (probably
rfcbot) for making a given unstable feature accessible on testing. This goes further to ensure that testing does not become a replacement for nightly, and that only features that are deemed "ready to test" are actually tested. It also reduces the possibility for weird interactions among unstable features since the set is controlled. The set of available features should be time-limited so that a feature does not live in testing forever, and put pressure on a decision ultimately being made. Josh proposed eight weeks, which I like. That is a maximum time — a feature can be pulled and stabilized sooner. I think there should be a recommendation to delay removal from testing until a stabilized feature is available on beta, but that's secondary.
This is not the first time this has been proposed in one form or another. Some related reading: