Idea: Semi-stabilization

[newpavlov]

Summary

In addition to stable and unstable feature statuses introduce an additional "semi-stable" status. While technically it will be equivalent to unstable, the responsible team announces that feature in question is ready for stabilization and no breaking changes are planned, so all interested parties are welcomed to use the feature and even publish crates which use it. In other words, while it's possible that breaking changes will be introduced to semi-stable feature (e.g. if serious flaw will be found), the team ensures that probability of such event is assumed very low.

Additionally make #[feature(stable_feature)] a warning, instead of compile error. This will allow crate developers to publish crates which depend on nightly, but with high probability that they will work on future Rust stable version without any changes.

Motivation

Since fundamental crates like serde or diesel started to work on stable Rust, we can see more and more signs of Nightly "demonization". Many users perceive nightly features as highly unstable and unsuited for investing their resources into using them. This in turn not only significantly reduces amount of experimentation which feature undergoes before stabilization, but also applies serious pressure on Rust teams to stabilize features (especially high-profile ones). The most notable example is pursuit to stabilize async/await (including parts of it like Pin) while most of async code (seemingly) continues to use futures v0.1, with tokio being a flagship here. See the following threads for more context:

• Reddit - Dive into anything
• Reddit - Dive into anything
• Reddit - Dive into anything

Also some stabilization RFCs argue that feature should be stabilized for it to be used more extensively, e.g. RFC 2480:

But publishing a library that uses unstable features, even optionally, comes with the expectation that it will be promptly updated whenever those features change. Some maintainers are not willing to commit to this.

I believe this trend is quite dangerous and seriously undermines Nightly experimentation model, which can be really harmful in the long-term.

Solution

Introduce semi-stabilized feature status. While this status will not add any technical guarantees over unstable status, it will provide informal indication that feature is ready for usage and it's on path to stabilization.

The easiest way of implementing this proposal will be to simply mark tracking issues with an additional tag and doing an announcement in TWiR.

A more involved approach will be to introduce #[semistable("feature_name", issue="N", version="1")] attribute. Note the version field, while enabling semi-stable feature you'll be able to provide version value (i.e. instead of #![feature("feature_name")] you will use #![feature("feature_name", version="1")]). In rare cases of breaking semi-stable feature the version number will change, thus instead of getting confusing compiler errors users will get a human readable error, which will notify them about a breaking change.

Additionally to promote usage of semi-stable features across ecosystem we could make #![feature("stabilized_feature", version="..")] a warning instead of an error on stable Rust (assuming feature got stabilized with the same version number). This way crate authors will be able to publish crates which use semi-stable features and which will work on future stable Rust version. For example tokio will be able to publish v0.2 which will use std futures and if no serious flaws will be found, tokio v0.2 will work on future stable Rust without any additional changes.

Another way to promote use of semi-stable features is to create an informal rule: after feature was semi-stabilized it will be automatically fully stabilized in ~3 releases if no serious flaws were detected. And if we want to be more conservative we could even require semi-stabilization step for every feature.

Yes, there is a small risk that crate authors will have to bear maintenance burden if breaking changes will be introduced to semi-stable feature, but I believe it's a much smaller price to pay than risk of stabilizing a flawed feature.

P.S.: Reddit discussion of this proposal can be found here.

[kornel]

Yes, I think some middle ground is needed. Currently features are only either “unfinished, don’t use it” or “done, frozen and final forever”.

Additionally, opting in to nightly compiler not only opts in to use of not-yet-stabilized features, but to work-in-progress compiler (to be fair, nightlies are generally remarkably stable, but I’m uneasy about never knowing what rustup update may bring).

When I started with Rust I was just toying with it, so I didn’t mind using cutting edge features, but now I’m using Rust for real, so I’m more risk averse.

I’d happily use new features that are said to be mostly done, won’t change in a way that can’t be easily fixed in projects using them, and are likely to stabilize in a few months.

[JohnTitor]

Sounds good to me. It’ll reduce pain when we use new features, I think.

[Centril]

@newpavlov

Additionally make #[feature(stable_feature)] a warning, instead of compile error.

This would make feature gating itself part of the language and thus its specification. That would add a whole lot of complexity for any formalization. I'm not inclined to support this.

Ideas about alpha channels or marking certain features as less likely to change seem fine tho.

[RustyYato]

@Centril

This would make feature gating itself part of the language and thus its specification

How so? This looks like an implementation detail of the compiler to me, not an extension to the language.

[Centril]

@RustyYato

How so? This looks like an implementation detail of the compiler to me, not an extension to the language.

If it is allowed on stable then we cannot break #![feature(the_stable_feature)] and so it is not an implementation detail of the compiler.

[newpavlov]

One way to side-step this issue is to simply warn on any #[feature(..)] in stable Rust, without any analysis of enabled features. It will be less convenient and a bit more confusing, but it will achieve approximately the same results.

[Centril]

That reduces the complexity somewhat but you are still fundamentally adding #[feature(..)] to the language as a stable attribute. I think the social aspect of what you desire is more important: we should communicate more clearly what we think is near stabilization and what is not. This can be done through other means…

• a list on some polished webpage
• a way to expressly disable non-semi-stable features: #![only_semi_stable_features_bikeshed_me]

[scottmcm]

@newpavlov

In other words, while it’s possible that breaking changes will be introduced to semi-stable feature (e.g. if serious flaw will be found), the team ensures that probability of such event is assumed very low.

It's unclear to me how this differs from just being stable. If a bunch of code is written using it and it's published to crates.io, then it's de-facto stable.

[newpavlov]

How about adding #![enable_semi_stable_features] attribute which will enable all semi-stable features and will warn on stable? Yes, it's still a stable language attribute and we lose granurality, but for stable channel it's just a no-op (plus a warning), so it shouldn't hinder formalization in any way.

I believe it's important to be able to publish crates which use semi-stable features and which in future will work on stable without any changes. Let's take tokio as the most evident example, without a version which uses std futures published on crates.io many project will not even think about using std futures. And I doubt tokio developers would want to publish v0.2.0 which forever will require Nightly. Yes, it's maybe possible to convince them "just publish v0.2 and yank it after futures get stabilized", but I don't think it will be frictionless. (cc @carllerche)

Also on reddit it was suggested to tie semi-stabilization to beta channel, though I am not sure how exactly it could look like.

@scottmcm

It’s unclear to me how this differs from just being stable. If a bunch of code is written using it and it’s published to crates.io, then it’s de-facto stable.

Not sure if I understand your point, you already can publish Nightly-dependent crates and other crates can depend on them, so I don't see how it makes them "de-facto stable".

[Centril]

@newpavlov

so it shouldn’t hinder formalization in any way.

This is true, but it still makes the notion of features a part of the stable language.

If this was an alpha channel with a weekly cadence or something like that where stability was not guaranteed and only semi-stable features were enabled I could see a case for that. It would need a commitment to not regard alpha as stable by all involved parties.

I think a more fruitful path forward is to have #[unstable(semistable = true, feature = "foo", issue = "0")] or something of that general form and then a way to ensure that you don't use features that aren't semi-stable on nightly/alpha.

@newpavlov

Let’s take tokio as the most evident example, without a version which uses std futures published on crates.io many project will not even think about using std futures.

I think you overestimate how much this would help this case. std::future::Future cannot be regarded as semi-stable right now.

@newpavlov

Not sure if I understand your point, you already can publish Nightly-dependent crates and other crates can depend on them, so I don’t see how it makes them “de-facto stable”.

There's a social "contract" where everyone involved agrees that unstable nightly features can break.

[newpavlov]

@Centril

This is true, but it still makes the notion of features a part of the stable language.

Why? If we'll forget about the name, this attribute on stable will not have any connection with features, for language specification it will be just "this attribute produces a warning and does not do anything else".

I think there may be a misunderstanding, semi-stable features will not be accessible on stable channel. In other words until feature is fully stabilized you still will have to use nigthly (or maybe also beta) to work with it. The only difference from today's nightly features is that there is an insurance that probability of breakage is extremely low (but non-zero nevertheless) and that crates which use semi-stable features will work (with high probability) on future stable release without any changes.

@Centril

I think you overestimate how much this would help this case. std::future::Future cannot be regarded as semi-stable right now.

Yes, I understand. But I am not talking about "right now", but about "future". After reading the posts linked in the OP I worry that as soon as futures will be regarded as "more or less stable" they will be pushed to stabilization without proper testing across ecosystem.

@Centril

There’s a social “contract” where everyone involved agrees that unstable nightly features can break.

Absolutely the same applies to semi-stable features as well, the only difference is probability of breakage.

[Centril]

@newpavlov

for language specification it will be just “this attribute produces a warning and does not do anything else”.

Yes sure, I understand that... but it feels distinctly awkward to have an attribute that has no static semantics in the language.

Why would you use it anyway if it does nothing on stable? Crates should be using #![cfg_attr(feature = "unstable", feature(foobar))] anyways. The only value I could see this having is if you use a nightly-only crate on stable and suddenly get a warning. This would serve as an update that something has been stabilized. We do however already warn on nightly when a feature gate is no longer necessary.

@newpavlov

The only difference from today’s nightly features is that there is an insurance that probability of breakage is extremely low (but non-zero nevertheless) and that crates which use semi-stable features will work (with high probability) on future stable release without any changes.

I think using cfg(accessible(::foo::bar)) and cfg(version(1.XX)) from https://github.com/rust-lang/rfcs/pull/2523 to conditionally work with newer compilers is a better solution to this overall.

@newpavlov

After reading the posts linked in the OP I worry that as soon as futures will be regarded as “more or less stable” they will be pushed to stabilization without proper testing across ecosystem.

I see how #![cfg_attr(any(feature = "alpha", feature = "nightly"), enable_semiunstable_features)] on an alpha channel could help this but I think for futures it will take too much time to set this up and get people used to it. I think #![only_allow_semistable_features] as a means of preventing dependence on more than semistable features in your application is a decent idea as long as it only exists on nightly.

[newpavlov]

@Centril

Why would you use it anyway if it does nothing on stable?

Because it will be a leftover from the time when your crate was developed on nightly. Of course after true stabilization release on the first update of your crate you should remove this attribute.

Imagine chain of crates A->B->C->D which depend on a single semi-stable feature (e.g. std futures or const generics), with the approach advocated by me on feature stabilization all crates will automatically work on stable, with your approach they'll have to go through painfull process of migrating the whole chain on stable. Also adding nightly/unstable feature to crates can be quite nontrivial, or even impossible if it significantly changes public API, and many crate authors simply choose to not support nightly features altogether because of that. See for example the following PR which I've encountered recently: https://github.com/carllerche/bytes/pull/153 I am sure we can find many similar examples.

@Centril

I think using cfg(accessible(::foo::bar)) and cfg(version(1.XX)) from https://github.com/rust-lang/rfcs/pull/2523 to conditionally work with newer compilers is a better solution to this overall.

It will work if feature is an inner implementation detail, but not if it (directly or indirectly) changes public API. Let's take digest crate as an example, on const-generics semi-stabilization I would be happy to publish next version of the crate which will fully utilize const generics, but with the current state of things I'l have to wait until feature true stabilization.

@Centril

I think #![only_allow_semistable_features] as a means of preventing dependence on more than semistable features in your application is a decent idea as long as it only exists on nightly.

I think you miss the point regarding importance of feature testing across ecosystem. Your approach will work for simple test crates, but not for projects which aim to do real work with a feature.

Imagine I have a web-app which depends on tokio, actix and who knows how many other crates which may use futures/generators/etc. I would like to utilize power of async/await language integration as soon as possible, but I (and maintainers of foundational crates) don't want to chase constant nightly changes. For it to work all foundational crates should utilize std futures, today and with your approach the only reasonable choice is to wait true stabilization, which is obviously quite sub-optimal for many reasons.

[Centril]

@newpavlov

with the approach advocated by me on feature stabilization all crates will automatically work on stable

You'll need to explain why this is the case for anything but nightly-only crates.

@newpavlov

with your approach they’ll have to go through painfull process of migrating the whole chain on stable.

Ideally crates that use cfg_attr(feature = "unstable", feature(foo)) will "just work" on stable if no changes are actually made to the semi-stable feature foo. If foo does go through changes, then things will break in either case. So I don't see how your assertion is correct either about your approach or mine. Even for nightly-only crates that use no feature = "unstable" they can simply conditionally compile the feature gate itself but avoid conditional complication of the uses of said feature.

Meanwhile version(..) and accessible(..) actually extend the number of stable versions a crate works with and does not need to break folks using older compiler versions.

@newpavlov

It will work if feature is an inner implementation detail, but not if it (directly or indirectly) changes public API.

Only if you do changes that causes errors before cfg-stripping occurs (read: changes to the grammar). An example of that is #[cfg(nope)] fn foo<T: Iterator<Item: Copy>>() {} or indeed const generics. Otherwise it is fine and will work for changes to public APIs. Fortunately, the futures API does not introduce new syntax.

@newpavlov

I think you miss the point regarding importance of feature testing across ecosystem.

Not so. I just doubt the degree to which your solution fixes the problem without adding to stagnation.

@newpavlov

Your approach will work for simple test crates, but not for projects which aim to do real work with a feature.

@kornel expressed that they would be willing to use features that are unlikely to substantially change so I don't see how this applies to them at least.

@newpavlov

I would like to utilize power of async/await language integration as soon as possible, but I (and maintainers of foundational crates) don’t want to chase constant nightly changes.

This is fundamentally a hard problem. Many people want stable things yesterday but don't want to contribute to experimentation and testing. We want to offer stability but no stagnation. Having your cake and eat it is difficult when there are so many conflicting goals. There's also a question of fairness. I think that if some people are pushing a certain feature that is important to them then they should be part of its testing, especially if they have the financial means to do so (read: the companies taking advantage of all the free labour that comes with FOSS work).

[kornel]

How about not using #feature, but instead adding default-deny warnings on use of a semi stable feature, so allow(feature-name) is the opt in?

[bill_myers]

I think the defining characteristic of semi-stable features should be that the Rust project is committed to stabilizing the feature in a timely manner, providing essentially all the current functionality, even if details change. Thus crates using such a feature can be expected to both continue working on new nightlies and also stop relying on non-stable features relatively soon, as long as the maintainer updates the crate when the compiler changes.

I think there should be a “semi-stable” opt-in in Cargo.toml that would be needed if you either use the feature flags or depend on other semi-stable crates. Likewise, there could be a similar opt-in for all unstable features, which would also allow to use stable releases to compile crates using unstable features. Obviously automatic CI tests on crates.io would consider failure to compile rates that are flagged in either of this way as mere warnings and not block compiler releases.

[notriddle]

This is sounding dangerously close to CSS vendor prefixes. They were kind of a dismal failure all around.

For an unstable attribute to have any teeth at all, random libraries can’t opt into them. It’s fine if the semi-stable is available on a stable release of rustc, but it’s not okay if an application developer can accidentally end up requiring a semi-stable feature to work because some library up there in the dependency tree depends on it.

[kornel]

Vendor prefixes were problematic for other vendors, and mainly due to unstable syntax being intentionally different than the stable one. Rust is being evolved by a single “vendor” for now, and semi-stable features are supposed to use the intended stable syntax, so these problems aren’t directly relevant.

But it’s an interesting point about libraries. Having a dependency of a dependency break is painful. So maybe the opt-in could be limited to binary crates?

[newpavlov]

How about enabling semi-stable features in Cargo.toml as was proposed by @bill_myers? This way we will keep notion of semi-stable features outside of stable language and will be able to use semi-stable feature versions for nice errors. Plus you will be able to quickly understand semi-stable features used by a crate.

In other words, something like this:

[package]
name = "foo"
version = "0.1.0"

[lang-features]
async = "1"
const-generics = "1"

Yes, cargo will have to keep table of stable and semi-stable features, but at the first glance it does not look like a too complex addition.

Stable toolchain will check versions of enabled lang features and if those feature/version pairs were indeed stabilized will emit a warning and will continue as usual, otherwise it will return compilation error.

Maybe we could even use it for enabling unstable features as well, e.g. by using foo = "0" to indicate instability.