Blog Post: Stability without stressing the !@#! out

internals policy
1

https://smallcultfollowing.com/babysteps/blog/2023/09/18/stability-without-stressing-the-out/

9 Likes
2

As someone who works on a production code base written in an unstable language, this makes a lot of sense to me.

There are certainly a category of users who actually don't mind small or moderate breaking changes, so currently Rust loses value on both:

  • giving features to these users earlier,

  • getting more feedback.

Nightly doesn't serve this use case well. Some production code bases don't mind churn, but they still require correctness. Nightly doesn't make a clear enough distinction between features which fully work, but are not yet stabilized, and features with gaps in implementation. A category for things that work as advertised but that might still change is genuinely useful.

I am not sure about this specific formulation for previews:

We are commited to keeping this functionality in some form, but we reserve the right to make changes.

The problem here is that this might prevent us from previewing features even if we think there is only a tiny chance that the feature doesn't work out.

What's more, "some form" is a weasel word here, which might not necessary be useful for the prospective users of the feature. Some form might end up being significantly worse than a proper feature, which would tip the trade off in favor of refactoring the code to avoid dependency on the (ersatz) feature, so in practice it might be equivalent to outright removal.

I think it would be more useful to phrase previews in terms of probabilities and risks. Preview features are fundamentally about revealing missing information, so they are inherently risky, and it makes sense to make that explicit in the manual. So, my definition of preview would be:

  • Preview features are production ready and work as advertised.
  • They don't come with guaranteed stability --- they way they work might change.
  • Non the less, instability is bounded in expectation.
  • There's 0.5 probability that a preview feature will see a minor change, something which could break the build, but is cargo-fixable, or is just genuinely unlikely to actually occure in practice.
  • There's 0.1 probability of a moderate change, which requires manual intervention at a significant amount of call-sites.
  • There's 0.01 probability of outright removal. If that is to happen, a generous ramp-off period of at least a couple of years will be provided

There's of course a risk that this will split the ecosystem into "stable" and "preview" dialects, but I won't worry about that much:

  • the crates.io is big enough and stable enough that it seems unlikely for "preview" subset to grow too large.
  • even if it does become a problem, we could fix it by terminating the preview process (that is, we stop preview for new features, and just wait until all current previews stabilize).
11 Likes
3

An example of a feature that could be in preview in cargo is MSRV-aware resolver. Its cheap to implement (though we still need to figure out some UX aspects of it). The hard part will be fixing the error messages and some behavior corner cases. This feature has a long road to GA but it'd be fine for people who are ok with some rough edges.

7 Likes
4

There's already a lot of Rust code that is nightly only, for reasons that are only "we want to use this unstable feature". If there was a preview channel:

  • Some people currently using nightly would switch to preview, and get more implementation stability.
  • Some people currently using stable because they don't want nightly bugs would switch to preview and start writing preview-only code.

The factor determining how much this would split the ecosystem, more than it already is, is how many people in the latter category would start publishing their preview-only code to crates.io and would otherwise have either not published or published stable-only code.

I suspect that this would not be a significant problem, because people already use nightly because they really want the features, and generally don't seem to be put off by the instability. If there was a significant worsening of this existing split, it would be because are significant numbers of people who care about stability/lack-of-bugs, but not as much as Rust stable does, and are also writing libraries that other people want to use.

1 Like
5

I strongly support this idea. All nightly features I use are those almost stable but blocked by one or two design choices I don't really care.

3 Likes
6

Preview features are usable on stable, but with opt-in:

  • Every project that uses any preview features, or which depends on crates that use preview features, must include preview-features = true in their Cargo.toml.

I think we can do better than that. A crate can declare whether it can still compile at all without preview features. I.e. preview-features = {off, if-all-supported, required}. Then a drownstream user can try to compile the crate with a rust version and see if that (still) works and if not just disable previews and get the baseline version. Rustc would check all the #![feature(...)] attributes and see if it understands them in the current version, if if-all-supported is passed then it generates a cfg() that indicates whether the requested features are supported so that APIs that require preview features can be gated behind that.

That's basically a combination of the nightly feature that many crates have combined with feature-detection. It does not solve the problem where a downstream crate wants to use an API of an upstream crate that uses preview features that don't compile because the if-all-supported logic determined they aren't supported... but as long as all crates use if-all-supported rather than required that still leaves the option to build a non-preview version of the dependency graph. Or maybe the preview switch should act more like a cargo feature in general. I.e. if a downstream crate depends on crate-APIs which depend on preview features then it must declare that too.

And if any crate has a required flag then cargo should at least warn about that since there's a small chance it might stop building in the future. Or even not-so-small chance if we have API breaks in previews and need to version them (e.g. a feature(afit_v2) or whatever) and it takes time for the libraries to update to the new version.

2 Likes
7

Iā€™ll put in that Iā€™m currently using nightly at work because of features we need (-Zpretty=expanded), but thereā€™s basically no rules we can use to choose a good nightly, as long as it built the components and targets we use. Of course nightlies are pretty high quality, but weā€™d still prefer to use a compiler built from the stable branch to do our nightly things, because at least thereā€™s a few weeks of extra stabilization there.

2 Likes
8

This seems pretty reasonable, there are a lot of things that suit 95% of use cases but can't become stable until it's near 100. I have also wondered about how well tested a lot of features are if they are really only tested by the people who go out of their way to try it out with a nonproduction project.

It seems reasonable that providing a timeline to stability should be a requirement to bring features to the preview level. Intending to fully stabilize in 4-5 releases seems reasonable, but it also seems very easy to slip into the perma-unstable land if there aren't concrete goals in place.

Mechanics wise, I'm not sure if the stable gating would be able to handle this or if we would need a rustc_stable. Anything at the preview level will probably require the same #!feature(some_feature)] / -Zsome-feature to opt in, just this could be usable without nightly.

#[stable(feature = "some_feature", since = "1.2.3", level = "preview")]

// or we could merge `unstable` into the same attribute
#[stability(feature = "other_feature", tracking_issue = "1234", level = "unstable")]
#[stability(feature = "some_feature", since = "1.2.3", level = "preview")]
#[stability(feature = "other_feature", since = "1.2.3", level = "stable")]
9

I don't think preview features should use #![feature], they should be clearly distinct. I really like the idea though. Making the status of features more transparent is something I fully support and have tried to improve myself with the internal_features distinction.

10
  • Every project that uses any preview features, or which depends on crates that use preview features, must include preview-features = true in their Cargo.toml.
  • Every crate that directly uses preview features must additionally include the appropriate feature gates.

I worry that this will result in the same sort of "oh, you're using nightly to compile? let me helpfully enable some features for you" design that causes issues like

I think it would be better if dependent crates must also list the features that their dependencies use, something akin to -Zallowed-features but probably in the cargo metadata. (I would also love to transition nightly features to something similar, except for special cases like std that should be allowed to activate any feature).

11

IMHO, this "preview" mode should be able to be enabled with a feature. Similar to the idea of unstable feature from stable/unstable/nightly-only features Ā· Issue #10881 Ā· rust-lang/cargo Ā· GitHub

Eg, i'd like my library crate to build on stable and be stable. But i'd like my user to be able to enable a feature such a way that my crate can offer more functionality in an unstable way because that'd need some feature in preview and that it would not be semver-compatible

Example syntax

[features]
async_traits = { unstable = true, requires = ["rust:preview"] }

where "rust:preview" is a fake unstable feature of the rust language itself that enables the preview mode of the compiler. Since it is in itself an unstable feature, it can only be used from unstable features.

3 Likes
12

What happens if, despite the best of intentions, a preview feature gets stalled? Perhaps someone's life circumstances changed. Or perhaps their boss had previously given them time to work on the feature, but once it was available as a preview, their boss decided that that was good enough and reassigned them to something considered more pressing. Or perhaps they just lost interest. One way or another, the bandwidth is no longer there, and nobody else is volunteering to pick up the slack.

Requiring a timeline for stability, as suggested in other posts in this thread, would help. And focusing on making feature development a team effort would help increase the bus factor.

And I suppose it's already the case that features can't get past the RFC stage without a long campaign and a persistent campaigner or campaigners. Someone who has already put in that effort is more likely to stay until to the finish line.

Still, there are no guarantees. As I see it, "open source works incrementally" and "follow-up never gets done" are two sides of the same coin. You can't have one without the other. The blog post defines "open source works incrementally" by saying that if people are "scratching their own itch", then "the pace of progress and polish increases dramatically". But if you rely on that phenomenon to drive progress, you necessarily run the risk that people stop feeling itchy and the follow-up never gets done.

That said, I support this idea anyway. It's not news that the way Rust artificially scares people away from unstable features (by intentionally tying them to a perceived-unstable compiler version) has become increasingly detrimental to testing those features, due to the ecosystem becoming more focused on stable over many years.

4 Likes
13

Macros by example! I think this is an example of a preview-quality feature which got stalled and never got completed.

That is to say, I think itā€™s ok not to worry about this case to much ā€” it likely that some feature will get stuck in preview limbo, but itā€™s unlikely to be a major problem, as there are many ways out:

  • just keep the feature in preview, if it is small and nobody cares
  • stabilize as is, if the feature is actually being used
  • find a champion to complete it, if there are strong feelings about the feature not being ready
  • If it is completely stuck, use ā€œ1% for outright removalsā€ budget to axe the feature (or attract a strongly motivated pusher-over-the-finish-line)

This last point is actually exactly why came to conclusion that ā€œremoval of preview featureā€ should be a thing! I also was thinking about adding some stabilization timeline, but I think that wonā€™t work ā€” adding deadlines doesnā€™t really help with open source problem of intermittent availability of resources. A ā€œpressure release valveā€ would work though!

Preview limbo might become a problem if many features systematically get stuck there, but that I think we can (slowly) fix, if needed, by tapering out the whole process top-down.

14

I'm going to go one step further and say that as a application and library developer using rust, nightly might as well not exist.

However putting on my embedded developer hat, nightly does still exist, but only out of necessity in core dependencies in the ecosystem. The day I can write my embedded code without needing to mess with nightly toolchains I will absolutely go to stable. And even today I don't really need to interact directly with nightly features in the embedded space, only via dependencies.

As such, this sort of in-between of a preview that likely won't see radical breaking changes (and with paths for migration when things do change) seems like a great idea to me.

3 Likes