Getting more testing of unstable features

[aturon]

Cargo only resolves dependencies such that downstream’s feature whitelist is respected

OK, that’s sounding a bit more plausible. We’ve sometimes talked about something similar for Rust version requirements. IIRC, @wycats had some concerns, but I don’t recall what they were.

[sfackler]

Another slightly simpler option would be to have releases of the exact same code as the stable release but with nightly features turned on - “1.16.0-unstable” or whatever. While I don’t think this would drive use in libraries all that much, it could help drive usage for people working on “end products”.

For example, I have a server at work I’m building, and we currently use the latest stable release. There are a couple of library and maybe language features here and there that we may pick up before stabilization, but I don’t feel comfortable jumping onto the nightly train itself. It’s easier to say “use the latest stable build” compared to “use the 2017-03-02 nightly”. Stable builds are also a bit less buggy since we’ve had 6 weeks to find bugs and backport fixes to beta. Knowing that I’d only have to go in and fix things due to changes in unstable features once every 6 weeks helps as well.

[petrochenkov]

It’s easier to say “use the latest stable build” compared to “use the 2017-03-02 nightly”. Stable builds are also a bit less buggy since we’ve had 6 weeks to find bugs and backport fixes to beta. Knowing that I’d only have to go in and fix things due to changes in unstable features once every 6 weeks helps as well.

I know this is not in line with the official position, but for these exact reasons I recommended using stable with enabled nightly features (RUSTC_BOOTSTRAP=1 or whatever works at the moment) to my colleagues in the past, as an alternative to pinned nightlies. It’s basically best from both worlds - you have a well defined version, which is reliable and has all regression fixes backported, and you are working mostly on stable, but if you need that one intrinsic you can have it as well.

[colin_kiegel]

Imagine this wonderful scenario:

Library author sees/designs cool new feature Library author makes release candidate using new feature Downstream tries out new RC, and reports back to library author yay/nay Library author reports back to Rust yay/nay on new feature Rust team, with this evidence, stabilizes the feature with confidence RC becomes actual release of library Experimental branches downstream can be merged; more cautious users can try out new library version + Rust version with confidence.

… whatever we do, we want there to be a clear notion of “fully stable Rust” which allows you to update the compiler and semver-compatible versions of libraries without fear of breakage.

I think it just occurred to me - or as we’d say in German: I just had a Geistesblitz . BUT it’s still very restrictive, which is a plus IMO.

• allow opting into unstable features on the stable toolchain
• reject publishing of crates on crates.io that use unstable features on the stable toolchain

It would be permissive and restrictive at the same time, which could really be a sweet spot.

The details could roughly look like this

• Introduce a flag unstable in the [package] section of Cargo.toml which defaults to false.

• rustc: #![...] will work unconditionally, but emit a lint

warning: The feature foo is not yet stabilized and may behave unexpectedly. It is also subject to change and might be removed completely in the next Rust version.

• cargo: #![...] with the default unstable = false will fail to build

error: Usage of unstable features in crates is only allowed if you add a flag unstable = true to the [package] section of Cargo.toml, or if you use the nightly toolchain. Please note that unstable crates can not be published to crates.io.

• cargo: #![...] with unstable = true would compile and everything, with the warning lints from rustc - but fail to publish

error: Crates which use unstable features on the stable toolchain can not be published to crates.io. Please remove the flag unstable = true from Cargo.toml in order to publish.

• cargo: #![...] with the default unstable = false would publish, but might emit a new warning

warning: Your crate uses unstable features and requires a nightly toolchain to compile. It is recommended to publish packages which compile on the stable toolchain. Are you sure you want to proceed?

• cargo: Unstable dependencies will fail to build if unstable = false

error: The dependency foo i marked as unstable and may stop working with future versions of Rust. Depending on unstable crates is only allowed if you add a flag unstable = true to the [package] section of Cargo.toml, or if you use the nightly toolchain. Please note that unstable crates can not be published to crates.io.

It’ll be possible to

• have an experimental crate with some pre-release git branch on github, but not published on crates.io - (which is what @Ericson2314 is aiming for, right?)
• other crates can opt into depending on that crate, but they would also be excluded from crates.io, because they can’t pull in dependencies from outside of crates.io

The little dance with unstable = true seems necessary to allow publishing of crates which target nightly (and may use unstable features), while preventing crates targeting stable to use unstable features (at least on crates.io).

Discussion

• There would be a clear incentive to avoid unstable features on the stable toolchain in order to participate in the ecosystem on crates.io.
• We would not introduce dependency hell on crates.io, but only on externally hosted packages, e.g. at github.
• Even if a large fraction of the community decides to use unstable features on the stable toolchain, we would still have a crystal clear separation: crates.io ecosystem with stability guarantees vs. the privately hosted no-risk-no-fun badlands. Everyone could pick their lot.

Thoughts?

[colin_kiegel]

PS: I still think the stable toolchain has a very significant timelag if you just want to get feedback on unstable features…

[Letheed]

Should we add the concept of release channel to crates? That would allow crate authors to release both stable and unstable crates, and upload them to crates.io. Stable would be the default and a crate marked as stable would forbid unstable dependencies. One could add an unstable dependency or switch from a stable to an unstable one to try out new features or a newer untested version of the software. Unstable features would be allowed on the stable compiler but would only compile if the crate is marked as unstable. This would obviously make releasing, and getting feedback on new features/APIs/etc. easier beyond just the scope of rust unstable features.

[colin_kiegel]

Technically speaking that should be possible. But I don’t have a good feeling about making “unstable on stable” too comfortable or even first-class on crates.io to be honest.

Quite fundamentally it seems wrong to me to encourage people to publish something inherently unstable.

The important insight I had today is that only the publishing aspect is problematic, building something unstable is not, or at least not as much.

And it’s actually good to have a reasonably strong incentive not to publish unstable crates for the stable toolchain. Otherwise we would end up with a domino effect, where each crate switching to this “unstable” channel would increase the incentive for other crates to follow along.

Too many regular consumers of unstable features would also mean more friction in developing Rust and ultimately less new features per year. We should optimize the development process of Rust for a high throughput of features, which includes minimizing friction when changing/removing unstable features.

I think it is important to find a sweet spot with just enough feedback for the Rust team, but as little extra burden as possible.

---

Yes we will still have to wait 52 weeks until Christmas, but we will get more presents / year.

[Letheed]

I’m not sure I see “a domino effect” happening. Looking at what we have today, people strive to avoid using unsafe and to release their software on the stable toolchain. Given how quickly chances of breakage would increase over time on an unstable channel, I can see people staying away from it.

Unstable features are slow moving targets, and that’s good. I don’t see the problem with having them on the stable compiler as long as you can’t release stable software with them.

One downside is that it would probably reduce the amount of use/testing that rust-nightly gets.

[cuviper]

I know this is not in line with the official position, but for these exact reasons I recommended using stable with enabled nightly features (RUSTC_BOOTSTRAP=1 or whatever works at the moment) to my colleagues in the past, as an alternative to pinned nightlies.

Yeah, this is basically all that a “1.16.0-unstable” would amount to, just flipping the default in the same code that checks RUSTC_BOOTSTRAP.

[tshepang]

For one to use an unstable feature, they need to add an annotation at the beginning of the source file. That makes it very clear that you are taking some risk. All Rust developers, I believe, will be aware of that risk, so I think this should not be a concern really.

[kornel]

Another angle could be to split unstable features into truly unstable and almost stable:

• experimental — unbaked features that could be removed, rust’s internal hacks that will never be stabilized, and generally unreliable stuff.

• stabilization-candidate — features that are intended to land in stable in the next one or two releases, but might have minor changes still.

The difference is in difficulty of fixing the breakage. If a feature is removed that’s a big trouble for authors and may require a significant rewrite. But if the feature is only tweaked a bit before becoming stable, that’s much more acceptable and likely to be fixed quickly.

[tshepang]

This adds complexity and needless work. I think a better indicator is the tracking issue for the unstable feature you are interested in, for it may give some idea of when the feature may get stabilised. Also, I don’t imagine ‘experiments’ go as far as reaching master, but are instead done in forks.

[mystor]

We can break all crates which currently require unstable features, so let’s focus on consumers of non-nightly compilers:

• We don’t want to allow you to accidentally depend on unstable features
• We want to make it clear to consumers of crates which are unstable that they are using unstable features.

So, we do something like the following:

• Remove the distinction between nightly and stable rust with regard to unstable features - all versions of rust are only differentiated based on how frequently they are updated.
• Make rustc refuse to build code with unstable features or accept unstable compiler flags without the --unstable-features command-line flag.
• Make rlibs built with --unstable-features enabled be marked as unstable. Require --unstable-features to one of these crates with extern crate.
• Add a new flag, unstable, to the [package] (bikesheddable) section of Cargo.toml. When this flag is set, --unstable-features will be passed to rustc when building build.rs and the crate in question.
• Add a pseudo-feature unstable to Cargo.toml which can be listed as a feature dependency (for example, new-feature = [ "unstable" ]). When this pseudo-feature is enabled, it will be as though the unstable flag on the crate being built is set to true
• When publishing a crate to crates.io, if it has the flag unstable = true enabled on a particular version, add a badge next to the name noting that this version of the crate depends on unstable rust features.
  • Perhaps cause dependency resolution to ignore these versions when unstable = true is not set? I don’t think this is a good idea because it could cause some really confusing dependency resolution issues - so I would favour it just breaking the build, but you never know.

Notes:

• We might want to add a filter option to crates.io which allows filtering out unstable crates.
• The unstable badge could be colored based on how old the version published was, or if we record the version of rustc used to publish the crate, whether or not that version of rustc is recent.
• Instead of an unstable = true flag, we could accept a list, which allows listing features which are depended upon, such as unstable = ["specialization"].
• We could expose a cfg(unstable) feature which would be set when --unstable-features was passed to rustc.
• Unstable crates which are uploaded to crates.io could be marked by the rustc version which they were uploaded with, for example stable 1.16, beta 1.17, or nightly 20-03-2017
  • Would we want dependency resolution to take this info into account? I’m not sure how much work we want to put into making this work nicely for people across release versions. Unstable stuff is unstable.
  • This date could be displayed on the badge on crates.io?

This has the advantage of making it easier and probably more possible to experiment with new unstable features. I don’t think it should create too much of a dependence on unstable features in the ecosystem however, because it makes it very visible that you’re using unstable features.

It will probably greatly drop the number of people using the nightly release channel, as many people who are only using nightly so that they can use unstable features when writing experimental code will probably switch to stable to get access to a more stable environment of unstable crates.

[colin_kiegel]

I still believe publishing unstable crates to crates.io is not a good idea:

• I think it sets a quality bar if unstable crates are rejected by crates.io
• It puts less pressure on the rust team not to break unstable crates
• We don’t need filters on crates.io: “Yes there is a package foo, but oh, it’s unstable”
• It avoids advertising to opt into unstable. Oh look, yet another crate with shiny bonus features - all I need is opt into unstable… Ok, you got me now
• It avoids an ecosystem split on crates.io where a significant fraction of packages might opt-into unstable, leading to a domino effect
• Human beings are known to be incredibly bad at objective risk assessment. And it’s a won’tfix. ^^ This unstable seems pretty stable to me, it didn’t break for 30 weeks, can I use it in production now? No, sorry…
• It makes it even more explicit that unstable crates have a much different level of support and guarantees
• Last but not least it’s really easy enough to depend on crates hosted on github or somewhere else. And if that seems too unstable, then unstable isn’t for you. IMO.

[kornel]

There’s already #[feature()] syntax that opts-in to unstable features. Maybe that can be reused instead of new flags and configs?

[kornel]

Maybe unstable packages could be allowed on crates.io, but would be ranked lower than stable crates?

So if there are stable alternatives, they’d be promoted first. But if there are no alternatives, then the unstable crates would be still discoverable.

[mystor]

We already allow unstable crates on crates.io - they just are unmarked. If you depend on them and happen to use a nightly compiler, you might not even notice that you were depending on unstable features until someone tried to build your crate with the stable compiler. The major changes here are:

A) Instead of downloading a different toolchain to use the crate you add a flag to Cargo.toml, and B) Unstable crates are now clearly marked as unstable, and you can’t accidentally depend on them even if you’re uring a nightly compiler.

I think in order to increase testing of unstable features it is essential to allow publishing of unstable crates in some form or another, because otherwise, people won’t be able to easily use / experiment with the new unstable APIs.

@kornel: #![feature()] (and the program text in general) is not visible to cargo, while this new flag in Cargo.toml would be. It’s really important that cargo can see it so that we can make this information easily visible on crates.io. In addition, this would make it more clear from looking at feature flags what features will require unstable to build, rather than needing to run the compiler first and discover you need to write #![feature(unstable_dependencies)] to build.

[Storyyeller]

The talk about hiding/demoting unstable crates on crates.io seems to assume that stability is a binary attribute. How do you handle crates with optional unstable functionality?

[kornel]

Another idea:

allow depending on an unstable crate if it has a matching version that is compatible with stable

In case the unstable version of a dependency breaks, there will always be an easy way to downgrade to a version that works on stable. This can even be automatic based on cargo settings or stability.

Authors will be able to add unstable features to existing crates as long as they don’t break (semver) compatibility with older versions.

Users will be able to set minimum required version to the most recent stable-compatible version, and take advantage of lax semver match to get a newer unstable version where possible.

[aturon]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.