pre-RFC: LTS for the Rust Toolchain

[zackw]

If the Rust project does this, I think we need to dogfood it. One way to do that would be to say that starting from the first LTS release, non-LTS releases of the compiler and libstd can be built with whatever LTS release was current at the time they were released, as well as with some sliding window of contemporary non-LTS releases (as is currently the case).

Yes, this is a significant amount of extra work to put on the compiler team -- but work that needs doing in order for the LTS toolchain to be useful! It would, for instance, mean that std polyfills, language support for writing crate polyfills, first-class feature probes in cfg, and so on become more important project goals internally, thus aligning the project's interests better with the interests of potential LTS users.

[beepster]

Yes, this is a significant amount of extra work to put on the compiler team

Okay then we shouldn't do that. This is effectively replacing the financial motivation with technical motivation.

[zackw]

@beepster

Yes, this is a significant amount of extra work to put on the compiler team

Okay then we shouldn't do that

Does it alter your opinion if I say that I think most of the extra work is work that ought to be done anyway? And that making the compiler team dogfood the LTS, would enlarge the group who benefits from that work, to include the set of people most easily able to do that work.

[fintelia]

I think it would be helpful to clarify what ecosystem changes you're hoping will happen in response to an LTS toolchain. Like is Canonical planning to pressure uutils into making their MSRV vastly more conservative so that it always includes the most recent LTS, even if that's from nearly two years ago? What about the hundreds of crates that uutils depends on?

Stepping back, if the rule becomes that only Rust applications that support LTS Rust are allowed in any of the LTS distros, then that would apply a huge amount of pressure throughout the ecosystem to target LTS Rust for any even moderately popular library crate so that their users aren't locked out of distros.

[CAD97]

@fintelia

Like is Canonical planning to [...]

(Disclaimer: this is my understanding only and not an official representation of Canonical's interests.)

The only interest in Rust LTS at Canonical that I know of is a desire to enlarge the group of users willing to use Rust from normal risk tolerance to include low risk tolerance potential users.

Separately, uutils can have long term support even if it uses a compiler that is not itself long term support. This is why, for example, Canonical backports newer rustc releases to older Ubuntu LTS releases — to be able to compile newer security releases of software included in Ubuntu LTS. These backported compilers themselves are not covered by Ubuntu LTS except as necessary to be able to compile Ubuntu LTS and no further.

(Again: this is not intended to communicate official Canonical policy. This is just my understanding of it, and as a still quite new employee.)

Canonical very much does not want to "muscle in" and force its desires onto the Rust Project and/or community. We want to help make Rust usable in domains it currently isn't reaching for one reason or another.

I don't want to devolve into PR speak. But I, at least, am trying to do the right thing for Rust. And largely, things I would be interested in pursuing even were I not at Canonical, just with actual funding of my time.

[Vorpal]

@zackw

Yes, this is a significant amount of extra work to put on the compiler team -- but work that needs doing in order for the LTS toolchain to be useful! It would, for instance, mean that std polyfills, language support for writing crate polyfills, first-class feature probes in cfg, and so on become more important project goals internally, thus aligning the project's interests better with the interests of potential LTS users.

To me this sounds like a good reason to not do LTS at all. I dont think it is worth that cost, and it makes it much harder to dogfood nightly features, that we actually want to move forward with new features (that benefit the vast majority of users instead of a few lagging enterprise users).

[zackw]

I'm not surprised you say that. I know you don't value language features that would make it easy for people not to have to chase the highest version numbers of all their dependencies (including the language itself). I do value those features. I find your arguments against them unpersuasive. But this thread isn't the place to continue that argument.

In this thread, what I am saying is that LTS compilers would have greater value to lots of people -- not just "a few lagging enterprise users" -- if they came along with the missing language features that would make it easy for both library and binary crate authors to say "My MSRV is the most recent LTS and my minimum supported version for all my dependencies is the earliest one with that same MSRV."

Contrariwise, if the core teams aren't willing to put in the work to ship those features, and aren't willing to demonstrate their support for wide use of the LTS releases by dogfooding them as suggested above, then I don't think there's much point in Rust-the-project doing LTS releases.

[jhpratt]

I originally created it for use with the time crate to maintain a lower MSRV, but eventually decided it wasn't worthwhile given my MSRV policy. At least on crates.io, there wasn't a single other user, so I had no reason to continue maintenance. If LTS were a thing and someone funded it (even a small amount would be sufficient), I'd be happy to take this back up. I'd even be willing to backfill the versions since it was last updated.

The maintenance itself was lightweight — mostly copy-pasting stdlib code and making slight alterations as necessary. Not everything could be backported due to language constraints, which I'm sure would only be worse given the 30+ versions released in the meantime.

aside

I'm fairly certain no one outside of crates.io was using this crate either. Josh Triplett is the only person to have ever mentioned it to be after I dropped maintenance, and even then (standback#13) he didn't know if anyone was actually using the crate.

[Vorpal]

A major source of testing nightly compiler features comes from the compiler dogfooding those featurrs itself. There are very few users outside of the compiler who want to rely on nightly unless they have no other option. A big part of this is because there is no stability guarantee (which is kind of the point). As such I believe that if the compiler had to be able to build with a very old rust compiler this would massively slow down development of language features that we want (pattern types, polonius, new trait solver, ...).

Now, library features wouldn't be affected by this nearly as much, since the new std is built with the new compiler. But the compiler still would not be able to dog food those new library features.

You don't present a solution as to how the compiler would be able to continue dogfood nightly features, and this I believe is key to why rust is such a nice language to work with, and why most features feel well designed. Unlike C++ where someone dreamt up a feature, did some small scale tests, got it into a standard, and only years later when it is implemented does everyone realise the design flaws.

Now, another solution would be to make it less risky for a third party project to use unstable festures. Perhaps (and I have floated this idea earlier) we could have semi-stable features, where the rust project says “there will be something like this, and we will announce breaking changes and migration instructions with 2 releases advance notice, and we won't remove it entirely". Perhaps this could be accessible on the beta compilers, once the features are getting close to done? Still, this is probably not as good as dogfooding for finding what works well.

[bjorn3]

For library features having a backport standard library should work. When I made a branch to bootstrap from older rustc, I partially did it that way (also partially just removing the library feature uses). For language features that is not an option. Nowadays we don't use that many quickly moving language features though. Most of their uses are in the standard library I think, which doesn't matter for bootstrapping anymore.

[zackw]

@Vorpal

A major source of testing nightly compiler features comes from the compiler dogfooding those featurrs itself. There are very few users outside of the compiler who want to rely on nightly unless they have no other option

[...]

another solution would be to make it less risky for a third party project to use unstable festures

I'm very much in favor of making it less risky (and, equally important, officially encouraged) for third-party projects to use unstable features, for exactly this reason. I actually think the internal testing these features get now is inadequate, because a compiler and its runtime are each one specific kind of program and there are lots of other kinds. And I agree that in the absence of widespread third-party testing we do not want to give up the internal testing we get now.

But consider what language mechanisms we would need to make it less risky to try out unstable features: fundamentally, mechanisms that make it easy for the program to adapt to what features are and are not available. These are the same mechanisms that would make it easy for the program to support building with an old compiler as well as a bleeding edge one.

So, if we do it right, I don't think we would have to give up internal testing of nightly features in order to support building the nightly compiler using the most recent LTS.

[CAD97]

@teor

Just so we're clear, LTS only has security fixes in the compiler and tooling, based on the current threat model (for example, excluding malicious code).

I don't think I addressed this yet — my version is that LTS would only cover running tooling under the current threat model, and would not cover the behavior of code emitted by the tooling. It MAY cover security issues in the standard library; I'm not actually fully certain there.

LTS in the abstract also often includes "high-impact bug fixes." That's where soundness issues, miscompiles, etc would fall. All of this is, ultimately, a judgement call from the party providing the support in a proactive support model. Currently, my own judgement would suggest pushing fixes for "high impact stable-to-stable" (or perhaps even limited to "lts-to-lts") regressions. The ease in backporting also plays some role in determining if a backport patch is appropriate, tbqh.

@skade

An upstream LTS is "niche but broad, no liability", while a downstream compiler service is very much whatever the customer wants, so niche in a niche.

I want to explicitly second that this lines up with my understanding. Upstream LTS from the Project is actually less about the support itself but more about declaring what version downstreams should coordinate in providing long term support for. There needs to be some baseline support (e.g. backporting true CVE fixes) but once the baseline exists it's significantly easier for the ecosystem to build more thorough solutions.

@Vorpal

Even if the source is available, what could we do to encourage companies who want it to pay for it?

FWIW, there are potential legal avenues where LTS branches are made source-available but commercial usage requires buying a license. Whether the Project wants to use such an avenue is a different question.

@epage

However, this may not need to be a huge burden, especially a discrete, rather than continuous support policy.

To put another pin on this: a discrete support policy is often significantly easier to work with for conformance purposes than a continuous support policy. Even paid policies are generally continuous extensions of discrete timelines.

@bjorn3

a backport standard library

This is not a suggestion, but one potential avenue is along the -Zbuild-std direction and making std into more normal of a crate, and making std able to build with the LTS compiler for some stretch of time. This doesn't even need to be the full time that the LTS compiler has support.

I do agree that features that make it easier to support both new and older compilers are a significant boon to LTS efforts. But I don't think LTS needs to be blocked on them, rather that the initiatives can advance in parallel and each gets more useful as the other continues.

[mathstuf]

This sounds like it ties back to the "build probe" topic of a week or so ago. Sure, I might be interested in a nightly feature, but how to use it in my crate without it languishing unreleased or having to figure out how to bridge semver with it (as an MSRV can have an upper bound when a nightly feature changes).

[zackw]

Yeah, I have been meaning to start a separate thread just about that. Gotta concentrate on the day job for the rest of the afternoon though.

[CAD97]

I updated the pre-RFC in the OP:

• Reduced LTS from every two years for five years to yearly for 60 weeks.
• Minor edits to reflect the shortened LTS period.
• Structural rewrite to the "When to use LTS" section.
• Added nonfree/noncommercial licensing for LTS as an explicitly discussed alternative.

[Mark_Simulacrum]

For the intended audience of the LTS channel, do any of them actually want to consume rustup releases? One of the larger burdens this places on the Rust project is to maintain CI infrastructure able to build artifacts for an older release. We frequently have to patch our current CI to mirror some newly-flaky tarball we're fetching, deal with version changes in provided toolchains on macOS/Windows runners, etc.

I'm wondering how much it would practically lose in terms of value add to say that the LTS is provided in the form of patches to the source tarball (or a new source tarball), with no additional artifacts made available.

This would also make it much less likely that people use the LTS "just because" which seems like a reasonably good ecosystem effect.

[zackw]

@Mark_Simulacrum

This would also make it much less likely that people use the LTS "just because" which seems like a reasonably good ecosystem effect.

I would consider that exactly the opposite of my preferred outcome.

[CAD97]

@Mark_Simulacrum

For the intended audience of the LTS channel, do any of them actually want to consume rustup releases?

At the very least, I suspect the enterprise users to be capable of building the compiler for internal use if they only need to do so for "critical bug and security fixes" (i.e. what would be pushed to the LTS channel).

However, there are also library developers that want to target an LTS toolchain to enable downstreams on LTS to use the library. They can (and maybe should) use stable for most development, but it wouldn't be a great look to say they have to use a known insecure version of the toolchain to confirm compatibility with the LTS version of the toolchain.

The clippy lint for using items stabilized after the package.rust-version is a significant part of what developers get from actually testing on the older toolchain, but nothing can fully replace actually testing on that version, even if stable could fully emulate the language and library (lack of) features enabled in the older toolchain.

@zackw

I would consider [pushing ecosystem developers away from using the LTS toolchain] exactly the opposite of my preferred outcome.

The Rust project wants people using the latest stable and getting the latest in performance improvements, as well as access to newly stabilized features, telling us where pain points are so we can improve them. That's a large part of why we have frequent stable releases, as well as the strong stability guarantees around doing rustup update never^[1] breaking your code.

Providing an LTS toolchain is primarily about expanding the reach of Rust from the normal risk tolerance users to also include the low risk tolerance users. For what reason do you want to push users that would be well served by stable's "stability without stagnation" into "stagnating" on the LTS toolchain? The LTS toolchain isn't "more stable" or "more secure" than stable; it just changes less.

Users on the LTS toolchain don't need free access to the latest and greatest new library features. If they're willing to cargo update they should be willing to do rustup update as well. And if avoid rustup update they should avoid cargo update as well, and should be asking their Cargo dependencies for LTS of their libraries, not for building the latest on LTS toolchain.

I see no reasonable position which would require updating the toolchain no more frequently than an LTS timeline but which also allows feature updates of third-party libraries significantly more frequently than toolchain updates, other than "because it has always been this way" with C/C++ toolchains. But Rust's ecosystem doesn't have the amount of casual reliance on implementation details nor require the whole dependency tree to update to support the new epoch that C++ has.

There are good reasons to use LTS; that's why I'm trying to spearhead this go at discussion. The most widespread reasons I see, though, are either wanting to avoid using software not distributed by their Linux package manager of choice^[2], they justwant not to have to update as often^[3], or they want the Rust language and standard library to stop expanding^[4], all three of which I see as weak reasons in this day and age. I want to believe you have a good reason other than those, though.

I generally see wanting LTS as a symptom first, not a goal in itself. For some cases the proper resolution is to use LTS, but for a large quantity of cases it could be improving stable to meet the actual underlying requirements.

---

1. … with the usual practical caveats around name resolution, type inference, and soundness bugs. The only way to never break anything is to never change anything. Thus, LTS for the low risk tolerance users that are scared off of using stable by that possibility. ↩︎

2. In which case: why are you building and running packages from the open source ecosystem? That's not code supported by your Linux package manager vendor. ↩︎

3. If you can cargo add or cargo update, what's preventing you from rustup update? Is it needing to do full clean builds when the toolchain updates? The rebuild is annoying, sure, but you'd need to build the tree that cargo updated anyway, and the solution isn't rarely updating the toolchain, it's in further improving the compiler to reduce those costs (e.g. better cross workspace cache sharing). ↩︎

4. Using LTS won't stop stable development from continuing as it is currently. It just means you'll need to deal without new functionality or with more janky polyfill libraries until the next LTS comes about and then you'll have a lot of changes to learn about all at once. Whereas you could just as well ignore the new stuff on stable and get the same experience. ("But if my dependences use the new stuff I need to as well!" Or you could stay on the older version of the library like you would've had to on the LTS toolchain.) ↩︎

[comex]

@CAD97

• Reduced LTS from every two years for five years to yearly for 60 weeks.

Hrm, okay, but your original proposal had under Alternatives:

Shorter LTS periods

The shortest possible support timeline we should go with is three years. Rust is a foundational piece of technology used in places expecting long support timelines because of critical reliability and stability requirements. LTS needs to actually be long-term, otherwise it's not a meaningful improvement over the stable toolchain.

60 weeks is much shorter than three years. Have you changed your mind about whether this would be a meaningful improvement over the stable toolchain? Are you hoping the LTS period might be extended in the future?

Sorry to put you on the spot. I don't have strong opinions about any of this myself.

[chrisd]

Hm, we'd still need to retrain the ability to build in CI, no? We need to test patches, etc.