Btw, I'm trying to think of a team that wouldn't have significant extra work caused by an LTS. There's libs, cargo, etc. Infra and bootstrap would have more work to do. And security of course.
2 Likes
This is the most reasonable position I've seen from anyone proposing an LTS. Yes, this is exactly right: if we can have an LTS that only affects the people who self-select into it, that uses funding from the people who feel they need an LTS to do all the work, that doesn't push anyone to use it who otherwise wouldn't have, and that doesn't tempt or pressure anyone in the ecosystem to change their MSRV policy, that seems completely reasonable.
3 Likes
How is that going to work given that most rustc developers att volunteers who work on what interest them[1] (if I understand the process correctly)? This then relies on someone from each team being interested in supporting LTSes.
Or their employer potentially. ↩︎
Yeah, but I doubt that is how it will play out. Sure, it might for some corperations (but probably not given what I have seen in industry). But Debian etc will want to ship an LTS because that is what they do, what they have always done. And then there will be users complaining[1] "why do you need such a new compiler, I can't use this on stable Debian".
Maybe libraries will be somewhat sheltered from this (thanks to the MSRV aware resolver), but applications certainly won't. And this is why we can't have nice things.
Possibly whining even ↩︎
2 Likes
Hm, this is indeed an interesting point. However, I think there is a big difference between maintaining CI for all tier 1 targets (or all tier 2+ targets) and maintaining just enough that people can reasonably test against the LTS. For example, keeping just a subset of Linux builds (x86_64-unknown-linux-gnu for example) alive would likely make it much easier on the project to keep the toolchains building for a long period of time, and should be sufficient that 80-90% of libraries (anything not platform-specific) can test against the LTS. And platform-specific libraries may well want to test on the LTS platform since they likely want to interoperate more closely with that platform.
This is essentially also a response to:
1 Like
Pretty much yes. I'm working with other RCN members to draft a survey to send to the survey team on what support timelines prospective LTS customers would expect and how much they'd expect to pay for it, which will be used to inform future revisions before posting the proper RFC.
But the most significant hurdle and the most impactful part of official LTS is having it at all, mostly independent of the exact timeframe, because once it exists it becomes a onboarding ramp for
- Users who think they want LTS but will discover that updating the Rust Toolchain isn't onerous at all and switch to stable;
- Users who would be willing to use stable except for requirements for only using tooling that publishes a support policy with defined end of support deadlines;
- Users who need LTS and are willing to pay for any extended support as long as the bean counters can trivially find proof that support is available; and
- Companies who want to collaborate with the Rust Project to provide LTS but don't want to sell "Vendor Rust LTS" without coordination with the Project.
I do think still think that a support duration with at least a year where two releases are supported in parallel is a necessity for LTS to be taken seriously. But what changed is that I realized there's no real requirement for the forward edge to be LTS, because it can be the stable channel instead.
I previously considered every two years for three years to be the most conservative option that could reasonably satisfy LTS expectations. In this setup, an established LTS user stays on a given LTS channel for two years and is required to upgrade to the next LTS channel once that release has been around for a year. Node's new policy proves there's appetite for yearly LTS upgrades, though (otherwise they wouldn't be doing it).
Under the yearly LTS for 60 weeks plan, my imagined LTS user uses the LTS toolchain for any production builds of their application, as their risk tolerance threshold doesn't allow them to update their build stack at the rate stable requires them to in order to stay in support, but they're also testing on the stable toolchain continuously, reporting regressions to the upstream project and preparing to make their update guaranteed smooth in practice without blocking LTS work. This seems like a nice compromise between the positions of "stable has long term support and is sufficiently stable for any reasonable requirements" and "we don't risk adopting freshly released software until it's been battle-tested and had the bugs you only find in production ironed out."
Yes, this "ideal LTS user" doesn't follow the policy for using LTS outlined in the pre-RFC since they're doing active development targeting the LTS channel. IMHO, the user that actually needs LTS already knows what their LTS usage plan looks like; we don't need to tell them and we aren't going to change their minds. But the people who don't strictly need LTS but think that they want LTS because it's somehow "more stable" or they use LTS in other ecosystems deserve guidance on if they actually need to use LTS. That's the crowd that the guidelines are aimed at — users who default to LTS but are willing to learn once they've made the jump to consider using Rust.
While we would need to run tests, we could drop some of what's only needed for distribution builds.
I like the idea of adding a new "Tier 0" target support level for "LTS Tier". Stable support stays the same, but on the LTS channel everything except LTS targets are considered tier 3 with no official support.
It's sort of already the case that using a toolchain older than latest stable means you effectively have tier 3 support (pithily "it probably used to work and it probably still does"). So "introducing LTS for the Rust toolchain" could just as well be "introducing the new LTS target tier".
The requirements for LTS Tier being roughly
- Everything in Tier 1 with Host Tools
- The target and its CI must have long term support
- The build environment for Rust must not impose new mandatory requirements for building within the LTS timeline
- (Maybe: some company has guaranteed monetary sponsorship of LTS for that target for at least the LTS period through the Foundation)
I think every top level team would in practice be at least potentially impacted by providing LTS, with the libs-api maybe being an exception.
Thank you. This is why I volunteered to spearhead this attempt to drive discussion; I hope that because I was a Rust Project member first[1] and now am an employee of Canonical where I can directly observe some of the enterprise pressures behind wanting LTS that I can meditate and translate between two very different audiences successfully.
The strictest sense of this, unfortunately, is definitionally impossible. There mere existence of a supported toolchain will cause social pressures for a library to support that toolchain along lines of "I want everybody using a supported toolchain to be able to use my library, because that gets me more users." I would argue that pressure already exists in part, as said potential users probably already care about a conservative MSRV policy already, but LTS will be more visible and legitimate with project backing (which is the entire point of the project endorsing an LTS toolchain).
What we can so is make it abundantly clear when an ask for targeting LTS is a valid ask, and that OSS maintainers don't owe anyone LTS support even for major ecosystem packages. I think I'm doing an okay job at riding that line so far.
Though potentially it'd be a good idea to add a note in the guide section that the existence of LTS does not suggest that "packages targeting stable" should adopt an MSRV of the LTS toolchain, and that they're encouraged to continue increasing their MSRV (as a patch or minor level change as appropriate) ignoring the presence of LTS (except w.r.t. version bumping).
Ideally, guaranteed funding through the Foundation for work supporting LTS backports and releases. If we can't get that, whatever process has previously created some backports for older toolchains when we've patched CVEs in the past.
I can't speak to what people will do in the future. Exceptions to "don't update things" exist in distros already for highly security critical packages like the browser (e.g. Firefox). Despite being even more security critical, compilers don't traditionally have that kind of exception to stable inclusion because historically GCC and systems software hasn't ever really required it[2].
I have some hope that we can work with distros to justify giving /usr/bin/rustc a similar exception to ship newer Rust compilers because of our strong commitment to stable backwards compatibility[3], as well as coordinating on ways to encourage users to use the Rust toolchain in multiversion-aware ways, even if not rustup, instead of just /usr/bin/rustc.
People rarely use the system Python nowadays; they use virtualenv, conda, or some other multiversion support more advanced than /usr/bin/python2 vs /usr/bin/python3. The response to "why doesn't this work with the system Rust" should be showing the user the multiversioned approach and why everyone uses it. It could even be managed by e.g. apt install rustup-toolchain-$channel for the purists who don't want to install software from anywhere else than the OS distribution[4].
The downside of "essaying" multi-response posts: a new reply coming in, while you were writing a later part, that would've loved to see something earlier in the post that was already finished.
I honestly think that only proving LTS Tier support for x86_64-unknown-linux-gnu is a really good choice for balancing providing LTS access to the community and requiring enterprise to pay to rely on LTS. Put a requirement on CI overhead to the project on the LTS Tier policy along with LTS maintainer bandwidth and you have a solid mostly objective reason to include a target in LTS Tier or not.
I'm excited that every time I come back to new posts in this thread I get ideas for how to further improve the pre-RFC. That's a sign of this being a productive discussion.
And fully on the side of "why do you need LTS" at that. ↩︎
Assuming your LTS distro isn't updating to the new feature updates from upstream, of course, and is instead backporting the security fixes themselves. ↩︎
Unstable ABI is a complaint some distros have due to wanting to force Rust into a C-shaped hole by requiring dynamic linking, but people are starting to move away from that as a requirement (eg flatpacks, snaps). And as far as rustc plugins go, requirements for stability from features upstream doesn't support for external usage is ridiculous IMO. Just don't allow mixing eg rustc and clippy from different upstream distribution versions. ↩︎
Weird take in today's era, but okay. ↩︎
4 Likes
FWIW, at Red Hat we ship every Rust update in Fedora, and rebase every ~6 months in RHEL too. We also do this for the entire Clang/LLVM suite.
(RHEL does freeze /usr/bin/gcc, but provides newer gcc-toolset-N alternatives too.)
3 Likes
I think a potential benefit to having an official LTS policy is that, for maintainers who already try to support older versions of Rust, now it becomes a good coordination point for what version to use.
Right now, if I want to support people who don't keep to the latest stable, my general workflow is:
- Start with some older Rust version as the designated minimum.
- If I accidentally use a feature newer than that minimum, vibe out the tradeoff of bumping to that newer version vs. rewriting to no longer use that feature.
If there was an official LTS version, then it becomes an easy decision to say "MSRV is at most the oldest/newest active LTS version" and now there's a solid rule for when I can bump the version to use new features. I haven't publicly released anything big enough to get people to whine about my MSRV policies yet, but I'd expect an official LTS would help reduce that for projects which stick to it (though maybe at the cost of some whining at projects which don't support the LTS version).
3 Likes
I'd encourage you to include a probing question or two about if/how customers are planning to get LTS support from ecosystem crates. Please don't assume that companies understand that checks made out to the Rust Foundation won't translate into any money reaching crate maintainers.
My understanding is that the backporting work involves working closely with whichever volunteer(s) handled the fix that's being backported. Which isn't a setup that seems particularly easy to compensate the specific people doing the work. But if people start feeling that the Foundation is "selling" the free labor of Project members, that would be a really destructive outcome.
2 Likes
I think there's a flaw in your reasoning here: I use Rust from my distro package manager. But, in the rare cases where I use third-party dependencies, I get those from my distro package manager too (rather than from crates.io). There's no reason a distro can't package Rust crates, and some of them do.
(On the other hand, I generally try to use the most recent version of Rust available in the package manager, to get at new features, so am probably not in the target audience for an LTS – although I also try to avoid using new features if I don't need them, so that the program will work on older rustc if necessary for whatever reason. This has become important on some past occasions where I switch to a computer that is using an older operating system and thus only has older versions of Rust available; in particular I was writing my programs in 2021 edition because not every computer I used had a rustc new enough to recognise 2024 edition.)
It's worth noting that this policy makes it impossible for me to work on rustc itself (because it doesn't build using older Rust executables, and in fact isn't actually written in the same Rust everyone else gets to use – in order to build rustc with a stable compiler, you need to set an environment variable that makes it behave more like a nightly).
1 Like
We have an MSRV-aware resolver now. Use the latest stable; people who are running LTS will automatically get compatible versions of crates. If you want new crates, use new Rust.
One thing we could do is have extremely clear documentation, along the lines of: "The MSRV-aware resolver will ensure that users running older versions of Rust (including LTS versions) will automatically get compatible versions of your crate. You should not do any additional work to support LTS versions. If you receive requests to support LTS versions, and do not have a paid support contract to support them, we encourage you to decline."
(Alternatively/additionally, we could make some arrangement to provide support for crates willing to do that work, supported by the LTS funding.)
5 Likes
I don't have the necessary accumen to design such a program, but:
If we assume that the Rust Foundation will sell some kind of LTS contracts somehow, that funding should absolutely be earmarked for maintenance. First pick goes to the people contributing to the toolchain LTS of course, but the cost of proactive LTS is orthogonal to the number of LTS customers, and thus revenue. Once there's enough in this maintenance fund that LTS maintenance is always covered (big if, I know), the rest of the money should be distributed to the larger ecosystem in some manner.
A program where developers can sign up to receive LTS funding and potentially offer support contracts through the Rust Foundation as part of "Ecosystem Rust LTS" would be awesome. It's been argued many times that a significant part of the OSS funding issue is bridging between the users who want support and the many many spread out developers that work on OSS tech stacks. But I have no early idea how such a system would be managed,,, any of the details really.
I'd say this is a solid future possibility enabled by LTS funding, though. I hope that if we can make it to that scale, we can find people who know how to design and run such a system (and are willing to do so on a limited compensation plan).
1 Like
I think we shouldn't limit it to having host tools, it seems reasonable to me to support some tier 1 targets without host tools (e.g. a target for the Linux kernel or some embedded firmware or even something like i686-unknown-linux-gnu) and just say what we support is cross-compiling from some other LTS Tier target.
2 Likes
The bottom of State of the Rust/Cargo crates ecosystem // Lib.rs has some stats on rustc versions used, I found this a pretty good way to determine MSRV policy. I go for N-1 or N-2 (so pretty aggressive) but dont bump unless I need to due to dependencies or I want some newer feature.
That said, I also have a counter point to LTS rallying. It means using newer features in libraries will take longer. So this means less nice things for us who track stable. Libraries were pretty quick to adapt to async in traits when that stabilised (my recollection was "a couple of releases") but now it might be a year or two instead if they track LTS.
A better approach is to rely on the MSRV aware resolver to give the lagging LTS users an older version of the library, and let stable users get the features they want. What about security fixes for LTS users? Well, IMO you shouldn't bother unless you are getting paid to do so, but you could always backport if someone yells enough or is willing to put in the work and do it for you. Just make sure to leave a gap in version numbers by bumping minor on MSRV bump if you care about this.
3 Likes
Well, that might mean some rustc contributors simply say they aren't interested in doing LTS work (for whatever reason, for example, if it is a hobby for them they might not want to spend their limited time on that). If there isn't a LTS team to pick that up, then what? And if the compiler changed a lot in between that patch might be non-trivial to backport and depending on where in the compiler it might require very niche knowledge.
Speaking of which, if unsoundness fixes are going to be backported, is that even possible generally? We know there are several unsoundness cases that the new trait solver will fix, that really aren't fixable without that multi-year work. Something like that can't be backported to an LTS. Now those unsoundness tend to be very hard to hit in normal code, but what if we have a case in the future that is more reachable and can't be backported?
One thing I hope we do someday is adding infinite subdivision to version numbers, so that there's always a version between any two versions.
We could also build it against a current glibc 
So I'm a bit confused here. How does this help with getting corporate LTS users to pay? Would non-x86_64 Linux only be available to those who are willing to pay? If it isn't available at all (thus saving on CI maintenance work), there is no economic incentive. If it is available, you still need to spend time and effort on CI maintenance. And even if that is paid for, that is time and effort that talented developers could have spent on something more useful and productive to stable users.[1]
Also, if builds are available for other platforms to those who pay, there is another issue: Th e whole thing will be moot because Debian and Ubuntu[2] will have builds of the LTS toolchain. (As will RHEL, RockyLinux, AlmaLinux, etc.) So getting the aarch64 Linux toolchain will also be trivial without paying. And Debian targets many more obscure platforms (ppc etc). That leaves just Windows and Mac builds as potential income sources to cover the costs.[3]
(As much as I dislike the concept of LTS: What I also don't get is why you dont simply use a frozen in time container image as the base for building a hypothetical LTS toolchain, then it shouldn't be affected by updates to the underlying CI platform, as long as you can run the container. I'm out of the loop on Windows and Mac, but surely they have something like that by now?)
"You can't always get what you want"?
The closest I've seen for macOS is tart which still runs full VMs. Windows has containers, but they are so constrained that we don't use them (e.g., max disk size is baked in at image build time, "magic" base images to install Visual Studio, severe performance issues, etc.). Some places I've griped about it to mentioned that they went with WINE and/or mingw-based cross-compilation instead.