@sylvestre can maybe talk more about how the Debian team decides when to upgrade?
I think a good start to improve the story for linux distro users would be to improve the packaging situation of rustup, which currently is only packaged by a fraction of distros. People are legitimately afraid of curl | sh
, as it might have unwanted side effects. E.g. some installers install stuff from custom ppas which are a nightmare to deal with every time you upgrade your distro version, just to give one example. Allowing a apt install rustup && cargo install foo
workflow would be much nicer. But this is something that's outside of the rust project's powers, as it needs the distro to do the packaging.
All the rust project can do is try to convince the distros that packaging rustup is the best for their users, as due to how the crates.io
ecosystem works, cargo install
is broken for many people with the distro provided rustc
. E.g. this is an existing thread in the debian bug tracker where packaging rustup is being discussed.
Beyond that, resolver support for rust-version
would be a good idea, but I'm not sure it's going to be enough to provide a smooth experience, as one project that updates their MSRV but doesn't opt into rust-version
, or keeps an outdated record of it, is enough to break it for users. Single projects use hundreds if not thousands of crates from a large number of authors, and one dependency is enough to break cargo install
.
And there is also the other effect to consider that if version resolution was made dependent on rust-version
, and crate authors disagreed with this choice, they might be deliberately removing rust-version
from their future releases, of even make it inacurate. Convincing a large fraction of the crates.io
ecosystem to put correct rust-version
numbers might be possible, but I doubt we'll get all crate authors on board, especially as many have the expectations that all their users use the latest rustc. Also, users using outdated versions of their software generally means a burden on maintainers they might not be willing to bear, e.g. they file reports for bugs already fixed in newer versions. Having to close them as dupes might not be something the crate authors want to do, and we would put them into that situation without asking them. This conflict is nothing new, it already exists in linux distros where some software authors are mad that stable distros push older versions of their software to users (e.g. the xscreensaver message drama).
I think experimentation will show that relying on rust-version
only is not enough, and that for MSRV based version resolution it should only be used as a hint. I think fully automatted solutions will help a lot with this, so some step that determines MSRV automatically via trial compilations with different rustc versions (generally logarithmic overhead in terms of total rustc releases, and if rust-version
is present and accurate it would even be O(1)). Of course none of this will stop a motivated crate maintainer, they could detect the CI environment in the build container and just always fail for example.
Regarding flag days, from what I've obseved, every now and then there will be a big shiny new feature and large fractions of the crates.io
ecosystem will jump onto it, and this will disrupt the people who have grown accustomed to old MSRVs in the past. But appreciation for MSRVs is also growing I think, so I'm way more confident that this problem is going to be solved than I was 3 years ago.
Don't we already require cargo publish
(without --no-verify
) to be done on the claimed rust-version
toolchain? If we don't, it was discussed as a possibility on the rust-version
RFC, and I think integrally required for whichever toolchain adds rust-version
-aware version resolution.
We can't fix old packages, but in the limit, all existing packages will be upgraded, so we can have nice things (eventually), but only if we provide the prerequisites before that eventually.
to break
cargo install
.
cargo install
should default to --locked
at some point imho. Also we should be discouraging cargo install
as a way of distributing software; it's convenient but wasteful to recompile on every user machine. (Especially because you don't even get the default of compiling for this specific machine with -Ctarget-cpu=native
unless you opt in to that when you cargo install
.) Installation from source is a good option to have as a fallback, but please provide prebuilt binaries for installation through something like cargo-binstall.
experimentation
This is absolutely something that should happen, and it can happen outside of cargo! I imagine some e.g. cargo pubgrub
which offers an update
/generate-lockfile
interface which does version resolution powered by pubgrub and the sparse HTTP registry, and since all it does is set the lockfile for cargo
to use, it can do whatever version resolution options it wants.
(Even if only for a stable-available minimal version resolution I want this. The tool might want to wrap arbitrary cargo
invocations in order to do a lockfile update if necessary and then run the command with --locked
/--frozen
(I never know which is preferred).)
I do agree at this point that rust-version
-aware version resolution shouldn't be the default, though. The default should be to pull the most recent version resolution and say "hey, you don't have a recent enough toolchain version; either run rustup update
or use cargo update --downgrade-to-my-toolchain
to use potentially old crates that work on your current toolchain." (Or maybe less verbose.)
AFAIK we have been pretty consistent in saying that indeed only the latest version is ever supported, and will ever get point releases. Is that really not stated somewhere official?
We often carry patches that fix soundness bugs in LLVM, so I would consider a build of rustc with upstream LLVM as rather suspicious and prone to miscompilations. @bjorn3 mentioned an example, but this is a common pattern.
So until LLVM becomes more reliable, I am not sure if it is a good idea to tell people to use rustc with upstream LLVM. We ensure rustc works(-ish) for some range of LLVM versions, but e.g. when we get bug reports about miscompilations, we should be clear that only rustc with our LLVM is supported.
Re addressing cargo install
issues of building from source: today I came across cargo-quickinstall
by @alsuren (repo, blog post). It works by pre-building artifacts for Windows, macOS, and Linux (currently only Ubuntu 20.04 is supported) on GitHub runners and serving those artifacts on cargo quickinstall my-crate
and it falls back to cargo install
behavior if the artifact was not found.
I do understand that it creates more trust issues for security conscious people, but it is an alternative. Potentially distros could pre-build artifacts for popular crates themselves and host those artifacts on their servers to reduce the trust issues.
Edit: I see that @alsuren also mentions cargo-binstall
, which works similarly to download pre-built artifacts.
Practically speaking, with the flags for cross language LTO being stable in rustc, you're effectively close to saying it is fine to use upstream LLVM, because in many cases upstream LLVM's linker plugin is the one that will be used to LTO-compile everything. Sure, rustc might have already got past most problems effectively with its own LLVM before dumping its IR for processing by the linker, but the risk is still there.
Also note that at least Debian's rustc is compiled against the system LLVM, although that LLVM might carry some patches from the rustc fork.
If distros build the packages and serve the built packages to their users, wouldn't it make more sense to install the packages not with cargo
but with apt-get
or dnf
or nixos-rebuild
or ...? This seems like the classic distro way of installing software and the best option to me, and presumably to other people loyal to that way.
We say that you have to exactly match the LLVM version used by rustc: Linker-plugin-based LTO - The rustc book LLVM does not support loading bitcode files produced by newer versions of LLVM. Attempting to do so can lead to miscompilations and in fact will whenever a fundamental change to LLVM's ir is occuring. For example when a newer LLVM version introduces a flag to disable some UB and an optimization sets this flag, an older LLVM version will ignore this flag and optimize further with the aaaumption that this is UB when it shouldn't be. Loading bitcode files produced by older LLVM versions generally works as LLVM knows how to migrate to the newer ir, but there is no guarantee. This means that you can't use a system install of LLVM as linker plugin with a rustup distributed rustc unless your distro happens to ship the right LLVM version. Instead you should install the matching LLVM version manually or use rust-lld, which is linked against the exact same LLVM version as rustc by definition.
This says the same version, not the same forked version. It even mentions that sometimes rustc uses an unstable version, but the version reported by rustc is a good approximation. rust-lld is also not mentioned.
AFAIK, attempting to load bitcode files produced by newer versions of LLVM triggers a load error. It won't go far enough to cause miscompilations because it won't compile at all.
If rustc has strong feelings about not using upstream LLVM for cross-language LTO, it should mark its own bitcode files with a version that no version of upstream LLVM will accept to load.
I agree that using distro native package management for is the best. I am offering alternatives if for some reason distros don’t want to do that.
It doesn't seem to be the case that LLVM checks the version of the producer of the bitcode file. While attempting to load bitcode produced by the current rustc version fails to load with LLVM 11, this is not an explicit version incompatibility mismatch, but an error about an unknown attribute kind.
$ echo 'fn main() { println!("Hello World!"); }' | rustc -Clinker-plugin-lto=/usr/lib/llvm-11/lib/LLVMgold.so -
error: linking with `cc` failed: exit status: 1
|
= note: "cc" [...]
= note: /usr/bin/ld: error: LLVM gold plugin has failed to create LTO module: Unknown attribute kind (140732709536344) (Producer: 'LLVM14.0.5-rust-1.62.0-stable' Reader: 'LLVM 11.0.1')
collect2: error: ld returned 1 exit status
And for rustc 1.45.0 + LLVM 9:
$ echo 'fn main() { println!("Hello World!"); }' | rustc +1.45.0 -Clinker-plugin-lto=/usr/lib/llvm-9/lib/LLVMgold.so -
error: linking with `cc` failed: exit code: 1
|
= note: "cc" [...]
= note: /usr/bin/ld: error: Failed to link module rust_out.rust_out.7rcbfp3g-cgu.0.rcgu.o: Invalid summary version 8. Version should be in the range [1-7].
collect2: error: ld returned 1 exit status
And for rustc 1.47.0 + LLVM 10:
$ echo 'fn main() { println!("Hello World!"); }' | rustc +1.47.0 -Clinker-plugin-lto=/usr/lib/llvm-10/lib/LLVMgold.so -
error: linking with `cc` failed: exit code: 1
|
= note: "cc" [...]
= note: /usr/bin/ld: error: LLVM gold plugin has failed to create LTO module: Invalid record
collect2: error: ld returned 1 exit status
In every case it seems more by accident than calculated that it fails to load. I couldn't easily test more combinations (llvm 9 and 11 are available on my Debian installation and llvm 10 on Ubuntu 20.04), but I'm sure there is a combination of incompatible LLVM versions that works.
Except LLVM doesn't even check the version it seems, so that doesn't work.
That is because the unstable versions we use are close enough to the eventual release that incompatible changes to the bitcode are highly unlikely. I think only bugfixes happen between the unstable version we use and the release. Please correct me if I'm wrong on this.
We used to take rather arbitrary snapshots of LLVM development, but that hasn't been the case for years. Now we usually upgrade soon after the LLVM release branch is created, ~rc1, and keep updating that branch for later rcs and then through the point releases. It's possible that incompatible changes could happen during that rc phase, but not likely.
Still, LTO using distro clang
and rustc
backed by the same libLLVM
should always work.
GitHub - dtolnay/install: Fast `cargo install` action using a GitHub-based binary cache seems similar, though that one is specific to GHA.
Looks like it really isn't, but there is this saying
We will not be issuing a patch release for Rust versions prior to 1.26.0.
Most policies are means to serve specific goals, policies are not goals in themselves. For stable distro releases, the package update policies are primarily to protect against breakages; and for most non-rolling distros this means pinning down the package version, allowing no new features except for security patches.
While this may be needed to guarantee stability for perhaps most packages - like Python, GCC, Java for sure - you don't actually need to pin down stable Rust versions to prevent breakages due to the strong backwards compatibility guarantee Rust provides. Sure, stable-to-stable regressions exist, but in the grand scheme of things more bugs are fixed than introduced, and if you delay stable upgrade by a few weeks, you can be pretty confident that any regression you're likely to encounter is fixed in a point release, should one exists.
As @cuviper pointed out in https://github.com/rust-lang/libs-team/issues/72#issuecomment-1187717977 (and separately on the Fedora mailing list), Red Hat is sufficiently confident with Rust's stability that Rust toolchains are on rolling releases in all supported RHEL and Fedora versions. It's been this way for a while and there's been no horror story, so I think we have some compelling arguments to lobby for Rust to be put on rolling releases in Debian too. It should lag our stable release schedule by a few weeks, paired with a Fedora Bodhi-style pre-release testing pipeline, but not years.
As for LLVM version, since Debian is happy to ship rustc-mozilla just for Firefox, I don't see why we can't have a llvm-rust to be used exclusively by Rust toolchains.
I'm very much in favour of distros packaging rustup. @Kixunil said rustup currently doesn't verify signatures, I don't think this is right. It's been verifying signatures for a while now: Need better sig-fail diagnostics by default. "warning: Signature verification failed" is insufficient. · Issue #2462 · rust-lang/rustup · GitHub. It's currently a warning, not a hard error, but I still count it as verification. Having rustup packaged allows a chain of trust from distros down to the downloaded toolchain.
At least both Fedora and Debian are on board with packaging rustup. I had a conversation on Fedora's mailing list and @decathorpe (not sure he's on IRLO) started updating required dependencies for rustup. We're currently blocked on Please update to a newer version of the cipher dependency · Issue #41 · emgre/win-crypto-ng · GitHub but looks like there will be movement soon. @infinity0 on the Debian side has expressed interest in packaging rustup a while back, and I think it's currently blocked on debcargo not playing nicely with rustup's workspace dependency: https://salsa.debian.org/rust-team/debcargo/-/issues/34. These are technical issues, not policy ones, so it's reasonable to expect rustup hitting your distro repo soon (subject to your distro's chronoperception, of course).
Regarding "Debian + LLVM + rustc": if the Rust team needs correctness patches on top of the upstream LLVM release, one option is to ask Debian (or: any other distribution we might be talking about, but let's focus on Debian here) to apply those patches as well. In my OCaml experience, debian OCaml maintainers will routinely apply patches (on top of OCaml) to respect their own policies or fix bugs reported by their users. It's perfectly possible that they would be willing to fix LLVM as well, and I would suppose that it would become their policy to do so if rustc is packaged in Debian (which happens to already be the case today) and LLVM bugs break rustc.
Aside 1: my understanding from this discussion is that you have good, active discussion channels with Fedora people, but less surface contact with the Debian people. People describe Debian as an external entity there, to be observed by the outside. If you had/have active people in the intersection of the two communities (Debian package + Rust contributors), and you setup a good communication channel, you should be able to get a lot of progress on Rust packaging within Debian. I think it would be worth doing this for Rust, because Debian is an extremely influential Linux distribution -- I suspect it reaches more programmers than Fedora+RHEL combined.
Aside 2: in the OCaml community, we also use our own package manager (opam) for most stuff, so it may appear that packaging ocaml stuff inside Debian is low-priority. Most Rust programmers are going to neglect their distribution versions and use rustup+cargo for all their Rust hacking. In our experience, on the other hand, Distribution packaging is actually important for users of end-user programs. Rust programmers will never mind going to a non-standard packaging environment with cargo
, it's part of the learning curve. But most users of Rust programs will prefer to use their distribution, instead of having to learn a new package manager. Given the high activity around reimplementing userspace programs in Rust, I think Rust should care about this.