Beta testing rustup.rs

Oh, hey! It looks like installing source code for nightly works now!

Try:

rustup update nightly
rustup default nightly
rustup component add rust-src
ls `rustc --print sysroot`/lib/rustlib/src/rust

cc @matklad @phildawes look at this cool stuff! Very lightly tested…

3 Likes

Wonderful! And this opens the door for the https://github.com/rust-lang/rfcs/pull/1133, which would be even better :fireworks:

Is this only supported for the nightly toolchain? Because I couldn't get it to work with the stable toolchain.

Yes, only in nightly. The upstream documentation packaging landed this week.

1 Like

It looks like rustup is still segfaulting.

rustup 0.6.3 is out. Upgrade with rustup self update.

This release is just trying yet again to fix crashes in rustup-init.

0.6.3

Contributors: Brian Anderson

1 Like

@brson: some further follow-ups over at https://aur.archlinux.org/packages/rustup/.

The more I think about this, the trickier it seems. The biggest problem seems to be that, when building Rust packages from source (which Arch users do more than many other distros, but which all distros must do in order to provide binary artifacts), it is hard to have any guarantees about what version of Rust you’ll get if you simply run cargo or rustc. If you know that rustup is being used, then it’s easy to override the version, but if you don’t (and a build tool generally shouldn’t need to – it should just require “rust”), then you’re sort of lost.

It’d be great if there was a way to tell rustup “ignore defaults and use at least this version”, but without relying on actually running the rustup command. Perhaps an environment variable that rustup will use. That way, a build script could do something like:

export RUSTC_MIN_VERSION=1.10
cargo build --release
rustc foo.rs

which works regardless of whether a “regular” rust install is used, or a rustup-managed version.

This is still not ideal, because it requires the build script (or the build tool, but that’d be worse) to specify the version dependency outside of the packaging system (in Arch, you’d normally just say depends=(rust>=1.10) in the package’s metadata), but it’s a decent first step.

Thoughts?

@jonhoo I responded on the aur thread to the questions I saw there.

This can be more-or-less accomplished today with the RUSTUP_TOOLCHAIN env var, with the exception that it will not just automatically install that toolchain during the build. But to accomplish this at the project level I might rather have rustup read a config value out of the Cargo.toml metadata section or from a .rustup-version file (ala rbenv).

Is the problem you are concerned about only for the automatic building of packages, and do those packages all get built out of the same directory? If so you could have the rustup package drop the correct value into a .rustup-version file in that directory that contains whatever the correct version for the standard rustc package is; and the rustup package could install that toolchain globally (with yet-undesigned rustup global toolchain support).

Frankly, I don’t think rustup belongs in distro packaging. Don’t get me wrong, it’s a great tool that I do use, but in a distro I think all package updates need to go through the distro’s own package manager. It’s rare that the distro should want multiple versions of a system compiler at all, and when you do it’s usually something like compat-gcc-XY, at least in Fedora (like this).

Personally, I’m using “rustup toolchain link system /usr ; rustup default system”, and then I can override with upstream toolchains as needed. But I’ll be expecting any Fedora-packaged Rust programs to play nice with Fedora’s rustc and cargo.

Will rustup use a version of rustc/cargo newer than RUSTUP_TOOLCHAIN if one is installed?

No, packages in Arch are fairly often built in an ad-hoc manner as well. Packages that are released in binary form are built through a single, automated system. However, user-contributed packages are built by the users by downloading a tarball with a PKGBUILD file in it, and then running makepkg in the extracted directory. Users also have the ability to do the same for official distributed packages by downloading their PKGBUILD files. There is no good location for a .rustup-version file for all packages -- one would have to be included with every package that uses rust.

I think it should certainly be possible to install rustup using a distro packaging system. The question is then whether rustup can be then also be used to provide rust for other packages that need it, and whether rustup should conflict with the stable, stand-alone rust package provided by the distro. One could easily create a rustup package that was not labeled as providing rust, and that was still marked as conflicting with the main rust package, but this would be very inconvenient, as you wouldn't be able to install any rust-based packages (since nothing provides rust).

The Arch rustup package currently both provides and conflicts with the main rust package, and this works fine as long as the rustup default is set to something that can build any package. The discussion is basically what happens if the user has set the default toolchain to be something that is too outdated to build a particular package (e.g., they haven't run rustup update in a while), or how do you cater for packages that require rust nightly (which rustup can also provide).

A second issue is how to avoid having the rustup package be marked as conflicting with the rust package, but this seems less problematic to me.

Perhaps ccache is a useful example to follow. On Fedora, it installs all of its conflicting binaries in %{_libdir}/ccache/, then comes with /etc/profile.d/ scripts to put that ahead in the PATH. I could see a rustup package doing something similar, along with a default configuration pointing to the normal /usr toolchain.

That’s a good idea for how to avoid the conflict, but that doesn’t fix the aforementioned issues with rustc versioning.

I just realized that my personal preferences (system rust is always stable, rustup commands are explicit) would be very easy to satisfy, if there were a rustup package on crates.io. I already have ~/.cargo/bin added to my PATH to get cargo install working, so doing a nightly build could just be:

cargo install rustup    # This doesn't seem to exist.
rustup install nightly
rustup run nightly ...

Was it a deliberate decision not to put a package on crates.io? If it’s to avoid confusion with the standard install flow, maybe we could call the package something like “rustup-minimal” or “rustup-not-the-standard-install-flow-we-swear”?

1 Like

Seems a bit chicken-and-egg, no? You use rustup to install cargo, and then use cargo to install rustup?

I've been experimenting with a setup that could make distro rust and rustup peacefully coexist, but leave full configurability to the user, up to and including running their own rustup installation. It involves installing another set of proxy executables in the system path and moving rust and rustup out of the way, but so that the proxies know about them. In more detail:

  • The system rust is in /usr/lib/rust, exactly as it would've been installed by rustup (it has bin/, lib/ etc.) This could be further subdivided with separate named versions and placed under control of an alternatives scheme, I didn't try to go that far;

  • The system rustup executables are in /usr/lib/rustup or /usr/libexec/rustup, whatever the distro convention is;

  • The proxies are in /usr/bin. A Bourne shell prototype of a proxy is available. The same script can be used as rustup, rustc, cargo, and rustdoc by naming it correspondingly. (I haven't tested debugger invocation.)

The proxies would be in a separate package, and the system rust and rustup packages would depend on it. A proxy checks if there is a valid rustup toolchain directory in $HOME/.multirust or $RUSTUP_HOME. If there is, and the system rustup is installed, the appropriate rustup proxy is invoked (modulo setting USE_SYSTEM_RUST, which requests the use of distro rust.) To use rustup after installing it, the user must first run rustup init, which links the system rust as the toolchain named system and sets it as default.

Note that this a) doesn't require the modification of $PATH, b) always results in a working configuration, even between installing rustup and initializing it. The need for a separate initialization step is a bit unfortunate, but IMO tolerable.

It's true that rustup and the distro package manager are a kind of competition, but with the Rust development model in mind, I don't think it would be realistic to expect a distribution to package all artifacts available to a developer. Rustup fills that niche (and then some!), and I'm reasonably convinced that it can be made to play nicely with what the distro provides.

Unfortunately this still doesn't solve the versioning issue mentioned further above. Specifically, if the user has rustup installed, but hasn't run rustup update in a while, packages which depend on a newer version of rust will fail to build. And the version dependency cannot be expressed in any way.

The idea would be to use the system package manager to install cargo, and cargo to install rustup. This works well if the goal is “never interfere with the version of rustc/cargo that the system provides, but make nightly rust available if you want to invoke it explicitly.” That would be weirder for folks who want cargo to be rustup’s version in their PATH, though, like you pointed out.

1 Like

The scheme I've described can override rustup and require the use of a system-provided compiler (a dependency that can be expressed), by setting an environment variable, so the state of rustup's toolchain collection shouldn't be relevant. (I'm assuming that packages can only meaningfully depend on the system compiler.) Or am I missing something else?

Well, I'd like for the rustup package to "provide" rust, so that if a package requires rust, it is sufficient for rustup to be installed. that way I don't have to install both rustup and rust, keeping two copies of rust on my system

While I tend to agree that there would at least be less friction if people just installed it from us, distros are going to package it - it's what they do! So I'm inclined to do what I can to make it a decent experience.

It will not. Though I can imagine why one would want that, could you say more about why that is desirable?

Is downloading and building in an arbitrary location the common way to build both user-contributed and official packages from source? IOW there's no single command that just does both? Is there any automatic mechanism for enforcing build deps, that is, that the rustc package is installed?

Does the /etc/profile.d/ script take effect automatically? I agree that this looks promising.

Publishing the binary isn't something I've considered carefully, but it does result in no proxies, which are a core part of the design, and a scenario I haven't put any consideration into until now. One could imagine being able to bootstrap the rustup install via the system cargo, then having an option to later generate the proxies. Though off the top of my head I'd prefer in this situation for the proxies to still exist and to default to the system toolchain.