Beta testing rustup.rs

Just a quick idea. What if you had the following, when installing both rustup and rust-X.Y (system rust) in parallel?

/usr/bin/rustup - rustup binary
/usr/bin/rustc - rustup rustc proxy
/usr/bin/rustc-X.Y - system rustc
/usr/bin/cargo - rustup cargo proxy
/usr/bin/cargo-X.Y - system cargo

In this case, you could have rustup use the system binaries as the defaults if they are installed and if they aren’t, you could default to an install with no toolchains. If you do that, you can install all the other toolchains in user home directories. I think this would only require a rustup config file in /etc/ to specify the system toolchain, when installed and it would be optional and rustup would never write one.

Edit: I’m mostly familiar with debian and it’s alternatives mechanism, which would allow something like this to work (though I’m not sure how they resolve alternative priorities):

/usr/bin/rustup - rustup binary
/usr/bin/rustc - symlink managed by distro alternatives
/usr/bin/rustc-rustup - rustup rustc proxy
/usr/bin/rustc-X.Y - system rustc
/usr/bin/cargo - symlink managed by distro alternatives
/usr/bin/cargo-rustup - rustup cargo proxy
/usr/bin/cargo-X.Y - system cargo

This would potentially allow you to get symlinks from rustc -> rustc-X.Y without rustup installed and then install rustup and have the symlinks change to rustc -> rustc-rustup, and rustc-rustup would default to calling rustc-X.Y, but would allow toolchain overriding, etc. like normal.

3 Likes

The idea to install rustup without its proxies is doable, but rustup would be quite crippled. It does break the primary functionality.

Installing rustup without its proxies is the mode that would work the best for me. Here's my thinking, partly repeating what's in that AUR package thread. Curious whether anyone else's thinking is similar:

  • Like @brson mentioned above, it substantially simplifies the story around Linux packaging, where rustc and cargo are already installed as independent packages. Some package written in Rust might want to take a build dependency on rustup, to compile with nightly for example, but that's a drag if the rustup package conflicts with the rustc package.
  • I'm not a big fan of global state in my build tools. Instead of remembering what default toolchain mode I'm in right now, I like to define separate aliases that always refer to a specific version. Something like
alias rustcn="rustup run nightly rustc"
alias cargon="rustup run nightly cargo"
  • Changing what's in the PATH can have a lot of unintended side effects, if you're on a system that builds things from source. That includes AUR packages on Arch, for example, but also Vim plugins like YouCompleteMe that have a native Rust component. Usually you're building those things to use them, not to hack on them, and it's confusing if your rustup config breaks those builds. (Running rustup default 1.0.0 would probably break everything.)

Of course all of this is totally different on Windows where Rustup is the official way to install the whole toolchain. Can anyone talk about the situation with Homebrew on OSX?

@ahmedcharles Either of those schemes is workable in rustup, as long as they are in the distros. Assuming a distro that doesn’t have Debian’s alternatives mechanism, it seems like it could be problematic for distros since they wouldn’t want all their rustc’s to be called rustc-X.Y when rustup is not installed.

@oconnor663 Thanks for the details!

RedHat family distros also have an alternatives mechanism similar to Debian’s. It’s a bit unavoidable when you want to package VIm and elVIs, or Sun Java and OpenJDK.

If you don't have alternatives, a distro can package rustc-X.Y with versioned names and have a rustc package that conflicts with the rustup package and the rustc package provides symlinks. I believe that's a fairly standard way of packaging compilers?

Though, I agree with @notriddle, in that distros probably have a way to deal with alternatives/conflicts in a reasonable way at this point. If they don't, rust (at least) won't be alone in causing them grief.

1 Like

rustup 0.6.2 is out. Update with rustup self update.

This release primarily fixes a miscompilation bug that results in segfaults in rustup on multiple architectures in a variety of scenarios. The rustup build is currently pinned to nightly-2016-08-10, it’s not clear where the bug is, and there is no fix upstream yet.

0.6.2

Contributors: Brian Anderson, Diggory Blake, Knight, Marco A L Barbosa

Oh, hey! It looks like installing source code for nightly works now!

Try:

rustup update nightly
rustup default nightly
rustup component add rust-src
ls `rustc --print sysroot`/lib/rustlib/src/rust

cc @matklad @phildawes look at this cool stuff! Very lightly tested…

3 Likes

Wonderful! And this opens the door for the https://github.com/rust-lang/rfcs/pull/1133, which would be even better :fireworks:

Is this only supported for the nightly toolchain? Because I couldn't get it to work with the stable toolchain.

Yes, only in nightly. The upstream documentation packaging landed this week.

1 Like

It looks like rustup is still segfaulting.

rustup 0.6.3 is out. Upgrade with rustup self update.

This release is just trying yet again to fix crashes in rustup-init.

0.6.3

Contributors: Brian Anderson

1 Like

@brson: some further follow-ups over at https://aur.archlinux.org/packages/rustup/.

The more I think about this, the trickier it seems. The biggest problem seems to be that, when building Rust packages from source (which Arch users do more than many other distros, but which all distros must do in order to provide binary artifacts), it is hard to have any guarantees about what version of Rust you’ll get if you simply run cargo or rustc. If you know that rustup is being used, then it’s easy to override the version, but if you don’t (and a build tool generally shouldn’t need to – it should just require “rust”), then you’re sort of lost.

It’d be great if there was a way to tell rustup “ignore defaults and use at least this version”, but without relying on actually running the rustup command. Perhaps an environment variable that rustup will use. That way, a build script could do something like:

export RUSTC_MIN_VERSION=1.10
cargo build --release
rustc foo.rs

which works regardless of whether a “regular” rust install is used, or a rustup-managed version.

This is still not ideal, because it requires the build script (or the build tool, but that’d be worse) to specify the version dependency outside of the packaging system (in Arch, you’d normally just say depends=(rust>=1.10) in the package’s metadata), but it’s a decent first step.

Thoughts?

@jonhoo I responded on the aur thread to the questions I saw there.

This can be more-or-less accomplished today with the RUSTUP_TOOLCHAIN env var, with the exception that it will not just automatically install that toolchain during the build. But to accomplish this at the project level I might rather have rustup read a config value out of the Cargo.toml metadata section or from a .rustup-version file (ala rbenv).

Is the problem you are concerned about only for the automatic building of packages, and do those packages all get built out of the same directory? If so you could have the rustup package drop the correct value into a .rustup-version file in that directory that contains whatever the correct version for the standard rustc package is; and the rustup package could install that toolchain globally (with yet-undesigned rustup global toolchain support).

Frankly, I don’t think rustup belongs in distro packaging. Don’t get me wrong, it’s a great tool that I do use, but in a distro I think all package updates need to go through the distro’s own package manager. It’s rare that the distro should want multiple versions of a system compiler at all, and when you do it’s usually something like compat-gcc-XY, at least in Fedora (like this).

Personally, I’m using “rustup toolchain link system /usr ; rustup default system”, and then I can override with upstream toolchains as needed. But I’ll be expecting any Fedora-packaged Rust programs to play nice with Fedora’s rustc and cargo.

Will rustup use a version of rustc/cargo newer than RUSTUP_TOOLCHAIN if one is installed?

No, packages in Arch are fairly often built in an ad-hoc manner as well. Packages that are released in binary form are built through a single, automated system. However, user-contributed packages are built by the users by downloading a tarball with a PKGBUILD file in it, and then running makepkg in the extracted directory. Users also have the ability to do the same for official distributed packages by downloading their PKGBUILD files. There is no good location for a .rustup-version file for all packages -- one would have to be included with every package that uses rust.

I think it should certainly be possible to install rustup using a distro packaging system. The question is then whether rustup can be then also be used to provide rust for other packages that need it, and whether rustup should conflict with the stable, stand-alone rust package provided by the distro. One could easily create a rustup package that was not labeled as providing rust, and that was still marked as conflicting with the main rust package, but this would be very inconvenient, as you wouldn't be able to install any rust-based packages (since nothing provides rust).

The Arch rustup package currently both provides and conflicts with the main rust package, and this works fine as long as the rustup default is set to something that can build any package. The discussion is basically what happens if the user has set the default toolchain to be something that is too outdated to build a particular package (e.g., they haven't run rustup update in a while), or how do you cater for packages that require rust nightly (which rustup can also provide).

A second issue is how to avoid having the rustup package be marked as conflicting with the rust package, but this seems less problematic to me.

Perhaps ccache is a useful example to follow. On Fedora, it installs all of its conflicting binaries in %{_libdir}/ccache/, then comes with /etc/profile.d/ scripts to put that ahead in the PATH. I could see a rustup package doing something similar, along with a default configuration pointing to the normal /usr toolchain.

That’s a good idea for how to avoid the conflict, but that doesn’t fix the aforementioned issues with rustc versioning.

I just realized that my personal preferences (system rust is always stable, rustup commands are explicit) would be very easy to satisfy, if there were a rustup package on crates.io. I already have ~/.cargo/bin added to my PATH to get cargo install working, so doing a nightly build could just be:

cargo install rustup    # This doesn't seem to exist.
rustup install nightly
rustup run nightly ...

Was it a deliberate decision not to put a package on crates.io? If it’s to avoid confusion with the standard install flow, maybe we could call the package something like “rustup-minimal” or “rustup-not-the-standard-install-flow-we-swear”?

1 Like

Seems a bit chicken-and-egg, no? You use rustup to install cargo, and then use cargo to install rustup?