Pre-RFC: Improve access to Rustup components

tools and infrastructure
1

I propose improving the discoverability of, and the update process for, tools using Rustup. The proposal is a series of fairly minor changes, the most significant of which adds an extra stage (and some more options) to the first-run installation of Rust.

Motivation

Rustup is a powerful tool for installing and updating Rust and its tools, and managing toolchains. It supports optional components including libraries, tools, source code, documentation, and data. However, Rustup was mostly designed and implemented before the tools ecosystem (other than Cargo and Rustdoc) was an important part of the Rust experience. Therefore, the experience with tools and Rustup is sub-optimal, especially for new users.

Primarily, we need better discoverability of tools; we believe that having the right tools will give new users a better experience of Rust. Since some tools can be missing from a nightly distribution, we should handle that case in a more user-friendly manner.

Description

This RFC proposes several new Rustup features with different levels of impact and complexity. They are presented here in rough priority order of importance. None of these are breaking changes in most senses, though ā€˜tool stubsā€™ could have some problematic interactions with other software, described inline. I would be very glad to hear of other suggestions.

Installation profiles

When first running Rustup (rustup-init, usually via the rustup script of batch file) the user is presented with some options (for example, which toolchain should be the default). I propose adding another option, which ā€˜profileā€™ to use (I do not expect the word ā€˜profileā€™ to be user facing, since it is already somewhat overloaded by Cargo). The possible profiles would be:

  • ā€˜minimalā€™ - just rustc, std, and Cargo (we might possibly want a ā€˜very minimalā€™ profile which is only rustc). Note that this would not include Rustdoc or the standard library docs which are included by default today.
  • ā€˜defaultā€™ - ā€˜minimalā€™ plus Rustfmt, Clippy, Rustdoc, and the standard library docs. Note that this does not include the RLS and its supporting data, since that is usually installed by an editor plugin and is usually a good user experience. Using the RLS without an editor plugin is very rare.
  • ā€˜completeā€™ - ā€˜defaultā€™ plus the RLS and itā€™s data (docs and save-analysis for the standard library), lldb on Linux, llvm-tools, and any other tools or data added as Rustup components in the future.
  • ā€˜customā€™ - ā€˜minimalā€™ plus interactively opt-in to each component

The profile is only used for first installation. If a component is later added to a profile, it will not be automatically installed for users who selected that profile.

Tool stubs

If a user does not have a tool installed and tries to run it, then Rustup should suggest installing it. E.g., if the user runs rustfmt but has not installed the Rustfmt component, then Rustup should inform the user that Rustfmt is not installed and offer to install it.

Implementing this should not be too difficult because Rustfmt creates symlinks to Rustup for all installed executables, it would just have to do this for executables which are not installed.

There is some backward compatability concern in the case that the user has installed a different executable with the same name already. However, Rustup is able to cope with this when a component is installed (by reporting an error to the user), and I think the same tactic could be used for components which have not been installed.

Better handling of missing tools

The current policy for tools other than Cargo and Rustdoc (currently Rustfmt, the RLS, and Clippy) is that PRs to the Rust master branch may land if they break a tool, except in the last week of a cycle (to ensure all tools are present for the beta fork). If a nightly should be released when a tool is failing to build or its test, it is not included in that nightly release (it is marked as unavailable in the Rustup manifest). If a user has a missing tool installed, then an update will fail with an error, unless the user uses --force. Similarly if a user tries to add a missing tool, then installation will fail with an error.

Some possible improvements:

  • better error messages explaining what is going on (this is low-hanging fruit and doesnā€™t need an RFC, itā€™s included for completeness)
  • display the current installation to make it easier to ā€˜unwindā€™ an update (potentially also offer a rollback command, though that is somewhat out of scope for this RFC)
  • when a tool is missing, offer to update to the most recent release which has the tool
  • an interactive mode so that users do not have to rerun with --force. Iā€™m not sure how to implement this: Rustup probably needs to remain non-interactive for default so that it can be used by tools and in CI.
  • if we implement tool stubs as described above, then the stub could give a specific error message in the case that a component is missing.

Questions

  • Any other ideas?
  • Which of these need a formal RFC, vs just implementation?
  • Do we need a formal policy or process for deciding which components to add to which profiles?
16 Likes
2

Timing-wise, I would like this to be ready for the 2018 edition announcement, but I donā€™t feel like it is an integral part of the 2018 edition.

I should also note that this is an a la carte RFC, rather than a prix fixe - we will probably want to implement a selection of the suggestions, perhaps not all at the same time.

3

Donning my Gentoo hat, we had quite heated discussion regarding how to deal with the components. What you are proposing seems to tie the components even more to rustup, would you consider providing hooks/templates distributions can use so the missing-component message would fit the distribution (e.g. cargo fmt telling to emerge rustfmt if cargo had been provided by portage.

2 Likes
4

Sounds good overall! I like the more fine-grained control, such as making docs non-mandatory.

The code doing this should try and detect whether it is connected to an interactive terminal, and only prompt the user in that case. This is to avoid waiting indefinitely for input in CI for example.

1 Like
5

@nrc When running the complete build I think I would want to get new tools as they come out, but perhaps it doesn't make quite as much sense for the other installations? Could you perhaps expand a bit on your reasoning here?

6

I like these ideas, particularly the installation of rustfmt etc by default. I suspect components are just not that configurable.

Iā€™ve also had the problem of ā€œadd a component that happens to not exist on this nightly, get confused why Iā€™m getting an errorā€, so I think we can do better there.

7

Iā€™m not sure if this is the right place, but one thing that greatly confused me when starting with rustup was the notion of toolchains. For example, I would install rustfmt-preview on nightly and then try to use it from a stable project. It took me a while to realize that installing a tool/component on a toolchain only installs it for that toolchain.

8

The other thing that I think users would appreciate more visibility into is location of installation. Currently, rustup/cargo default to $HOME/.rustup and $HOME/.cargo. A user can change this by setting RUSTUP_HOME and CARGO_HOME env variables, but it tooks some digging to find this out.

These settings are particularly important for users inside of some corporate or university network. For example, for me $HOME is on a network file system, which means that doing anything with rustup or cargo becomes intolerably slow (and symlinks are not well supported, which messes with incr. compilation) and cargo/rustup just eat up my quota. Instead, I use a scratch partition on my local disk, which is a fast SSD.

9

Hmm... I think for most users, RLS will be wanted. Thus; I'd recommend making "complete" the default and renaming "default" to something else.

10

To follow up on this thought, @nrc you mentioned that the VSCode plugin (etc) can auto-install the RLS, which is true ā€” but right now it does not handle the scenario where the current nightly doesnā€™t have the RLS installed very gracefully (i.e., the ā€œmust rollback to earlier nightlyā€ case). We can of course presumably make it do so, but if we defaulted to including the RLS, this would be a more unusual scenario.

We might have something like:

  • minimal
  • cli-only
  • default
  • custom

or whatever.

1 Like
11

(On the other hand, I guess that we expect most users to be on stableā€¦)

12

This list seems excellent and aptly named.

While VSCode does install RLS; the experience has not always been great I agree. In any case, I think the installation process would be smoother if rustup just did this and users would not have to go through an extra step. This would also be good for other editors as we can't assume that everyone will be using VSCode.

2 Likes
13

As an extension of these ideas, it would be nice if the initial Rustup install offered options to install additional support for any particular editors:

  • vi/vim
  • emacs
  • ā€¦

If that would make any sense that is?

14

I too have had some ā€œproblemsā€ with rustup in my first couple of days working with rust. Some things Iā€™d like to see related to this RFC:

  • intuitive toolchains ā€“ The concept of different toolchains didnā€™t really click with me at first, and I had some troubles understanding what was installed globally, and what wasnā€™t, especially when it comes to some of the extra tooling you mentioned (I believe cargo and rustup itself arenā€™t tied to a toolchain version, but everything else is, right?). Iā€™m also not even sure if I can actually have multiple stable versions of Rust installed using the toolchain, I donā€™t believe so (I can only switch between stable/beta/nightly right?) just checked this, you can!

  • auto update ā€“ I love, love some of the tools that tell me (but donā€™t yell at me) when I use their tool and thereā€™s actually a new version available. The one that springs to mind is the gcloud utility to communicate with Google Cloud. In fact, that tool also has the ā€œyou tried to use this subcommand, but it is not installed, shall I install it for you?ā€ functionality, which is great for discoverability.

  • release notes ā€“ Tied to auto updates, I like to be able to instantly see what has changed in the tools Iā€™m using. I like the change log windows that VS Code shows me when itā€™s updated. I donā€™t like that I have RLS auto-updating on in VS Code and whenever thereā€™s new changes, I can see it updating in the status bar, but I have no idea whatā€™s changed after itā€™s done updating. Maybe this is just the toolchain-nerd in me, but I like to stay up-to-date with all the latest improvements that can help me become more productive.

  • better documentation support ā€“ Couple of things here; thereā€™s rustup doc which opens up the ā€œglobalā€ documentation, but the first page isnā€™t that great, it has big/huge headers with one or two sentences after each header, and a link to a deeper page. I feel like this can become much more inviting, interactive and useful with some effort. Second, thereā€™s cargo docs which could use some introduction for new people, itā€™s not that explorable. It would also be nice if it supported live-reloading of your crate docs while you work on them, but thatā€™s not really relevant to this proposal.

  • REPL ā€“ this might be out of scope, but some simple REPL (as proposed in this RFC) that comes with some/most of your proposed ā€œprofilesā€, to quickly fire off some coding experiment is something I would greatly enjoy.

Some (most?) of these are probably out of scope for this RFC, but I wanted to let you know, in case anything pops up that you think is interesting.

All in all, improving the rustup story is definitely something I believe would help newcomers, as it has been something that has caused me to stop a couple of times to figure out how it works and what to do next.

3 Likes
15

FWIW, IntelliJ users won't need the RLS (at least for now).

16

This sounds like an excellent idea!

When running the complete build I think I would want to get new tools as they come out, but perhaps it doesnā€™t make quite as much sense for the other installations? Could you perhaps expand a bit on your reasoning here?

My basic reasoning is that people don't seem to like having things installed which they haven't opted in to. It also complicates implementation since the profile used needs to be preserved in Rustup's state. I also wanted to avoid thinking about users who already use Rustup without having chosen a profile (although only doing something for 'complete' side-steps that.

one thing that greatly confused me when starting with rustup was the notion of toolchains

I think we could definitely do better about making the install instructions self-documenting.

The other thing that I think users would appreciate more visibility into is location of installation

I agree, but I think it is pretty orthogonal to the ideas here.

but right now it does not handle the scenario where the current nightly doesnā€™t have the RLS installed very gracefully

this is more of an implementation issue, in fact I think it should be somewhat improved already. The extensions use Rustup to manage the RLS, so installing by default won't make any difference to their handling of it.

While VSCode does install RLS; the experience has not always been great I agree. In any case, I think the installation process would be smoother if rustup just did this and users would not have to go through an extra step. This would also be good for other editors as we canā€™t assume that everyone will be using VSCode.

Making installation of extensions slicker is a nice motivation. All editors that use the RLS should be able to install it too, not just VSCode.

As an extension of these ideas, it would be nice if the initial Rustup install offered options to install additional support for any particular editors:

This is definitely worth thinking about. Most editors allow installation of plugins from the command line, but I fear this is something that will easily go wrong for users with even slightly customised systems.

Some (most?) of these are probably out of scope for this RFC, but I wanted to let you know, in case anything pops up that you think is interesting.

These are good ideas (especially informing the user when a new version is available), but I think they are out of scope for this RFC.

FWIW, IntelliJ users wonā€™t need the RLS (at least for now).

This is true, and will be for the foreseeable future and is one reason not to install the RLS by default. OTOH, it's not a huge amount of disk space/bandwidth and is easy to remove.

17

Another good hint would be to automatically suggest cargo plugins. It could work like this:

$ cargo tree
`cargo tree` is not a built-in `cargo` command but it's provided by the following plugin:

cargo tree: a Cargo subcommand that visualizes a crate's dependency graph in a tree-like format
run `cargo install cargo-tree` to install this plugin

$ cargo add
`cargo add` is not a built-in `cargo` command but it's provided by the following plugins:

cargo-edit: a utility for managing cargo dependencies from the command line
run `cargo install cargo-edit` to install this plugin

cargo-add: this crate is deprecated in favor of cargo-edit
run `cargo install cargo-add` to install this plugin

install one of these plugins to make `cargo add` command available

This, however, requires a proper registry of cargo plugins, and I donā€™t know if it exists now.

Also maybe cargo plugins should also be managed by rustup and automatically updated with rustup update.

3 Likes
18

I think here you meant lldb on Darwin. We're not building lldb for Linux. We could... there's discussion over in https://github.com/rust-lang/rust/issues/48168. Though I tend to think if we're going to do this we should also ship a gdb for Linux.

19

Whoops, yeah, thatā€™s a typo

20

I talked this over with the dev-tools and Cargo teams. There was broad support. There was also a feeling that profiles should be an ongoing thing (and even to make everyone use the default profile if they havenā€™t opted in to one), and when components are added to a profile, then the component is automatically added for the user.

2 Likes