We do have to keep direct invocations in mind. This is just exploring ways of handling rustup always setting CARGO_HOME. There are others. I was thinking this would be a good topic for our teams to talk about in the All Hands
1 Like
What's preventing this from being implemented as an opt in? Can we make it experimental and hidden behind a environment variable?
This Pre-RFC is for a first-phase where it is opt-in.
2 Likes
The second CARGO_CONFIG_HOME should probably be CARGO_CACHE_HOME
"Cargo" should probably be "Rustup" in that final bullet
Maybe this has been considered/discussed previously, but I can't find anything in this thread or any of the prior-art.
Have we considered adding CARGO_USE_XDG or RUSTUP_USE_XDG or RUST_USE_XDG/RUSTUP_CARGO_USE_XDG instead?
Pros:
- The biggest benefit for users is that we don't need to duplicate 6 env variables (in addition to the 4-6 XDG variables) & we can just rely on 1 or 2
- The environment variable will enable the behavior that we want rustup & cargo to eventually have, not just a stop-gap where XDG is ignored or some files are kept in the wrong place to keep symlinks working
- If & when we decide to change the default behavior, we can have an easier transition period where the default value of the environment variable is true instead of false. We would also not need to maintain long-term backwards compatibility for these 6 custom environment variables
- It would be easier to present the choice to the user to setup this env var during rustup installation
- There has been a lot of discussion about this on this forum & on GitHub recently about the fact that this affects people who are not rust developers too. This crowd would probably be more pleased with a single variable rather than 6 and having a simple choice presented as part of setting up rustup. The most vocal contingent who are not satisfied with
CARGO_HOME/RUSTUP_HOMEwould probably still not be satisfied - The implementation would probably be simpler & avoids the
BIN_HOMEissue (would default to$XDG_BIN_HOME/cargo)
Cons:
- I don't know if XDG is an appropriate suffix for windows, so we might want something like
_USE_V2_DIRS - The point of this RFC is to split it into phases to avoid the unnecessary bike-shedding, but this approach will probably require answering a lot of those questions now
- Some people may find use for having non-XDG paths for rustup/cargo specifically
- In this approach, it might be harder to use symlinks for backward compatibility if we go ahead & move some files around. This means that some files maybe duplicated so that both new & old cargo/rustc versions keep working
As a nix user, this won't matter to me as home-manager is pretty good about setting such env vars. But I think that even though this way requires us to decide now where we want rustup/cargo to eventually land, it would be easier to implement, maintain, explain, & maybe eventually change the default.
3 Likes
The end goal of this is to be using OS native paths by default. *_USE_XDG represents a transition which, ideally, would be shortlived. However, this adds to our compatibility story and adds more states we have to track and account for in the design, likely making the overall end goal further out.
1 Like
At least since Rust toolchain 1.21, Cargo uses home crate, which shares the same CARGO_HOME definition with rustup. I wonder if there is a way that rustup could set CARGO_HOME only for Cargo older than 1.21.
That is one of the options I've been wondering about. One area of risk is how to coordinate on changing bin-dir in the future.
Another angle we need to deal with is how will a new rustup with new cargo with new paths deal with the user installing an old cargo.
Do we really need to support all of these possibilities:
- old rustup, old cargo
- old rustup, new cargo
- new rustup, old cargo
- new rustup, new cargo
#2 in particular seems very rare. Could we handle that some other way?
Nothing to do
This is Cargo respecting CARGO_HOME which we will already have to do
Dropping this would negate the benefits of rustup
Easy 
1 Like
I think the problem that could occur is if someone sets CARGO_*_HOME - old rustup will not respect it, but new cargo will, leading to a bin dir disagreement.
Unless we make CARGO_HOME higher priority in the fallback.
On the contrary, that seems quite likely: consider the case where you have an old rustup installed via your Linux distribution, and you use that rustup to install a brand-new toolchain including cargo.
5 Likes
Maybe making CARGO_HOME a higher priority in the fallback is the easiest solution:
- old rustup, old cargo: no changes, already works
- old rustup, new cargo: old rustup always sets
CARGO_HOMEwhich overrides the new vars - new rustup, old cargo: new rustup detects versions of cargo that don't support the new vars and does some special handling
- create a dir
CARGO_CACHE_HOME/old_cargo - copy/symlink cargo config in there
- invoke old cargo with
CARGO_HOME=CARGO_CACHE_HOME/old_cargo - error when running
cargo installon old toolchain through proxy
- create a dir
- new rustup, new cargo: just works
it's worth noting that CACHEDIR.TAG is intended to be cross-platfrom, indeed, the filename was chosen so that it works on any filesystem, including ones with strict limits on filenames.
The name may be cross-platform-compatible, but the behavior it expects to trigger is not automatically cross-platform. It depends on the platform recognizing this file name as special, and acting accordingly.
macOS does not recognize this file as special. To macOS it's not any different than some MAKE.FART.NOISES file.
Apple tends to arrogantly support only Apple's ways of doing things. In this case Apple supports only a custom extended filesystem attribute or a list of paths in settings for excluding directories from backups. Apple only supports an unofficial .noindex directory name suffix or a list of paths in settings for excluding directories from realtime search indexing (and these lists automatically include paths to specific cache directories designated by Apple). I'm not aware of any configurabilty or extension point in macOS that could make CACHEDIR.TAG do anything. macOS doesn't have a way for applications to say where they've chosen to put their caches, it's the opposite, the applications are supposed to ask macOS where they're allowed to put their cache.
1 Like
And this is is why you should follow the platform approach on each platform. XDG for *nix (except Mac OS X), Mac specific conventions on Mac, Windows specific conventions on Windows.
This should be the (long term) default, and if users want to override that there will still be the enviornment variables. I expect it to be pretty rare, but there are use cases for overriding (especially on *nix, which has higher variability in general): I might for example want to put the cache dir on a local disk if my home directory is mounted over NFS. Rare, but not entirely unheard of.
That doesn't stop third party apps from respecting CACHEDIR.TAG. In fact if they are cross-platform and support it on Linux I did expect them to support it on every other platform too.
The two main pain points — Time Machine and Spotlight are made by Apple, and are built into macOS. Most users don't use any 3rd party replacements for these, since these features are good enough in general, and are more integrated into macOS than 3rd party apps can. 3rd party Spotlight replacements tend to reuse Spotlight's index anyway, so they can't add support for the CACHEDIR.TAG at the time of the indexing.
XDG vs Library paths has been re-litigated multiple times for multiple tools. As a macOS user, I can say that most CLI tools follow XDG spec & most GUI tools follow Library/Apple spec. If you want, I can compile a list of major tools which follow these guidelines. There is a case to be made that Rust/Cargo should follow what other CLI tools do as that would probably be what most people expect. I would say advising the user to ignore ~/.cache in Time Machine settings while installing rustup on macOS should be enough. Any person using rust a lot already has to add exceptions for the folders where they store their rust projects because of large & much more rapidly changing target dirs.
1 Like