Pre-RFC: Split `CARGO_HOME`

That file has no meaning on macOS. macOS does not follow Linux conventions (to the point of seeming spiteful about not following them on purpose).

AFAIK officially you're just not supposed to put cache directories anywhere except path given by foundation framework when queried for an NSCachesDirectory. The undocumented way is to rename the directory to have a .noindex suffix, e.g. ~/.cargo.noindex.

BTW: macOS also doesn't follow .nobackup convention. I had to add Apple-specific library call to Cargo to exclude the target/ directories from backups (sadly there's no similar API for disk search). Apple strongly prefers exposing high-level library APIs over using any conventions or anything in the file system directly.

3 Likes

There is whatever these translate to as well (I assume xattrs but I haven’t checked):

(that second one doesn’t say if it works on whole directories though)

Neither of these directly say they turn off indexing though.

1 Like

FYI I've updated the original post based on feedback. For more fine-grained updates, see my branch's history.

1 Like

Can just Cargo.toml files be expanded instead of the whole tree? Or it could be some trait over the storage and turn into git cat-file or a traversal of the .crate file to get the relevant information.

Sounds like a use case mirroring my own usage of target as symlinks off into a uniform location for such things (including build trees of non-Rust software).

For clarity, I didn't say not to use an app data folder, just that there's no specific reason to use Roaming instead of Local.

And actually the most proper way to retrieve the app data folder isn't the environment variable, it's the Known Folders API with FOLDERID_LocalAppData.

I thought I'd note that cargo-audit uses the home crate to obtain cargo_home() from which it constructs e.g. ~/.cargo/advisory-db as a cache of the advisory DB to scan against. However, it seems like we should have no trouble migrating to e.g. cargo_data_home() in future releases.

The reason to use Roaming rather than Local is because of convention, and because Microsoft recommends it. Local is for less important data. People may, for instance, back up Roaming but not Local.

1 Like

The advisory database seems like it belongs in the cache dir more than it belongs in the data dir.

Agreed, given that it's caching a network resource that can be re-downloaded.

There already is a directories crate that understands the appropriate layout for each platform. I suggest just using it. Some thought was already spent mapping the respective recommendations to each other, no need to do it again.

I don't know which “other CLI developer tooling”, but it would be best if all rust-related tools converged on using directories and then they'd behave consistently. And if tools for other languages differ, that's not a good reason to repeat their mistakes.

2 Likes

Just because a package in the ecosystem did something is not justification for us to blindly follow it. Having a decision document from them would be the most relevant from any existing packages.

I've done another update of the Pre-RFC.

This went all in on the fallback scheme approach though I do wonder if there is a way to make a hard-split work as it'd make old + new rust Just Work. The details to work out are around

  • rustup setting CARGO_HOME
  • rustup not supporting layered config except there wouldn't be concurrent rustup versions, only cargo
  • if rustup / cargo can handle multiple bin dir

Thanks for all of your work on this, and for keeping the broader, cross-platform issues in mind. It eases the mind and is good to see progress in this area.

With rustup assuming complete control of its bin dir, what effects could that have on ~/.local/bin (the XDG spec location on Linux for installing binaries)? Does rustup, for example, delete/move binaries other than those it created, or do blanket deletions?

When there's a shared ~/.local/bin folder, with programs dropping binaries into it, there's always a potential for conflict, though conflicts are fairly rare. Partly, the rarity is because developers know that if they're making a program for general use, they must find an executable name that's unique in the /bin, /usr/bin, /usr/local/bin namespace -- which also typically means they'll be unique when the ~/.local/bin namespace is added in as well.

Is it possible, though, that arustup could be (reasonably) modified to only assume ownership of its own files placed in the folder, rather than the entire bin folder?

Its been a while since I've looked into the rustup side of things. However, solving that is likely not a blocker for this milestone of the work so long as we decide to have a CARGO_BIN_HOME because this provides an opt-in mechanism for people to put the folders wherever they want and that can give us an opportunity to try things out and explore improvements throughout the toolchain, including rustup. Hopefully we can iterate and improve rustup to resolve any blockers to allow XDG_BIN_HOME but that can all come later when its easier to directly vet the work.

2 Likes

100%.

Could the legacy location be set to be the first matching target? As in:

  • cargo_data_home: Returns the first match
  1. ~/.cargo, if exists
  2. CARGO_DATA_HOME, if set
  3. CARGO_HOME, if set
  4. cargo_home(), if present
  5. linux or macOS:
  6. $XDG_DATA_HOME/cargo, if set
  7. ~/.local/share/cargo

This might remove any need to migrate old users

1 Like

If I understand correctly, if the CARGO_DATA_HOME varible is unset, there is no change in behaviour nor any need for any migration, either. This being because step 3, "cargo_home(), if it exists", acts as fallback to the default legacy value of ~/.cargo. (And if it does not exist then it does go through to the XDG directories)

As implementation note, cargo_home() already checks CARGO_HOME before ~/.cargo, so no need to check it manually. This does not affect the RFC though.

All in all, seems well though out. @epage, typo warning: you have repeated CARGO_CONFIG_HOME in the guide-level explanation. I suppose you meant CARGO_CACHE_HOME in one of them.