[Pre-RFC] Put CACHEDIR.TAG into target/

tools and infrastructure cargo
#1

As far as I can tell, target directory only contains things that can be easily re-generated using cargo build or cargo doc. Also, target far outweigs the source code even in an almost-empty crate. Thus it's a good candidate for exclusion from backups.

I currently achieve this by passing an exclusion regex to my backup program, but that's suboptimal: someday I'll create a directory called "target" and won't notice that it's not being backed up. In my opinion, a much better solution would be for Cargo to automatically mark its target directories with a CACHEDIR.TAG. I can then do borg create --exclude-caches …, which will automatically exclude these directories from the backup. (GNU Tar and Restic support that CLI option as well, and I bet your backup solution does, too).

Contribution guidelines on github.com/rust-lang/cargo told me to post here before opening an issue there, so: what do you folk think about this idea? Is this incompatible with something? Would this break anything in your personal workflow? I never worked on Cargo, but I'm pretty confident I can implement this if I have a green light.

4 Likes
#2

One caution is that I think this should only be added when cargo first creates the target directory. If it already exists, let it be. Also consider CARGO_TARGET_DIR -- I guess it can be treated the same way.

2 Likes
#3

I don't think we should add this to CARGO_TARGET_DIR, because in that case, you might have a shared directory with other things.

But we should absolutely create it when we're creating the default target directory under the source directory.

#4

Be careful with that, cargo clean will still wipe the whole thing out!

But yeah, it seems fair that if you use CARGO_TARGET_DIR that you should handle adding that to your configs yourself, since it's now a known location.

1 Like
#5

With my Cargo team hat on, I'm in favor of this. If someone posted a PR implementing this (including tests and documentation, and only applying when cargo initially creates the directory and it isn't from CARGO_TARGET_DIR), I'd happily support merging it, assuming others on the Cargo team don't object.

2 Likes
#6

Great! I'll have a bash at implementing it, then.

#7

I like it but as @CAD97 noted, cargo clean will remove it. Is there a way of protecting it from cargo clean?

#8

You shouldn't be setting CARGO_TARGET_DIR to a shared directory in any case; it could easily trample other files in the directory even without cargo clean. Instead, you should set CARGO_TARGET_DIR to ${MY_CACHE_DIR}/cargo or similar.

#9

cargo clean will also remove the entire target directory, so it should remove the CACHEDIR.TAG file as well.

#10

From the specification:

User-friendly file managers and other directory hierarchy browsing software may also want to notice the presence of a cache directory tag in a directory, and inform the user that the contents of the directory is automatically managed and is not likely to make much sense if "browsed" directly.

Does anyone know if some software does this? It could be annoying for target, but I doubt any file manager actually implements this suggestion.

#11

IMHO Cargo should distinguish between build products and build temporaries (e.g. Xcode has such distinction, and I suspect most other build systems). Currently both are dumped in the same place, making it impractical to point caches to a dedicated temp directory, without losing quick access to build products (i.e. crate's own executables and own top-level libraries).

There's an issue for this, although it suffered from a bikeshed about Linux standards:

Once Cargo is able to put temporary files in a proper dedicated system-level temporary directory, these files won't be causing headaches for backups any more.

3 Likes
#12

I'd love for someone to pick up and run with that. But in the meantime, I think adding CACHEDIR.TAG is a useful incremental improvement that doesn't strictly require such separation.