Pre-RFC: CARGO_TARGET_DIR environment variable

this is a simple extension to the cargo build script interface to allow build scripts to reliably produce things such as shell completions.

this would be simply set to the value of the variable that --target-dir controls (eg. if cargo build is run without arguments, it will be $PWD/target/debug).

however, to prevent misuse (and namespace collisions), it will only be set for the top-level binary crate. this will allow easier integration with external build systems (eg. a Makefile can install the completions generated by a build.rs using clap-complete, while letting the build script produce the completions in the same place when not being run by a makefile)

this variable will be set when compiling a crate's build script and when compiling the crate itself, but only if that crate:

  1. produces a binary --crate-type=bin
  2. is not somehow itself being built as a dependency of another crate
1 Like

Shouldn’t these kinds of artifacts go to the (unstable) --out-dir not the target dir?

well, OUT_DIR is already used, and i'm not really sure of the details of how --out-dir works, since unstable documentation is a bit hard to find.

Interesting, it actually got renamed since I last looked at it, a lot of the cargo unstable features are well documented on this page:

https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#artifact-dir

1 Like

--artifact-dir does throw a wrench in things. there's a few possibilites:

  1. just make it so files output to $CARGO_TARGET_DIR don't get put in --artifact-dir
  2. make it so these files get put directly into --artifact-dir and not target/debug (this is awkward since usually --artifact-dir results in a copy, not a move)
  3. provide an interface for build scripts to notify cargo of their outputs so they can be moved to --artifact-dir

i'm leaning towards implementing #1 and leaving #3 as a future possibility.

1 Like

Note that discussion for this is at Need a reliable way to get the target dir from the build script · Issue #9661 · rust-lang/cargo · GitHub and better support for generation and access of associated files · Issue #13663 · rust-lang/cargo · GitHub. Summary is at Need a reliable way to get the target dir from the build script · Issue #9661 · rust-lang/cargo · GitHub

It helps to limit the scope but that also difficult to define "top-level binary crate". There can be multiple binary targets per package and multiple binaries per workspace.

Overall, I would lean instead towards having a CARGO_STAGING_DIR that is given to each build script and we then have rules about how the content of the staging dir gets copied to the --artifact-dir. Or maybe there could be a build script directive to register something in OUT_DIR to do the same. There are still a lot of details to work out (like collisions).

1 Like

The easy and obvious answer to collisions in this staged approach would be to error.

I assume you mean "copied or reflinked"? Several modern file systems support COW copies these days, and that saves on disk space and SSD wear. Cargo should absolutely take advantage of that when possible.

The easy and obvious answer to collisions in this staged approach would be to error.

That is an answer but its something that would have to be carefully considered.

I assume you mean "copied or reflinked"? Several modern file systems support COW copies these days, and that saves on disk space and SSD wear. Cargo should absolutely take advantage of that when possible.

We don't make use of reflink yet in Cargo but its something that could be considered.

1 Like

whether a crate is "top-level" is not a property of the crate itself, but rather a property of a specific dependency graph being built. basically, if a crate is only being built because it is a dependency of another crate, it is not top-level.

If you run cargo build --workspace with both bins and libs (which people are likely to do out of simplicity), you made all workspace members "top-level", even if they are also dependencies. So the dependency graph doesn't say everything.

that's why the additional qualification of needing to be a binary crate is there. i'm not even sure if it's possible to depend on a binary crate (you can depend on a lib+bin package, but those are technically two different crates, and you would only depend on the lib side, i think)

What about cdylibs?

I would just put them under a "future possibilities" section for now. I can't say much about them since I've never had to use them.

I’ve long wondered if it would be possible to retrofit cdylib etc. as binary crate types, leaving [lib] to only mean rlib (and do something with proc-macros). That seems cleaner conceptually as it keeps [lib] just for what participates in Cargo’s dependency graph, and [[bin]] for final artifacts (and artifact-dependencies).

It is with artifact-dependencies. My guess would be that additional artifacts should also be put in the CARGO_<ARTIFACT-TYPE>_DIR_<DEP> for them.

I could see these being useful for non-bin crate types too (both as artifact-dependencies and top-level builds); generating a .h for a cdylib for example.

1 Like

It is possible for a crate to be compiled as rlib, staticlib and cdylib at the same time. For example because you want it to be usable both from Rust and C. Or you are making an app which has to work on iOS and Android. For iOS you would use the bin crate type, while for Android you would use cdylib. The easiest way to handle that is having a lib target produce both an rlib and cdylib and export the iOS main function to Rust and mark the Android main function as no_mangle. Then in the bin target you call the main function from the rlib for iOS and directly use the cdylib for Android.

1 Like

Yes, and I’ve run into issues from crates doing that because they have #[no_mangle] functions for the cdylib that also then get exported from the rlib and cause linker issues when using it as a rust dependency. If you really want to have such a setup you can just have path = "src/lib.rs" on a cdylib binary target. But in the majority of cases that I’ve seen it makes more sense to have a separate crate defining the public C api on top of the pure rust library.