More granual `build.rs` companion: `links.rs`

[dhm]

The problem

Click to fold

I'd like to kickstart the discussion around one aspect which makes "cargo check does not compile/codegen-machine-code nor involve linkage" a blatant lie, one which makes it so cargo check/clippy, when invoked by the user, or when invoked by rust-analyzer, is not as fast as it could be:

1. Every build.rs in the dependency tree MAY generate Rust code which may affect cargo check-ing;

2. Therefore, every build.rs "build crate" in the dependency tree needs to be:

   • checked (which is sensible),
   • compiled (and linked as a binary),
   • and executed,

   before the main crate of its encompassing package (usually, a [lib] crate) gets to be itself checked and have its "Rust metadata" generated before every downstream dependent thereof gets to be itself, in turn, checked and so on.

3. Notably, many build.rs scripts out there stem from the "conventional …-sys packages", i.e., from Rust packages wrapping some C library (or some otherwise FFI-bridgeable library) through:

   • a build.rs crate which often, for convenience, compiles from scratch a bundled version of this library, often linking it into a C library, and then emitting linkage directives to Cargo for the final binary artifact to link against such library,
   • (and a [lib] crate shim which declares, in Rust, the C-y signatures embedded within it, so dependent Rust code know how to call into it).

Now, taking a step back and looking at all of this, we can see the inefficiency: we're compiling and executing build.rs scripts so many of them, in turn, compile C libraries, for them to be available to Cargo should some cargo build/run/test/bench occur.

But in the context of a cargo check or cargo clippy or cargo doc, there is no such build/run/test/bench occurring, resulting in:

• at best, having unnecessarily frontloaded the time and resources involved into compiling part of the final artifacts (e.g., in the case of some cargo check/clippy prior to running some cargo build/run/test/bench);
• at worst, having done it fully unnecessarily. For instance, rust-analyzer will never run cargo build for its flycheck diagnostics or its code navigation utilities. When using a separate target/ dir for it (as advisable for users wishing to concurrently use cargo … on their own), it means that any such C compilation artifacts have been produced in vain.
  • even worse, sometimes the rust-analyzer environment is somehow not configured well enough for the C compilation to even be able to succeed (e.g., some improper CC or CMAKE env var or whatnot setup). In such a case, rust-analyzer diagnostics will end up tainted with such a failure.

For the remainder of the post, I'll refer to such build.rs scripts as "conceptual links.rs build scripts"

• for the build.rs doing both Rust code generation and C compilation-and/or-linkage, these could be split, at least conceptually, into its pure code-generating part ("conceptual build.rs"), and its merely compile-and/or-linking part ("conceptual links.rs").
  • Note that this happens more often than may initially seem, since it's one of the "official" ways to use rust-bindgen.

Palliatives / working around it with the current tools

Click to see

• Magic RUSTC_WRAPPER strings

  For instance, rust-analyzer sets, by default, within its configuration, RUSTC_WRAPPER=rust-analyzer:

  @https://rust-analyzer.github.io/book/configuration.html#cargo.buildScripts.useRustcWrapper

  rust-analyzer.cargo.buildScripts.useRustcWrapper

  Default: true

  Use RUSTC_WRAPPER=rust-analyzer when running build scripts to avoid checking unnecessary things.

  • (Emphasis mine.)

  I imagine this means that conceptual links.rs build scripts willing to be friendly to such a situation would then be expected to have the following bail-out:

  //! "Conceptual links.rs" logic within a build.rs
  
  fn main() {
      if ::std::env::var("RUSTC_WRAPPER").as_deref().is_ok_and(|s| s == "rust-analyzer") {
          eprintln!("\
              `rust-analyzer` environment detected: \
               skipping compilation/linkage of C dep `libfoo`\
          ");
          return;
      }
      // actual logic here
  }
  

  Whilst it is nice for this to exist, it is a pity for the package author to need to think of doing this. And from this search, it doesn't seem to be done much in the wild.

• links = "libname" "abuse"

  It turns out that there is one area of Cargo which is links-aware: the package.links Cargo.toml field

  "Conventionally …-sys" crates are expected to declare themselves as such by setting this entry to the name of the C/FFI library they wrap.

  From there, an end workspace (.cargo/config.toml) depending on such a package can opt into "hijacking"/bypassing/circumventing the build.rs of such a package:

  @https://doc.rust-lang.org/1.92.0/cargo/reference/build-scripts.html#overriding-build-scripts

  Overriding Build Scripts

  If a manifest contains a links key, then Cargo supports overriding the build script specified with a custom library. The purpose of this functionality is to prevent running the build script in question altogether and instead supply the metadata ahead of time.

  To override a build script, place the following configuration in any acceptable config.toml file.

  [target.x86_64-unknown-linux-gnu.foo]
  rustc-link-lib = ["foo"]
  rustc-link-search = ["/path/to/foo"]
  rustc-flags = "-L /some/path"
  rustc-cfg = ['key="value"']
  rustc-env = {key = "value"}
  rustc-cdylib-link-arg = ["…"]
  metadata_key1 = "value"
  metadata_key2 = "value"
  

  With this configuration, if a package declares that it links to foo then the build script will not be compiled or run, and the metadata specified will be used instead.

  • Source

  To clarify (because, imho, these two sections are not very well explained by the reference):

  1. Say there is some C library libfoo;

  2. There will probably be some ::foo-sys Rust package wrapping it;

  3. With a build.rs potentially compiling and definitely emitting linkage directives against such a library;

  4. ::foo-sys would thus be expected to declare some package.links = "libfoo" directive/entry in its Cargo.toml package manifest

  5. Any user of this package / of its [lib] crate, either a direct user or some transitive downstream dependent, within their own workspace, can then declare to be overriding its (compilation, if any, and) linkage setup for the libfoo C library, by using the following:

     [target.<TARGET_TUPLE>.libfoo]
     rustc-link-lib = ["foo"]  # This is `-lfoo`, to find `libfoo.{so,dylib,a}` on Unix, `foo.{dll,lib}` on Windows.
     rustc-link-search = ["/path/to/foo"]  # This is `-L/path/to/foo`.
     

     i.e.,

     [target.<TARGET_TUPLE>]
     libfoo.'rustc-link-lib' = ["foo"],
     libfoo.'rustc-link-search' = ["/some/dir"]
     

     i.e.,

     [target.<TARGET_TUPLE>]
     libfoo = { 'rustc-link-lib' = ["foo"], 'rustc-link-search' = ["/some/dir"] }
     

     which boils down to:

     [target.<TARGET_TUPLE>]
     libfoo = { … }
     

  Setting this effectively disables all of ::foo-sys' build.rs script execution

  Granted, this can be a handy trick for actual cargo build/run/test/bench compilations that manage to set up stuff in a way where the desired C library can be found, compiled, at some predictable path (this is the very point of the feature!).

  But it turns out that this can be (ab)used to make certain, specially-crafted, cargo check/clippy commands skip these build.rs scripts:

  1. It turns out that setting an empty libfoo = {} "object" suffices to trigger this; or, oddly enough, embedding a dummy key-pair within it works too:

     libfoo = { 'build.rs' = 'skip' }
     # or
     libfoo.'build.rs' = 'skip'
     

  2. To add such a setting on-the-fly / only for specific commands, cargo supports single-line .cargo/config.toml additions, like this:

     RUST_HOST_TUPLE="$(rustc --print host-tuple)"
     cargo <check/clippy> … \
         --config "target.'${RUST_HOST_TUPLE}'.libfoo.'build.rs'='skip'"
     

     • (An empty object cannot be set this way; I imagine so that the approach be additive. Hence the dummy key-pair approach.)

  Limitations of .cargo/config.toml links overrides

  • when used, on-the-fly, for cargo check/clippy, it's hacky, cumbersome (needs knowing of every potential …-sys crate in the dep tree, and the links name each uses, multiplied by the "matrix" of every potential target tuple which may be involved), and can ironically back-fire if the cargo command ends up followed by an actual cargo build, as doing so might require recompiling everything given the adjustment of the .cargo/config.toml. It might be advisable to only use this in conjuction with custom --profiles (or otherwise separate target/ dirs) so these cached artifacts be properly insulated from the ones involved in actual builds; e.g., within rust-analyzer config:

    • Relevant snippet from my rust-analyzer config

      "rust-analyzer.cargo.extraArgs": [
          "--config", "target.'aarch64-apple-darwin'.rocksdb.'build.rs'='skip'",
          "--config", "target.'aarch64-apple-darwin'.snappy.'build.rs'='skip'",
          "--config", "target.'aarch64-apple-darwin'.titan.'build.rs'='skip'",
      ],
      "rust-analyzer.cargo.targetDir": true,
      

  • Even when used legitimately, as per the rust-bindgen example above, bypassing the build.rs currently also means that its code generation part, if any (such as that of generating "FFI headers" in Rust) will also be skipped (unless the package were to split itself into two, one with a package.links + build.rs, and the other, with just the build.rs…).

---

The suggested solution

Would be for Cargo to fully acknowledge and embrace this package.links with a companion build script (the "conceptual links.rs build script"), and reïfy this concept:

1. Allow for package.links to be set to an object rather than just a string, with, at least:

   • name entry, to be set to its identifying name,
   • script entry: to be set to point to some .rs file.
     • Optionally, it could be set to true to default to "links.rs"; and/or having such a file alongside a Cargo.toml and a links = "libfoo" kind of entry could be tantamount to all this.

   [package]
   links = { name = "libfoo", script = "links.rs" }
   

2. Such a links.rs-specified Rust file shall:

   • be interpreted like a build.rs does currently in Rust:

     • involving build-dependencies,
     • living in the build/host universe (e.g., w.r.t. Cargo resolver or whatnot);
     • being bypassable through links overrides (the build.rs being the one bypassed as a fallback, for retro-compat).

   • but only involved:

     • for actual cargo-compiling commands, such as build/run/test/bench, i.e., it would be skipped when running cargo check/clippy/doc.
     • invoked in the tail-end of the compilation pipeline.

     Or, more precisely, deemed to have no dependents inside Cargo pipelining but for the compilation/linkage of the very final artifact(s) (which cargo check/clippy/doc do not produce).

[kornel]

I agree this is a problem, but adding more build-time executables adds even more problems.

It's not even a big improvement. It gives Cargo only one bit of information and one coarse lever to pull at the cost of making a more cumbersome API for crates and more expensive builds.

Build and link don't split that cleanly

"-sys" build scripts can often be configured to either find a pre-built library or build from a vendored source (as a fallback). Then they may need to run bindgen on whatever version that has been found or built. Even when bundling pre-generated bindings, the script may need to know which version of the library to target.

So you can't have the code generation part live in complete isolation without the find-or-build part. With separate scripts at best you could duplicate the find-vs-vendored logic, and hope it matches across both script runs, and doesn't get half-cached, or otherwise you may get a crashy ABI mismatch.

Build scripts themselves are expensive.

These are extra targets to build and link. Debug builds generate lots of code bloat to link.

On Windows and macOS you may have an antivirus/corporate spyware, and/or code signing checks that delay the first run of all new executables. Having more build-time executables multiplies that cost.

Cargo has an annoying non-composable interpretation of rerun-if-changed, which forces build-time libraries to have bad defaults, which makes accurate caching of build scripts even harder. And aligning the cache directives across two scripts that must agree is even harder. There could be some config-syncing protocol between the build and link scripts, but that depends on build script authors knowing about it and using it.

And if the build script doesn't tell Cargo that it's cacheable, then Cargo is forced to re-run it and then rebuild everything that depends on it.

Build scripts have a crude API for bash, not Rust

Cargo's protocol for communication with build scripts is a combination of env vars and a custom line-based microsyntax. It has no type safety. No efficient way to communicate back and forth with Cargo. It makes scripts mostly opaque. It all relies on every author implementing the custom text protocol from scratch correctly.

I've tried to make build scripts support a totally basic thing: error messages with more than one line of text. But because of the custom in-band DIY protocol, it was a long bikseshed endeavour that ultimately failed. Passing of a string with \n is a multi-year effort with no acceptable solution! I don't want more features built on this foundation!

This should have been a native Rust API. It isn't a Rust API only because in Cargo's early history build scripts were actually in bash, not Rust. And now Rust programs suffer a bash API with all the problems that bash APIs have and none of the benefits that Rust APIs have.

So I'd rather see the build scripts in their current form scrapped entirely, rather than this cumbersome API proliferate^[1]

---

1. some turing-complete pile of hacks is still necessary to deal with all the snowflake C libraries to replicate the work done by Linux distros for all the platforms that aren't Linux and libraries that aren't in a distro, but I'd rather use a Rust API for it ↩︎

[epage]

So building off of Kornel's post that that route won't work, another alternative is telling build scripts that linking won't happen (we have an Issue requesting this). We also have an issue about reusing cargo check in cargo build. These are in conflict and we need to decide which path to go down.

[epage]

Might be good to post a separate post with a more concrete idea, including how the processes would communicate, how to keep build times down, etc.

Note we now have an official build-rs package for wrapping the protocol.

[CAD97]

Potentially wild idea — a new buildscript directive that tells Cargo "everything is ready to build the crate" that allows the crate to be built in parallel to the rest of the build script, which is only allowed to emit directives (and built objects) that affect linking and/or Cargo's rebuild heuristics after that point. This would allow any calls to other build systems to not block the pipeline, hopefully improving parallel availability instead of bottlenecking on compiling external code that may not even be needed for the current (e.g. check) build.

[epage]

See https://github.com/rust-lang/cargo/issues/16062

[bjorn3]

@CAD97

which is only allowed to emit directives (and built objects) that affect linking

That would also mean no static libraries can be built after this point unless they are explicitly marked as nobundle (is that even stable?).