Pre-RFC Allow cargo build --dependencies-only

Docker build cache doesn't work with cargo build.

Please read more details in original ticket https://github.com/rust-lang/cargo/issues/2644 opened by nagisa opened this issue on May 4, 2016

Why? Because Docker cache is handled by checksuming all added resources using COPY ( see https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache ) and then if anything si changed in added resources in future build run all cached instructions after this COPY are invalidated and executed again.

Example of current build:

 FROM: rust

 COPY . /build/

 RUN cargo build

This will download all dependencies on every docker build commend when any file in project is changed. There are several ways how to solve this e.g. use experimental build feature to mount dependencies from developer host machine into docker build, but this is still considered experimental and filesystem mounting is not simple between host machine and docker vm on platforms other than Linux.

Example in case of --dependencies-only support:

FROM: rust

COPY Cargo.* /build/

RUN cargo build --dependencies-only

COPY src /build/

RUN cargo build

This would allow to leverage Docker cache which will get invalidated after Cargo.* files are changes which is expected behaviour. As workaround one could create fake source file and build fake hello word project and then build with proper src files but this is really ugly workaround.

Let's add support to install only dependencies to simplify ergonomics of building rust projects not just in docker.

12 Likes

I think thid would be a good idea. Docker is used a lot out there in the world and I can imagine that there are other tools that would also love have this. It doesn't seem that difficult to implement either, but I might be wrong about that.

The "inverse" of this — cargo clean --workspace-local maybe — is also useful for cache-at-end workflows like CI tends to be. Rust-analyzer has a special pre-cache step that manually cleans (partial) artifacts from local crates to remove the often changing local files and only cache dependencies.

On Travis this causes cache-hits on the save, meaning that the cache doesn't have to be re-saved. On GH Workflows, each cache is uniquely keyed so this isn't required to avoid re persisting caches, but it does still save cache space.

3 Likes

Hello, I have implemented this feature, by adding an additional package selection option, which I named --only-remote. It allows splitting the build into 2 stages:

  1. cargo build --only-remote -Z unstable-options
  2. cargo build

First execution downloads and compiles all non-local dependencies (including transitive), which makes the second run much faster, as it builds only local sources and links them.

The implementation is bit different than the implementation of the existing package selection options, namely --workspace, --package, --exclude, because if they exclude a package, they exclude also their dependencies. The --only-remote option on the other hand, if it decides to exclude a package, it keeps processing its dependencies. So this is a major difference at the implementation level. Nevertheless all the options can be safely combined.

More details at

Please let me know how to proceed. Do you want me to make a PR out of the implementation? I also wouldn't mind if somebody else would take my implementation and refine it to follow this project's standards for documentation and testing.

7 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.