Idea: Enable Cargo to cache dependencies locally

tools and infrastructure
1
  1. Add Cargo.toml config line specifying path to global build cache shared for all dependencies. Add cargo command to update this path (useful for setup).
    Complication 1: what about build.rs?
  2. Ignore this default when there is a path attribute on a dependecy.

I've seen this, but sccache solution doesn't look easy to use and thus reduce language accessibility.

Main pain point solved buy this: recompiling things like actix-web take ethernity.

1 Like
2

For one thing there is a .config flag to set the target directory, and an associated env var. For another have you looked at the previous discussion of this topic?

3

I would like to have some builtin cache mechanism. sscache seems too complicated and I wasn't able to run it.

But proposal to add cache dir to Cargo.Toml is bad, as this should be configured on target computer, not in repo.

Most annoying usecase is when installing global tools, like cargo install xxx then build fails becouse of missing native dependency and you fix it, and start all build from scratch.

Dependencies that gets into target for downloaded projects represent lesser problem IMHO. I believe most downloaded projects would use different versions of crates and combination of features, so no build reuse would be possible.

P.S. considering how big space is consumed by target dir build cache would kill all free space.

4

Ok, then:

Allow cargo to operate via its own group on UNIX like systems, and via service on Windows: operating via user, who requested compilation won't allow global build cache, via separate user => security hole, specify user-local path in Cargo.toml => repository sync problems. So, unix group will remove security problem of dedicated user, but will allow multiple users to share one build cache for all the things (except for different compiler versions). For a build reuse we would probably require a kind of stable ABI for both static(this case) and dynamic(not this case, but also desirable) linking. Also, as mentioned above, feature flags will be a problem: they can change whole project, thus I don't think we could support their build caching properly with adequate effort, so I would propose simply cache each feature set as separate crate.

Total, 3 downsides of the approach:

  • Amount of implementation work;
  • Unclear situation with caching feature-flagging dependencies;
  • Unclear situation with different compiler version support;
5

Actually, we can. It just would waste disk space. Cargo already creates separate folders for different configs. If you go into target dir you'll notice that to each crate build hash is added. It consists of a compiler version and build flags. So the mechanism is in place.

This would create security issues as other users could tamper with files. I'd suggest to use cache specific to current user. Actualyl in most cases there would be single user on machine.

6

Isn't this question to machines admin?

Also, note that any user on machine can be infected by malware, in this case we also cannot protect cache anyway - separating privileges further harms usability.

Thanks for the flags detail, but this way changing compiler version would require to recompile world. We might need a stable ABI, its versioning, keep track of which versions of internal representation used in incremental cache, to recompile only if necessary. This way we could keep maximum of binary artifacts from previous compilation.

7

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.