Feature Request to decrease compilation time

I was programming on an external program thinking why every time we do cargo check, build or run it recompiles every package in the project. The principal problem is that every file has to recompile even if it doesn't have any changes, so why not to store the last md5 or hash algorithm of the entire package and compare to the current code stored, and if are equal just reuse the old compiled code. I think this will increase compilation time in very big projects, and prevent from recompiling code that already has been compiled.

Maybe this is already in use, thanks for all!!

I'm going to assume that you meant that you want to decrease compilation time rather than increase. Cargo checks for every compilation artifact if the mtime is newer than all source files from which it was built. It will only rebuild if the mtime is older than any source file. This is the same technique used by make. If things get rebuilt despite this you can set the CARGO_LOG env var to cargo::core::compiler::fingerprint=info to cause cargo to produce debug logs showing why cargo thinks that something needs to be recompiled. For example if I create a new project build it and touch src/main.rs this will give:

[2022-10-15T12:07:01Z INFO  cargo::core::compiler::fingerprint] stale: changed "/tmp/foo/src/main.rs"
[2022-10-15T12:07:01Z INFO  cargo::core::compiler::fingerprint]           (vs) "/tmp/foo/target/debug/.fingerprint/foo-f846d772ae2588d4/dep-bin-foo"
[2022-10-15T12:07:01Z INFO  cargo::core::compiler::fingerprint]                FileTime { seconds: 1665835616, nanos: 26793929 } != FileTime { seconds: 1665835620, nanos: 694808564 }
[2022-10-15T12:07:01Z INFO  cargo::core::compiler::fingerprint] fingerprint error for foo v0.1.0 (/tmp/foo)/Build/TargetInner { name: "foo", doc: true, ..: with_path("/tmp/foo/src/main.rs", Edition2021) }
[2022-10-15T12:07:01Z INFO  cargo::core::compiler::fingerprint]     err: current filesystem status shows we're outdated
   Compiling foo v0.1.0 (/tmp/foo)
    Finished dev [unoptimized + debuginfo] target(s) in 0.29s
3 Likes

Yes i was thinking about decreasing, miss that. Thanks for the reply, is the same logic but using another method.

Thats clever!!

Another way to increase performance is to use multithreading (i don't know if is in use), but maybe each component depends on another, so maybe this is quite tricky.

Cargo also runs rustc instances that don't depend on each other in parallel. Rustc itself is partially parallelized. Optimizations by LLVM happen in paralle, but the frontend is unfortunately not yet parallelized. There is a flag to build a parallel rustc, but it still has some bugs and the effort to work on it has mostly stalled.

If you are observing that cargo rebuilds every package every time you run any build, then that is not normal behavior. It should only rebuild (and list in the output) the packages that you modified (or packages that depend on a package you modified).

It might be a bug, or it might be something weird like your filesystem not supporting modification times, but it also might be some problem with how your project is set up. If you're in fact seeing this then it would be worth making a post on https://users.rust-lang.org/ to ask for help with that, with more details about your project setup (preferably the full source code if it's public).

I've observed this with editor support tools like ALE and such. They run cargo $command (usually check) in the background and may clobber your build tree. I enhanced ALE for this in this PR, but other tools may need similar help.

Unless the editor is also specifying different compilation options, the worst that should do is blocking other commands, not causing rebuilds.

Looking over the comments and commit message, that could have been it. I think it may have been cargo-tarpaulin ping-ponging build state then.