Distributed build systems for Rust

[kaseyc]

Dealing with the build times for the compiler made me wonder if anyone has worked on some sort of distributed build system for Rust (like Incredibuild or distcc), or if it is feasibly doable with Rust’s compilation model, though I can’t see why it wouldn’t be. Alternately, how difficult would it be to add support for Rust to an existing tool?

Having one of these systems would definitely be beneficial to the language going forward for use in production, and it would make hacking on the compiler a lot more efficient due to the reduced build times.

Thanks!

[Gankro]

I think one thing that would help is if a compiler/make wizard sat down and wrote out some more recipes like the ones found in make tips. Actually a total clean up of make tips would be great, since it seems to be largely out of date (dead links, and huge chunks of text marked “old”).

That and perhaps a discussion of how to get the most out of your builds. Things like:

• How do I manage my repository to get the most out of NO_REBUILD=1? If I have a full build, and want to apply a patch and then minimally rebuild, what is the best way to do that? If I jump between two different conflicting patches, will that work? How divergent do two branches need to be before I can’t correctly jump between them and NO_REBUILD? How smart is NO_REBUILD?

• Should I be feeding -j[num processors] or anything else into Rust’s make?

• Should I be feeding other optimization flags or anything else besides PLEASE_BENCH=1 if I intend to run any of Rust’s benchmarks?

• What exactly do the different compiler stages do? Why would I want to only build a specific stage? When can I only build a specific stage?

[DiamondLovesYou]

From my experience, the big killer of build times is librustc itself. That said, I would bet that paralleling both rustic::middle::trans && the llvm optimization passes would result in a reasonable speedup over a single thread, while being relatively easy to implement (though not actually distributed like you requested).

My point is that there is likely little to gain from distributed-ness w.r.t. Rust proper.

[l0kod]

Ninja should be an inspiration for rustc: https://martine.github.io/ninja/

[BenoitJGirard]

I’d love support for a compilation cache, so that recompiling a crate with the same code and the same options just fetches it from the cache if available,

I’ve seen that approach taken, with the cache being a key/value database, to distribute compilation of C++ object files and it works really well. Key was hash of cpp file fed to the precompiler plus command-line arguments to the compiler; before compiling a file a local file cache was consulted, then the database.

[l0kod]

Rust could even cache compiled modules instead of files or maybe even smaller piece of code like functions to improve and optimize repetitive compilations.

[kaseyc]

According to this page rustc should support ccache. Though I’m not sure of it still works,or if there is more you need to do to use it than pass -enable-ccache to ./configure.

[l0kod]

Incremental recompilation: https://github.com/rust-lang/rust/issues/2369

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.