Fellow Rustaceans I bring good news! Today’s nightly may be able to improve
compile times in release mode for your project by 2x or more, and if you’d like
to read no further, please execute the following:
$ cargo +nightly build
$ RUSTFLAGS='-C codegen-units=16 -Z thinlto' cargo +nightly build --release
and report back with (a) did compile times get faster/slower in either mode and
(b) did your project’s runtime in release mode get slower. If the answers are
yes and no, then we’d still love to know!
In this chapter of the war against Rust’s long compile times we’re going to
take a look at LLVM optimizations and code generation. Although not true for
every Rust project, more often than not I’ve seen the costliest part of a
compilation be LLVM optimizations and/or generation of machine code. In debug
mode code generation is often quite expensive and in release mode the
optimizations tend to dwarf the code generation.
As a bit of a recap of the current state of the compiler, today rustc will by
default create one codegen unit which is fed to LLVM for optimizations and
code generation. This basically means that your entire crate is compiled to one
object file which is then eventually fed to the linker at a later date. Due to
a variety of factors rustc tends to generate quite large single codegen units
which then naturally takes quite some time in LLVM!
These huge modules we’re sending to LLVM are one of the most fertile grounds
for optimization. There’s various tricks you can employ as a crate author to
reduce the size of the codegen unit sent to LLVM, but we’ve also been
investigating ways of optimizing codegen units as they are today. One of the
most obvious optimizations to look at first is parallelization! LLVM is,
however, not parallelized internally during these phases which means that while
you’re waiting for the large codegen unit to be optimized and/or translated to
machine code you’re just burning one CPU on what is likely an otherwise-idle
multi-core system.
It turns out that for quite some time we’ve actually had the ability to
run optimization passes and code generation in parallel. You can see this in
action today by configuring your project’s Cargo.toml:
[profile.release]
codegen-units = 32
This instructs rustc to split the one large codegen unit into 32 different
portions. Each codegen unit is then submitted to LLVM in parallel which ensures
that we’re always using all your cores for doing work while compiling a crate.
This sounds great, but you may be wondering why we haven’t turned this on by
default! Unfortunately it turns out that multiple codegen units don’t come
without their downsides.
One of the first problems we ran into with multiple codegen units was managing
all this parallelization across multiple instances of rustc. For example if
Cargo spawns 10 rustc processes and then each of those rustc processes spawns 30
separate threads that can quickly overload a system! To solve this we
implemented a jobserver in Cargo, an implementation of a GNU
jobserver. This, when integrated into rustc, effectively
enabled a global rate limit for Rust compilations, ensuring that cargo build -j10 indeed never had more than 10 units of work at any one point in time.
The next problem we ran into was that rustc was consuming excessive amounts of
memory! The problem here was that we would build up all codegen
units in memory and keep them all there before we submitted them to LLVM. Worse
still, while working on some codegen units all the others were just sitting idle
in memory. To solve this problem we implemented "async translation"
which enabled rustc to optimize/code generate code in parallel with translation
to LLVM. Better still we’d pause translation of code to LLVM when enough
codegen units were ready. Overall, this helped keep memory under control when
lots of codegen units came into the picture.
The final problem we’ve now faced (modulo a huge slew of other assorted bugs) is
that codegen units hurts runtime performance! Rust has almost always compiled
code in one codegen unit by default, and a large motivation factor for this is
that it enables more inlining and optimization opportunities in LLVM. When we
split one codegen unit into many then LLVM loses optimization opportunities.
Sure enough in practice this causes huge regressions as
optimization in Rust relies so heavily on inlining.
We’re hoping to solve this runtime performance problem with an implementation
of ThinLTO. Announced some time ago, ThinLTO is a novel
implementation of LTO in LLVM which optimizes for incremental compilation and
parallelization. Glossing over all a bunch of details, it basically means that
you can get the runtime benefits of a full LTO build without paying the
significant cost that comes with today’s LTO.
The primary way we’re thinking of leveraging ThinLTO right now is to regain the
performance loss of multiple codegen units when compiling one crate. This means
that when you compile a crate with multiple codegen units in release mode you’ll
initially lose a lot of inlining and optimization opportunities, but with
ThinLTO we should be able to regain all of those losses while still retaining
the parallel codegen and optimization benefits.
Now that’s quite a lot of background! Long story short is that we’re hoping to
improve compile times in rustc by ensuring that any work in LLVM uses all the
cores available on a machine. This means that the often longest part of a
compilation, LLVM, should now be parallelized and scale better with the size of
the hardware it’s running on.
Parallel code generation was enabled by default for debug mode
and is continuing to receive perf improvements. Note that the loss
of inlining opportunities doesn’t matter to much in debug mode!
The next step is to enable parallel code generation by default in release mode,
but we’re hesitant to do this until we’re sure that we won’t cause 10x
regressions throughout the ecosystem! We’re hoping that ThinLTO can help us
solve this problem, but we need your help to test it out and find bugs to ensure
it’s production ready!
The implementation of ThinLTO just hit nightly and likely has a bug or two
in it, so it’d be greatly appreciated if you could test out your local project
and see how compile times and runtimes are affected. With your feedback we
should soon be able to turn this on by default and ensure we all get benefits of
faster compiles by default!
