As we come down the home stretch for 2017, @michaelwoerister and others have been hard at work getting incremental compilation ready to ride the trains to stable. But we need your help to get it across the finish line!
Here’s how you can help:
Grab a project with poor compile times
Do cargo +nightly build --release
Do:
rm -rf target, then
time cargo +nightly build --release
Do:
rm -rf target, then
CARGO_INCREMENTAL=1 time cargo +nightly build --release
Do:
touch lib.rs, then
CARGO_INCREMENTAL=1 time cargo +nightly build --release
Do:
A trivial change to a method body (like adding a no-op), then
CARGO_INCREMENTAL=1 time cargo +nightly build --release
Post the project and numbers here!
We want to hear about the timings you see, in part so that we can beef up the benchmark suite with a much wider range of examples. In the next several weeks, we expect some major improvements to incremental to land (e.g., being able to skip typeck), and we’d like to track the improvements to these numbers as that happens.
Note: these timings are based on the release profile because, today, incremental doesn’t help much with debug. That will be changing in the very near future.
For compiler team members: beyond extra benchmarks, what improvements to http://perf.rust-lang.org/ would be most helpful to track performance in incremental mode? Is it enough to have an easy-to-get “view”, using today’s knobs but not having to set them yourself? Or is there a view of the data you want but can’t get with today’s perf?
For everyone: if you have more thoughts about what/how we should be measuring or presenting measurements, please speak up!
This is for a moderately sized project I’ve been developing at work (can’t share the repo though ). Commit bcc8172.
$ unset CARGO_INCREMENTAL
$ rustc +nightly --version --verbose
rustc 1.23.0-nightly (2be4cc040 2017-11-01)
binary: rustc
commit-hash: 2be4cc040211a85b17f21e813ff62351ae4de642
commit-date: 2017-11-01
host: x86_64-unknown-linux-gnu
release: 1.23.0-nightly
LLVM version: 4.0
# point 3
$ rm -rf target
$ time cargo +nightly build --release
1570.75s user 10.33s system 271% cpu 9:42.47 total
# point 4
$ rm -rf target
$ CARGO_INCREMENTAL=1 time cargo +nightly build --release
1608.94s user 12.92s system 349% cpu 7:43.97 total
# point 5
$ touch lib.rs
$ CARGO_INCREMENTAL=1 time cargo +nightly build --release
2.51s user 0.21s system 100% cpu 2.691 total
# Point 6
# Add "let _x = 5;" to one of the files
$ CARGO_INCREMENTAL=1 time cargo +nightly build --release
2.77s user 0.20s system 107% cpu 2.766 total
rm -rf target/ && time cargo +nightly build --release
real 1m43.398s
user 7m44.359s
sys 0m16.399s
rm -rf target/ && CARGO_INCREMENTAL=1 time cargo +nightly build --release
87.72 real 486.96 user 18.64 sys
touch src/main.rs && CARGO_INCREMENTAL=1 time cargo +nightly build --release
1.63 real 1.27 user 0.34 sys
CARGO_INCREMENTAL=1 time cargo +nightly build --release
2.51 real 2.75 user 0.36 sys
time cargo +nightly build --release
7.94 real 7.60 user 0.31 sys
The numbers for step 7 are measuring undoing the previous noop change (I added a (); line to a function in src/main.rs) and seeing how long that takes without incremental compilation enabled.
Hi,
I’m working on one private project which performs some data processing and computation,
here’s a compilation speed comparison result:
94.353 total
59.860 total
3.761 total
3.956 total
I’m always benchmarking this app and it is worrying that I found that the incrementally compiled version has significantly worse performance than the normal one.
Normal release version processing time: 0.140sec total
CARGO_INCREMENTAL=1 release version processing time: 0.512sec total
Which means 3.6x slowdown for the incrementally compiled version. Is this expected?
The interface is ridiculously sluggish. The only way to use it is to click Hide All first thing and then enable various tests. Also, if you want to see why some spike occurred, you have to click-and-drag to zoom in and then select the exact point where performance dropped. It feels like these very common operations could be more streamlined. But honestly just being faster would be a big part of that.
It'd be useful, I think, to have the ability to group the tests in some logical ways so they can one can enable/disable test suites instead of individual tests.
Further, right now for a single incremental test foo we actually have N measurements that are displayed as if they were individual tests. In particular, things like foo@010-bar and foo@020-baz`. These are successive measurements of a single directory being built and rebuilt after changes have been applied. It would be very useful to group them together in some way so that they are enabled/disabled as a group.
The convention is:
000-baseline is a non-incremental build
010-incr is a "cold" incremental build with no incremental state
020-clean is a second incremental build with no changes -- kind of the "best case"
further changes are applying various diffs, always relative to the previous build
Therefore, comparing e.g. 020-clean to 000-baseline tells you how much faster an individual change was relative to today.
The default display normalizes all lines relative to themselves. This is perhaps useful for incremental tests but misleading: it seems like it would be useful to normalize foo@020-baz relative to foo@000-baseline -- i.e., how long did it take to do some incremental change, relative to the base, non-incremental build? Or at least it ought to be easy to get that data out. (You can change to absolute measurements now, but then you have to guesstimate the percentages if you want that.)
It'd be great to have the ability to generate a (short) URL that can be given so that other people can see the same view that you are seeing.
Finally, not a UI thing, but:
It'd be great to measure disk space consumed by the incremental directory. Even better if we can chart that depending on the kind of file.
Not expected, but we have seen incremental makes things slower, particularly in crates where there is just one module (which means that we don't get good re-use, but we still pay the overhead of tracking). We plan to make some changes to address this -- still, 3.6x is a lot! How private is this project? Maybe we can have you run some more detailed commands to get further data? If nothing else, running with -Zincremental-info might be helpful.
). Commit 
Maybe we can have you run some more detailed commands to get further data? If nothing else, running with