Help test parallel rustc!

Testing Beamium (a Prometheus to Warp10 metrics forwarder) at commit 819ef8c:

cargo +nightly-2019-12-18 build
534,97s user 30,78s system 604% cpu 1:33,65 total

cargo +nightly-2019-12-17 build
517,03s user 26,19s system 577% cpu 1:34,11 total

cargo +nightly-2019-12-18 build --release
1319,95s user 29,98s system 515% cpu 4:22,00 total

cargo +nightly-2019-12-17 build --release 
1356,64s user 28,18s system 496% cpu 4:39,15 total

cargo +nightly-2019-12-18 check
341,53s user 26,41s system 689% cpu 53,353 total

cargo +nightly-2019-12-17 check 
338,01s user 22,66s system 654% cpu 55,137 total

On Rust stable, just in case (1.40.0):

cargo build
526,18s user 27,23s system 619% cpu 1:29,33 total

cargo build --release 
1322,03s user 27,14s system 520% cpu 4:19,12 total

cargo check
354,53s user 23,55s system 630% cpu 1:00,01 total

Tested on Dell XPS 15 (9550):

  • IntelĀ® Coreā„¢ i7-6700HQ CPU @ 2.60GHz
  • 16GB of RAM
  • Ubuntu 19.10 64bits

Timing for grayarea-runtime on 40 Intel CPU 32Gb RAM Ubuntu 19 shows about 25% perf boost for debug build and check and 10% for release builds:

+nightly-2019-12-17 build
real    1m22.457s
user    8m46.634s
sys     0m37.100s
+nightly-2019-12-18 build
real    1m2.301s
user    10m8.246s
sys     0m51.905s
+nightly-2019-12-17 build --release
real    1m51.374s
user    26m32.222s
sys     0m42.592s
+nightly-2019-12-18 build--release
real    1m37.700s
user    29m30.259s
sys     0m58.299s
+nightly-2019-12-18 check
real    1m5.725s
user    5m40.797s
sys     0m26.380s
+nightly-2019-12-18 check
real    0m44.395s
user    6m37.955s
sys     0m39.464s

P.S. Give it first dry run to download deps

Building substrate 5dece712a on AMD EPYC Rome (64 cores/128 threads), 256 GB machine.

cargo clean && time cargo +nightly-2019-12-17 build
    Finished dev [unoptimized + debuginfo] target(s) in 6m 15s

real    6m17.934s
user    135m2.361s
sys     17m1.909s
cargo clean && time cargo +nightly-2019-12-18 build
    Finished dev [unoptimized + debuginfo] target(s) in 6m 29s

real    6m32.575s
user    228m14.238s
sys     32m0.462s
cargo clean && time cargo +nightly-2019-12-17 build --release
    Finished release [optimized] target(s) in 8m 38s

real    8m41.842s
user    411m59.930s
sys     23m30.786s
cargo clean && time cargo +nightly-2019-12-18 build --release
    Finished release [optimized] target(s) in 8m 45s

real    8m48.119s
user    537m58.740s
sys     45m5.127s
cargo clean && time cargo +nightly-2019-12-17 check
    Finished dev [unoptimized + debuginfo] target(s) in 4m 46s

real    4m49.572s
user    154m7.090s
sys     20m57.464s
cargo clean && time cargo +nightly-2019-12-18 check
    Finished dev [unoptimized + debuginfo] target(s) in 4m 26s

real    4m29.848s
user    210m40.669s
sys     39m16.688s

Running cargo +nightly-2019-12-18 check with RUSTFLAGS="-Zthreads=64" I am getting:

    Finished dev [unoptimized + debuginfo] target(s) in 20m 54s

real    20m57.729s
user    746m42.182s
sys     1532m35.727s

The second time I ran it I got

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: "SendError(..)"', src/libcore/result.rs:1188:5
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:84
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:61
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1025
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1426
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:65
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:50
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:193
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:210
  10: rustc_driver::report_ice
  11: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:475
  12: rust_begin_unwind
             at src/libstd/panicking.rs:375
  13: core::panicking::panic_fmt
             at src/libcore/panicking.rs:84
  14: core::result::unwrap_failed
             at src/libcore/result.rs:1188
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.41.0-nightly (3ed3b8bb7 2019-12-17) running on x86_64-unknown-linux-gnu

note: compiler flags: -Z threads=64 -C debuginfo=2 --crate-type lib

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack

Testing sv-parser at 6e887e2. The testing environment is Intel Xeon Gold 6134 CPU @ 3.20GHz (16cores/32threads) / 1.5TB DRAM. About 15-20% improved.

cargo clean && time cargo +nightly-2019-12-17 build

real    2m35.962s
user    5m6.225s
sys     0m21.336s

cargo clean && time cargo +nightly-2019-12-18 build

real    2m10.055s
user    5m40.738s
sys     0m26.435s

cargo clean && time cargo +nightly-2019-12-17 build --release

real    2m49.236s
user    16m11.879s
sys     1m29.441s

cargo clean && time cargo +nightly-2019-12-18 build --release

real    2m24.627s
user    17m15.008s
sys     1m26.060s

cargo clean && time cargo +nightly-2019-12-17 check

real    2m5.321s
user    2m55.150s
sys     0m13.223s

cargo clean && time cargo +nightly-2019-12-18 check

real    1m41.439s
user    3m26.564s
sys     0m15.532s

On my 2-cpu 8-thread server (and old xeon E5), all of my projects were faster with the 2019-12-18 nightly (across all projects, best was 1.51 times faster for check --release, worst as 1.01 times faster for another check --release). In no cases was it slower.

Adding another set of data points, I just grabbed a few repos people were using in this thread. Seeing awesome improvements, nice job to the team working on this.

Hardware

  • Processor: AMD 3970X
  • Memory: 64Gb DDR4 @ 3000MHz C15
  • Storage: PCIE-Gen4 Corsair MP600

Results

sv-parser @ 6e887e:
  2019-12-17 Build: 1:57.42
  2019-12-18 Build: 1:37.97

crates-io @ e0d149:
  2019-12-17 Build: 0:57.62
  2019-12-18 Build: 0:42.93

turtle @ 9efd23:
  2019-12-17 Build: 0:22.69
  2019-12-18 Build: 0:12.23

Build of Materialize is 5% faster on 2019-12-17.

FreeBSD 13.0-current, 32-core (64-thread) 2990wx, 64gb RAM.

cargo +nightly-2019-12-17 build --release  6592.05s user 372.31s system 2733% cpu 4:14.79 total
cargo +nightly-2019-12-18 build --release  7017.31s user 436.78s system 2791% cpu 4:27.08 total

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.