Exploring Crate Graph Build Times with `cargo build -Ztimings`

This is a great feature to have. Thank you :heart:

I almost missed this as it was already off of the front page of irlo, luckily the weekly summary email had it in there. Here’s a bump for anyone else who missed this.

1 Like

This looks great!

Here is Servo build from cargo clean on a 28-thread machine, with "Min unit time" set to 10 seconds. (There are 658 units in total.) We already knew parallelism opportunities were low at the end of the build, but it’s striking to see this "CPU Usage" graph!

5 Likes

I tried that and did ./x.py build --stage 2 src/rustc and it didn't work. At least, no output was produced.

Will x.py be using the installed nightly Cargo, or does it use its own version?

x.py default to downloading and using the compiler specified in src/stage0.txt. You can copy config.toml.example to config.toml and specify cargo and rustc keys in the [build] section to replace it. (I think there’s a command-line equivalent, but I don’t know where it’s documented.)

1 Like

Would it be possible to output SVG instead of using a canvas? One of my concerns is being able to quickly find (as in Ctrl-F) crates in the build timeline.

2 Likes

It was originally written with SVG, but the rendering performance was unbearable.

1 Like

From a naive look, it seems that this "script" crate should be your next build optimization target :wink:

(And if it cannot be broken down in smaller crates easily, it also seems like an excellent test case for the ongoing rustc parallelization work)

Thank you for this observation. I’ll just leave this here: https://github.com/servo/servo/issues/1799

5 Likes

Time to give that poor abandoned issue the attention that it deserves, then :slight_smile:

Ugh. If I have to spell it out…

That the script crate takes a long time to compile is not news. As the issue shows, we were discussing it five years ago. -Ztimings helps quantify exactly how bad it is, but Cargo has always printed Compiling… lines and you don’t need a precise timer to feel that multiples minutes is long, especially when you’re in the middle of the usual edit-compile-test cycle.

Everyone who contributes to Servo feels this pain. To suggest that it didn’t occur to us to try to improve the situation or that we merely failed to give it a little attention is, frankly, a bit insulting.

The reason this is still bad is that actually fixing it is really hard. Crates cannot have a dependency cycle, but items within a crate can. We had a GSOC student work multiple month on a possible approach, but that did not turn out as well as we’d hoped.

7 Likes

I tried this and again failed. If anyone gets it working and can give specific instructions, I would be grateful. I opened a PR to get this directly supported in x.py.

Do you have a link to the SVG version? I have some people who would be interested in that perf problem.

3 Likes

I'm sorry, I never got a notification for your response. (For some reason irlo discourse notifications are very spotty for me.)

Here's one using SVG: https://gistpreview.github.io/?dcccceb93a87316fcf455f7786ddb333

Here's one using Canvas: https://gistpreview.github.io/?65f4961cf2ac77e96f456ce02d1fddb2

If you scroll around, the SVG one is often blank and then pops into place, sometimes taking a second or two. It depends on which browser/os/gpu you have, but even on somewhat fast machines it was very noticeable to me.

You can check out commit 095f154f3753a1e9c724ec6b6d0abc6ce4ef2dc6 of the cargo repo to get the last version that had SVG support (it was removed in 77a47b3e3282c549192e96e71c4f535baccb9e0b).

1 Like

Oh, I love the SVG version. It draws instantly on both Firefox and Chromium with my iGPU, and it's searchable. True, the Firefox Dev Tools seem to barf on it, but that also happens on the canvas version.

I see the rendering delays, thanks. I'll pass this on to some people I know to try to improve the Chromium performance, at least. :slight_smile:

@ehuss This is being tracked in https://bugs.chromium.org/p/chromium/issues/detail?id=1025566 -- the recommendation is to break up the grid into separate paths to allow the algorithm to optimize better. That bug will stay open to track improving things without needing changes to the SVG.

FYI, that link isn't public.

1 Like