Evaluating pipelined rustc compilation

Thanks for working on this! It's great! I posted about this a few months ago so I'm pleased to see you've implemented more or less exactly what I was thinking about.

I'm glad that there are no perf regressions, but I'd be surprised to see much improvement on machines with <=4 cores. But the 10+ core machines often end up starving when there's a bottleneck crate in the middle of the dep graph.

Even though the .rmeta is being used as the interface, I'm assuming that it has internal implementation details, since Rust relies a lot on its inlining, and it also needs to make sure the SVH is propagated through properly. I'm going to implement pipelining in Buck when I get the chance, and since Buck is entirely content/content-hash driven, it will demonstrate this one way or the other.

2 Likes