Good evening all! The parallel rustc working group (you can find us on Zulip!) has been developing and working on a parallel compiler for quite some time now, and we're getting very close to shipping! What we'd like to do now is lay out a plan for how we can get users testing this before we turn it on by default.
Note: if you're not familiar, the parallelism in the "parallel compiler" is specifically referring to parallelism in the frontend static analysis (think typeck, borrowck, etc). Rustc already has a parallel LLVM backend, and that's here to stay and isn't changing.
We would like to start shipping nightlies as soon as we're confident that parallelism is ready to get turned on by default. We would like user testing before that, however. Unfortunately unlike other compiler features this isn't a simple flag to pass to the compiler, but rather you need to recompile the whole compiler in "parallel mode". Nightlies are not currently built in "parallel mode", and as a result can't be used to test parallelism.
We do, however, want users to be able to easily test these changes! That typically means using
rustup in one form or another. To that end the parallel working group would like to propose the following plan for testing parallel rustc:
- One day this week (tomorrow, Tuesday if possible) we will land a PR to the master branch of rust-lang/rust which enables the parallel compiler by default. This would make its way to the nightly channel, for example
- Immediately after the nightly is published, we revert this default, switching the master branch to what it is today (no parallelism in the frontend).
- Next, make a follow-up post on internals about gathering data. This way users would
rustupto one nightly to test parallel rustc, and use the previous nightly to have a baseline measurement of what today's performance looks like (assuming no major changes land in that one 24 hour window)
Would others be ok with this plan? We're relatively confident that the whole ecosystem won't break if one nightly has parallel turned on by default, but if lots of breakage does happen it's at least only one day of breakage! Note that we also want to get this done while we're all still working, it'd be disastrous for something like this to go out over the holidays when most aren't around to fix bugs!