I definitely think that being able to manually control what gets rebuild would be useful. Another very common example for me is that I change some #[test] fn, which requires rebuilding a ludicrous amount of stuff. To some extent incr. comp. will help here, but due to the nature of bootstrapping less than you might think – so long as we are still committed to preserving correctness.
I’m not sure that “roles” is the way to think of it. I feel like just having flags you can pass to rustbuild telling it to ignore the fact that things are out of date, or specialized targets that start from a given point regardless of what is out of date, might be good enough. We just have to document them well, and make clear their shortcomings.
This has the advantage that you start out with the safe targets and kind of “opt in” to the more advanced ones.
Still, thinking about “roles” – or at least common use cases – is a great idea to come up with what those targets are.
Ah. I see. Actually, it appears I’ve tried this before (based on some bugs I’ve filed).
Unfortunately, this isn’t accessible to new devs as it’s not really documented anywhere except ./configure --help and, on many distros (Arch at least) requires an additional --disable-codegen-tests flag due to some LLVM binaries that aren’t distributed by default (but the only way to figure this out is to google around and read old bug reports).
Maybe we should automatically disable codegen tests if FileCheck is not found. At the moment, debuginfo-gdb (respectively, debuginfo-lldb) tests are disabled if gdb (respectively, lldb) is not found. We could handle codegen tests similarly.
What is rustbuild’s story around build parallelism?
Right now if I want to utilize all of my CPU cores, I need to use make -jN, which works ~okay, though occasionally things fail, presumably because dependencies are not specified 100% correctly. Re-running make usually fixes it.
Unfortunately this breaks down on compiletest, which does its own parallelism, so we end up with N^2 threads of execution instead of N.
Would be nice if our build system was integrated end-to end in this respect.
The rustbuild build system currently relies purely on parallelism in the tools that it calls, it doesn’t actually invoke anything in parallel yet (but that may change). Each step of the build, though, always maxes out cores (e.g. cargo or libtest suites), so it ends up not mattering much.
I submit PR to rustc from time to time, and figuring out what is the correct make command to run interesting tests is always a mystery to me.
Could there perhaps be a script which analyses changes and suggests what is the most appropriate command to run? Like, if you edit parser.rs, then it should suggest syntax tests, if it’s standard library, then make check-stage1-std NO_REBUILD=1, etc.
And is there some way to avoid building LLVM? I also have to use @stebalien’s algorithm because of this
Maybe so. There’s already a small section about configure options. And I forgot that you might need --disable-codegen-tests to go with it, as @stebalien mentioned, if your LLVM doesn’t have FileCheck. (e.g. Fedora doesn’t package that.)