Brainstorming improvements for new rustc devs

For me, better support for out-of-tree (i.e., my distros package) LLVM would be nice.

This works for me, with ./configure --llvm-root=/usr/lib/llvm-3.8. If it doesn't work for you, please file a bug. This is supposed to work. I agree it is important to keep this working.

My understanding is that lldb-350 is buggy and lldb-360 is fixed.

Right - though lldb-350 seems to be the currently available version as updating xcode via the App Store does not update it beyond that.

I definitely think that being able to manually control what gets rebuild would be useful. Another very common example for me is that I change some #[test] fn, which requires rebuilding a ludicrous amount of stuff. To some extent incr. comp. will help here, but due to the nature of bootstrapping less than you might think -- so long as we are still committed to preserving correctness.

I'm not sure that "roles" is the way to think of it. I feel like just having flags you can pass to rustbuild telling it to ignore the fact that things are out of date, or specialized targets that start from a given point regardless of what is out of date, might be good enough. We just have to document them well, and make clear their shortcomings.

This has the advantage that you start out with the safe targets and kind of "opt in" to the more advanced ones.

Still, thinking about "roles" -- or at least common use cases -- is a great idea to come up with what those targets are.

Ah. I see. Actually, it appears Iā€™ve tried this before (based on some bugs Iā€™ve filed).

Unfortunately, this isnā€™t accessible to new devs as itā€™s not really documented anywhere except ./configure --help and, on many distros (Arch at least) requires an additional --disable-codegen-tests flag due to some LLVM binaries that arenā€™t distributed by default (but the only way to figure this out is to google around and read old bug reports).

Maybe we should automatically disable codegen tests if FileCheck is not found. At the moment, debuginfo-gdb (respectively, debuginfo-lldb) tests are disabled if gdb (respectively, lldb) is not found. We could handle codegen tests similarly.

What is rustbuildā€™s story around build parallelism?

Right now if I want to utilize all of my CPU cores, I need to use make -jN, which works ~okay, though occasionally things fail, presumably because dependencies are not specified 100% correctly. Re-running make usually fixes it.

Unfortunately this breaks down on compiletest, which does its own parallelism, so we end up with N^2 threads of execution instead of N.

Would be nice if our build system was integrated end-to end in this respect.

The rustbuild build system currently relies purely on parallelism in the tools that it calls, it doesnā€™t actually invoke anything in parallel yet (but that may change). Each step of the build, though, always maxes out cores (e.g. cargo or libtest suites), so it ends up not mattering much.

I submit PR to rustc from time to time, and figuring out what is the correct make command to run interesting tests is always a mystery to me.

Could there perhaps be a script which analyses changes and suggests what is the most appropriate command to run? Like, if you edit parser.rs, then it should suggest syntax tests, if itā€™s standard library, then make check-stage1-std NO_REBUILD=1, etc.

And is there some way to avoid building LLVM? I also have to use @stebalienā€™s algorithm because of this :frowning:

Use --llvm-root=/usr from your distro, or even point to your own cached build -- assuming the thing you're testing is independent of exact LLVM versions.

1 Like

Should this perhaps be documented in CONTRIBUTING.md?

Maybe so. Thereā€™s already a small section about configure options. And I forgot that you might need --disable-codegen-tests to go with it, as @stebalien mentioned, if your LLVM doesnā€™t have FileCheck. (e.g. Fedora doesnā€™t package that.)

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.