My understanding is that stage0 refers to the very initial bootstrap compiler. It can be downloaded by x.py (the default), or with some configuration, it can be provided by the system instead. However, under no circumstances should any part of it be compiled by x.py - it always comes from somewhere else.
However, this is not what I find. In my setup, I do not use the default download-the-bootstrap-compiler behavior. Instead I’ve installed a suitable version of cargo and rustc elsewhere, and set the rustc= and cargo= options in config.toml to point to them.
When I run x.py build --stage 0, I would expect that nothing happens - I’ve already provided a complete bootstrap compiler. However, it appears that this builds the full compiler. It populates the directories build/x86_64-unknown-linux-gnu/stage0-{rustc,std,sysroot,test}. There’s a fully working compiler at build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/rustc, which does reflect changes I’ve made to the source code. It doesn’t have RPATH set up though, and I have to invoke it with LD_LIBRARY_PATH pointing to each of build/x86_64-unknown-linux-gnu/stage0-{rustc,test,std}/x86_64-unknown-linux-gnu/release/deps.
So, what’s does stage0 actually mean? If I want the fastest edit-compile-test loop possible, is it always enough to build and test stage 0?
When you build --stage 0, that builds all components using the downloaded or provided stage0 compiler. If you then build --stage 1, it will use that just-built rustc to build the components again.
It may be enough to just build and test stage 0, but that depends on whether you’re making changes near any transitioning feature. You’ll see cfg(stage0) attributes around the code which lets it straddle the behavior between old and new compilers, especially as language features change.
Ok, that makes sense. Do you (or others) work with --stage 0 to speed things up? It seems like it’s not really a supported workflow, given the need for running the rustc with LD_LIBRARY_PATH.
This seems to work pretty well, about 2x as fast for me compared to python x.py build --stage 1 (15 mins vs 31 mins).
One other thing, I had to run python x.py build src/libstd --stage 1, and provide the path of the resulting stage1 libstd.rlib to rustc with -L, but that only takes about 1 extra minute to compile.
The stageN-{std,rustc,...} directories contain artifacst compiled by “the stage N compiler” and are copied into build/$triple/stage(N+1) to form a working compiler for the next stage of the bootstrap process. Terminology is a bit confusing here but the first rustc binary that gets built (and the first that reflects) is called the stage1 compiler and ultimately resides in build/$triple/stage1. The stage0 compiler is the downloaded binary, and is used to compile the contents of the stage0-{std,rustc,...} directories, sometimes also called “stage0-out”.
As you’ve noticed, working with the stageN-out directories is a bit involved, it’s much easier to let the build system move everything into stage(N+1) and use that. You need to pass --stage 1 to x.py for that to happen. Unfortunately, …
x.py build --stage 1 builds rustc twice, which is actually required for compiler plugins. x.py build --stage 1 src/libtest will give you an otherwise-working rustc sysroot in src/$triple/stage1 (building only src/libstd instead of src/libtest may also work, but it saves only a bit of time and the resulting rustc can’t run unit tests). You can also symlink this directory with rustup toolchain link to be able to run it more easily and get a cargo for free.
Thanks spearman, but could you explain a bit more? Why did you have to build src/libstd --stage 1? Do you always do that or only in certain circumstances (like if you’re testing a change to the std lib)? And could you give an example command line that you use to point to libstd.rlib with -L?
Ok, thanks rkruppe, that does clear some things up.
If I
am not dealing with any compiler plugins
am not concerned about differences related to cfg(stage0)
don’t mind the LD_LIBRARY_PATH shenanigans (I’ll just make a little wrapper script)
is there still any reason to use --stage 1, or the variant --stage 1 src/libtest?
When I tried to invoke the stage0-rustc compiler on an input file it generated the error:
error[E0463]: can't find crate for `std`
any of the libstd-*.rlibs I could find in the stage0-* directories were built with the stage0 compiler, so linking them with -L will notify that they were built with the wrong compiler:
error[E0514]: found crate `std` compiled by an incompatible version of rustc
Here is the script I used to invoke the stage0-rustc on an input file (after also running python x.py build src/libstd --stage 1): https://pastebin.com/TC5dug7V
Not sure if this is the best way to do this, I’ve only been playing around with rustbuild for a couple of days. Specifically I am making changes to the “MBE” macros in libsyntax (affecting phase 2 “configure & expand” of the compilation process), so if there is a shortcut to building and testing that part of the compiler I would be interested in any tips.
Edit: Forgot to mention I also tried with the #![no_std] attribute in my input file, but the error then was couldn’t find crate core, so I just stuck with relying on std.
As opposed to --stage 0? Well, plain --stage 1 takes far longer, so you probably shouldn't do it if you don't need it. But --stage 1 src/libtest takes only marginally longer, and while I can't point at anything specific that might be wrong about --stage 0, it's probably more likely to have weird issues than --stage 1 src/libtest. Also I don't believe rustup toolchain link will work with --stage 0.