Error after copying the entire repository directory and building

tools and infrastructure
1

I want to have two locally built toolchains built from different commits of the rust repository available at the same time. To benchmark some stuff between the two. I first tried to just copy the build/ dir after the first build, do the second build and then use rustup toolchain link to link into the two different build/ dir's stage2/ folders. But this failed. I can't remember the error...

I then tried building everything once and then copying the entire repository, checking out the other branch and building again. But this leads to build errors that I will show shortly.

What's the recommended way to keep multiple locally built toolchains around for testing and benchmarking? I'm clearly doing it wrong. I can just have two completely separate checkouts from the ground up. But I'd rather avoid compiling LLVM twice if I can avoid it.

./x.py build --stage 2
cd ../
cp -r rust rust_otherbranch
cd rust_otherbranch/
git checkout other-branch
./x.py build --stage 2

The last command from the above set of commands yield:

  running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I/fastsrc/rust_socketaddr/build/x86_64-unknown-linux-gnu/llvm/build/include" "-std=c++14" "-fno-exceptions" "-fno-rtti" "-D_GNU_SOURCE" "-D__STDC_CONSTANT_MACROS" "-D__STDC_FORMAT_MACROS" "-D__STDC_LIMIT_MACROS" "-DLLVM_COMPONENT_AARCH64" "-DLLVM_COMPONENT_ARM" "-DLLVM_COMPONENT_ASMPARSER" "-DLLVM_COMPONENT_AVR" "-DLLVM_COMPONENT_BITREADER" "-DLLVM_COMPONENT_BITWRITER" "-DLLVM_COMPONENT_COVERAGE" "-DLLVM_COMPONENT_HEXAGON" "-DLLVM_COMPONENT_INSTRUMENTATION" "-DLLVM_COMPONENT_IPO" "-DLLVM_COMPONENT_LINKER" "-DLLVM_COMPONENT_LTO" "-DLLVM_COMPONENT_MIPS" "-DLLVM_COMPONENT_MSP430" "-DLLVM_COMPONENT_NVPTX" "-DLLVM_COMPONENT_POWERPC" "-DLLVM_COMPONENT_RISCV" "-DLLVM_COMPONENT_SPARC" "-DLLVM_COMPONENT_SYSTEMZ" "-DLLVM_COMPONENT_WEBASSEMBLY" "-DLLVM_COMPONENT_X86" "-DLLVM_RUSTLLVM" "-DNDEBUG" "-o" "/fastsrc/rust_socketaddr/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/build/rustc_llvm-70ed50a745a8d2a7/out/llvm-wrapper/PassWrapper.o" "-c" "llvm-wrapper/PassWrapper.cpp"
  cargo:warning=In file included from llvm-wrapper/PassWrapper.cpp:6:
  cargo:warning=llvm-wrapper/LLVMWrapper.h:1:10: fatal error: llvm-c/BitReader.h: No such file or directory
  cargo:warning=    1 | #include "llvm-c/BitReader.h"
  cargo:warning=      |          ^~~~~~~~~~~~~~~~~~~~
  cargo:warning=compilation terminated.
  exit code: 1

  --- stderr


  error occurred: Command "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I/fastsrc/rust_socketaddr/build/x86_64-unknown-linux-gnu/llvm/build/include" "-std=c++14" "-fno-exceptions" "-fno-rtti" "-D_GNU_SOURCE" "-D__STDC_CONSTANT_MACROS" "-D__STDC_FORMAT_MACROS" "-D__STDC_LIMIT_MACROS" "-DLLVM_COMPONENT_AARCH64" "-DLLVM_COMPONENT_ARM" "-DLLVM_COMPONENT_ASMPARSER" "-DLLVM_COMPONENT_AVR" "-DLLVM_COMPONENT_BITREADER" "-DLLVM_COMPONENT_BITWRITER" "-DLLVM_COMPONENT_COVERAGE" "-DLLVM_COMPONENT_HEXAGON" "-DLLVM_COMPONENT_INSTRUMENTATION" "-DLLVM_COMPONENT_IPO" "-DLLVM_COMPONENT_LINKER" "-DLLVM_COMPONENT_LTO" "-DLLVM_COMPONENT_MIPS" "-DLLVM_COMPONENT_MSP430" "-DLLVM_COMPONENT_NVPTX" "-DLLVM_COMPONENT_POWERPC" "-DLLVM_COMPONENT_RISCV" "-DLLVM_COMPONENT_SPARC" "-DLLVM_COMPONENT_SYSTEMZ" "-DLLVM_COMPONENT_WEBASSEMBLY" "-DLLVM_COMPONENT_X86" "-DLLVM_RUSTLLVM" "-DNDEBUG" "-o" "/fastsrc/rust_socketaddr/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/build/rustc_llvm-70ed50a745a8d2a7/out/llvm-wrapper/PassWrapper.o" "-c" "llvm-wrapper/PassWrapper.cpp" with args "c++" did not execute successfully (status code exit code: 1).


warning: build failed, waiting for other jobs to finish...
error: build failed
command did not execute successfully: "/fastsrc/rust_socketaddr/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "8" "--release" "--features" " llvm max_level_info" "--manifest-path" "/fastsrc/rust_socketaddr/compiler/rustc/Cargo.toml" "--message-format" "json-render-diagnostics"
expected success, got: exit code: 101
failed to run: /fastsrc/rust_socketaddr/build/bootstrap/debug/bootstrap build --stage 2
2

You may want to use the cp -a flag instead of -r to preserve timestamps and other file properties.

You may also want to consider using a git worktree. The docs say there are issues with submodules, but I think people have successfully used them.

I suspect there might be some absolute paths hiding somewhere in the LLVM build. We generally try to make it so that you can safely move the target directories around, but LLVM is a big and complex beast. I would recommend just deleting the build directory and starting over. If you are on Linux, you can try the download-ci-llvm option to make that faster.

If you don't actually need two source checkouts, you can just copy the stageX directory out of the build directory. That should work, I'm not sure why you ran into an error with that.

1 Like
3

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.