How to collect code coverage for the rustc compiler itself?

compiler
1

I am trying to collect coverage information for the rustc compiler itself. Here's what I've done so far:

  • Modified config.toml with settings like assertions = true, profiler = true, and llvm-config paths.
  • Set the environment variable: export RUSTFLAGS="-C instrument-coverage".
  • Ran ./x build but encountered errors including:
    • profiler_builtins crate not compatible with #![no_core]
    • Symbol mangling warnings.

Has anyone successfully gathered coverage for rustc? Any recommended steps or configurations? The same question has been posted on Stack Overflow here: https://stackoverflow.com/questions/79112200/how-to-collect-code-coverage-for-the-rustc-compiler-itself

1 Like
2

You can try to use this fork for collecting rustc's coverage:

But I believe we definitely should have an official coverage collection patch for rustc.

3

I’ve been attempting to measure code coverage in rustc using a methodology inspired by RustSmith. Below are the steps I’ve followed, along with the issue I’m encountering:

  1. I modified the Rust source (commit b8bb296), specifically in src/bootstrap/src/core/builder/cargo.rs at line 551, by adding the following flags:

    if mode != Mode::Std {
    rustflags.arg("-Cinstrument-coverage");
    rustflags.arg("-Zprofile");
}

  2. This allowed me to successfully run ./x build && ./x install.

  3. I deleted all .profraw files before testing.

  4. I ran the rustc binary from the installation directory, which successfully generated a .profraw file.

  5. I used the following command to merge the profile data:

    ./build/x86_64-unknown-linux-gnu/ci-llvm/bin/llvm-profdata merge *.profraw -o info.profdata

  6. Then, I used llvm-cov to generate a coverage report:

    ./build/x86_64-unknown-linux-gnu/ci-llvm/bin/llvm-cov report /path/to/install/rustc -instr-profile=info.profdata

However, the generated coverage report only includes 36 lines corresponding to the main function in rustc, which is clearly incorrect. Even when I use rustc to compile a simple .rs file, the report still only shows 36 lines, with 100% coverage.

Am I using llvm-cov incorrectly?

4

Zprofile is apparently broken (and might get removed): Remove support for `-Zprofile` (gcov-style coverage instrumentation) by Zalathar · Pull Request #131829 · rust-lang/rust · GitHub

(I don't know how to get coverage to work though)

5

Thank you, everyone. The issue has been resolved, and the detailed steps are in the previous Stack Overflow link. It would be great if an official method for coverage collection could be provided in the future, similar to GCC, where enabling a flag would be sufficient without needing to modify the code.