Pre-pre-RFC: Automatically generate bug-reports


#1

It occurs to me that most compilers and toolchains have fairly rigorous guidelines regarding what information should be submitted with bug reports, but that this is a relatively unexplored area for built-in automation.

More specifically, the following seems to be a fairly common workflow:

  1. While working on a large project, notice something odd that seems like a bug.
  2. Try to reproduce, either in a small dummy project or in an online “playground”. This may involve copying and pasting a block of code from the original project, then editing it to remove dependencies, project-specific information (e.g. in comments and variable names), and so on.
    1. In some cases, the issue may not be reproducible without a specific set of flags passed to the compiler by the build system, so the user may need to dig through log files (or similar) to determine the arguments of the command that failed.
  3. Iterate on the dummy project until (hopefully) a minimal test case is produced.
  4. Look online for how to report bugs for the toolchain.
  5. Find a template for bug reports and fill in various fields by copying-and-pasting by rote (e.g. for rustc, running rustc --version --verbose and copying the output). Copy the code from the minimal test case into the issue.
  6. Write up a description of the expected/actual behavior, etc.
  7. Submit the template to the toolchain’s error tracking system.

It seems to me that all steps except 1, 3, and 6 could benefit from some sort of automation:

  • If some UI for selecting the offending code block were provided, the build tool could automatically generate a dummy project with the code block as a starting point; alternatively, it could submit the code to an online compiler site (in Rust’s case, the Playground) and open the link in the default browser.
    • If the issue generates a compiler error, the compiler would have a file name and line numbers to use as a basis for which block of code to use.
    • Variable names could be automatically anonymized and comments could be stripped. With some extra smarts, “dummy” versions of the classes involved could be generated.
  • The build system could easily generate a new build file that would use the same compiler arguments as the command that caused the error.
  • A bug report template could be downloaded automatically.
  • Some fields of the template could be filled in automatically.
  • The template could be submitted automatically.

So: should we add some set of Cargo subcommands that would perform some or all of these automated steps?


#2

As an alternative or something to do in addition, the Rust Playground could add a “submit compiler issue” option. This would probably be simpler than the cargo approach, since steps 1-3 are skipped.