Adding tests invoking external binaries

I am looking into adding tests invoking external binaries. This includes positive tests and negative tests. Does the framework already support this? Is there any documentation of how to add a script to do that?

Can you just invoke the command from a Rust style #[test] fn?

Is this question somehow specific to developing Rust itself (e.g., rustc)? If not then https://users.rust-lang.org/ is probably a better fit for this question.

If this question is specific to developing Rust, then it would be helpful for you to include more details (like which project (rustc?) and which tests you're working on).

3 Likes

yes it is. I modified a part of the rustc and I would like to combine external binaries to test my changes within the compiler's test framework. In particular I would like to be able to run the tests using

./x.py test src/test/xxx

to invoke my tests.

Oooh you're talking about compiler tests!

The compiletest framework is documented. Probably run-make would be useful for this case.

how does one run-make tests? I did

./x.py test src/test/run-make --stage 1 --no-doc

and it gives the following error messages:

Check compiletest suite=run-make mode=run-make (x86_64-apple-darwin -> x86_64-apple-darwin)
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 20, kind: Other, message: "Not a directory" }', src/tools/compiletest/src/
header.rs:38:20
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace