Thank you for pushing on this front Brian. In the external test harnesses thread I proposed an alternative:
I’d like to (first introduce as unstable and then) stabilize building blocks for external test harnesses:
- A simplified version of
TestDescAndFn, with just enough to describe whatrustc --testgenerates. (So probably without dynamic tests.)- Some way to override which crate/function is used instead of
test::test_main_static. Maybe#[test_harness] extern crate fancy_test;? (If benchmark are supported in this mechanism, TestDescAndFn would probably have to be generic over Bencher.)
Just to clarify:
There are only a few crates I see using
test::TestDesc(and thus constructing their own test suites dynamically): the out-of-tree compiletest and syntax, and (interestingly)url.
The url, idna, and html5ever crates each use crates.io: Rust Package Registry which is a copy of Rust’s src/libtest modified just enough to run on stable Rust. (The package name for Cargo and crates.io is rustc-test, but the library name for rustc and extern crate is test.) They do so to dynamically generate many tests that share the same code but take different input data. But rustc-tests can also be used with #[test] and #[bench] to run benchmarks on stable Rust today.