# Pre-RFC: Dynamic tests

**URL:** <https://internals.rust-lang.org/t/pre-rfc-dynamic-tests/6790>\
**Category:** language design\
**Created:** [February 20, 2018, 8:59am UTC](https://internals.rust-lang.org/t/pre-rfc-dynamic-tests/6790 "2018-02-20T08:59:52Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ishitatsuyuki](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/ishitatsuyuki/32/2163_2.png) [@ishitatsuyuki](https://internals.rust-lang.org/u/ishitatsuyuki)\
**Post date:** [February 20, 2018, 8:59am UTC](https://internals.rust-lang.org/t/pre-rfc-dynamic-tests/6790/1 "2018-02-20T08:59:52Z")

</div>

# Summary

Add ability to run generated test closures with `#[test]`.

# Motivation

We may want to test the same function on multiple test data, while still getting colored result output and automatic parallelism.

An example of this is compiletest inside rustc. It achieves the goal by [passing the test Vec on it’s own](https://manishearth.github.io/rust-internals-docs/test/fn.run_tests_console.html). While it works well, we need to rewrite the test harness completely and use our own `main()` for this.

# Guide-level explanation

To use dynamic tests, a `#[test(dynamic)]` attribute should be used, and the fn’s signature needs to be altered:

```rust
#![feature(conservative_impl_trait)]
#![feature(test)]
extern crate test;
use test::TestDesc;

#[test(dynamic)]
fn generate_tests() -> impl Iterator<Item=(TestDesc, Box<FnBox<()>>)> {
    // ... generate tests ...
}

```

`#[bench(dynamic)]` is also available similarly.

# Reference-level explanation

An iterator is used so we can lazy-generate the test cases in future if we refactor. Currently, [the test harness](https://manishearth.github.io/rust-internals-docs/test/fn.run_tests_console.html) requires all testcases to be put in a Vec.

# Alternatives, unresolved questions

- To make this available on stable, there are two obstacles:

- Is using impl Trait a good idea?

- Should we use TestDescAndFn? This also implies that we can generate all kind of tests (including benches) using the syntax.

---

<div class="post-metadata">

**Author:** ![oli-obk](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/oli-obk/32/3168_2.png) [@oli-obk](https://internals.rust-lang.org/u/oli-obk)\
**Post date:** [February 20, 2018, 12:37pm UTC](https://internals.rust-lang.org/t/pre-rfc-dynamic-tests/6790/2 "2018-02-20T12:37:39Z")

</div>

I think you’ll be able to implement that as part of

> <https://github.com/rust-lang/rfcs/pull/2318>

---

<div class="post-metadata">

**Author:** ![RReverser](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/rreverser/32/2189_2.png) [@RReverser](https://internals.rust-lang.org/u/RReverser)\
**Post date:** [February 22, 2018, 11:04am UTC](https://internals.rust-lang.org/t/pre-rfc-dynamic-tests/6790/3 "2018-02-22T11:04:10Z")

</div>

For what it’s worth, this is already possible using Servo’s fork of libtest as a workaround to run dynamically generated tests even on stable [https://crates.io/crates/rustc-test](https://crates.io/crates/rustc-test). I’m successfully using it on several projects myself. (Alternatively, if you want to run tests only on nightly, you can use libtest directly, but its API is rather unstable).

---

<div class="post-metadata">

**Author:** ![ishitatsuyuki](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/ishitatsuyuki/32/2163_2.png) [@ishitatsuyuki](https://internals.rust-lang.org/u/ishitatsuyuki)\
**Post date:** [February 22, 2018, 11:50am UTC](https://internals.rust-lang.org/t/pre-rfc-dynamic-tests/6790/4 "2018-02-22T11:50:49Z")

</div>

This however doesn’t resolve the complexity of replacing the test harness with a custom entrypoint, which is one goal I like to achieve with this RFC.

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/system/32/14092_2.png) [@system](https://internals.rust-lang.org/u/system)\
**Post date:** [March 25, 2019, 8:29am UTC](https://internals.rust-lang.org/t/pre-rfc-dynamic-tests/6790/5 "2019-03-25T08:29:40Z")

</div>

Closed on March 25, 2019, 8:29am UTC

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.
