Sort test output according order in the source file

Mingun · October 27, 2020, 4:04am

When developing new feature or creating tests for upcoming bugfix, I usually try to create all possible tests, most of them will fail at that moment. Then I go through test output and fix that tests. In order to keep things in mind, I fix tests in that order as it appears in the source file, but, unfortunally, test output not follow that order. It would be useful to sort output of the test command, or always or with flag

bjorn3 · October 27, 2020, 12:28pm

There is no option for this. The tests are always alphabetically sorted:

github.com

rust-lang/rust/blob/20b1e05a8d0e1773dc840a3286aa37916e87d84b/library/test/src/lib.rs#L388


            filtered.iter_mut().for_each(|test| test.desc.ignore = false);
        }
        RunIgnored::Only => {
            filtered.retain(|test| test.desc.ignore);
            filtered.iter_mut().for_each(|test| test.desc.ignore = false);
        }
        RunIgnored::No => {}
    }

    // Sort the tests alphabetically
    filtered.sort_by(|t1, t2| t1.desc.name.as_slice().cmp(t2.desc.name.as_slice()));

    filtered
}

pub fn convert_benchmarks_to_tests(tests: Vec<TestDescAndFn>) -> Vec<TestDescAndFn> {
    // convert benchmarks to tests, if we're not benchmarking them
    tests
        .into_iter()
        .map(|x| {
            let testfn = match x.testfn {

It shouldn't be too hard to add an option to disable this sort though.

bjorn3 · October 27, 2020, 12:37pm

This has been done since

It was likely done in part as before that commit the order of the printed failures was dependend on the order in which the tests actually failed, which is non-deterministic in presence of multithreading. The current code doesn't have this problem anymore.

Mingun · October 27, 2020, 3:50pm

Well, while for CI reports, alphabetical output may be more convenient, when analyzing errors, it is desirable to view the tests from top to bottom and follow the test log in the same order, i.e. sort the output according to the test positions in the file.

system · January 25, 2021, 3:51pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Path for stabilizing libtest's json output? tools and infrastructure	7	547	January 20, 2024
[Pre-RFC] Filter out unit tests BEFORE compiling the crate for faster edit-compile-test loops ideas (deprecated)	3	1305	March 25, 2019
Cargo test and backtraces tools and infrastructure	19	11141	March 25, 2019
Rust 1.61.0 prerelease testing announcements	7	1605	August 16, 2022
Indent/pretty print compilation steps as per dependency graph? cargo	15	1879	November 13, 2019

Sort test output according order in the source file

Related Topics