Packaged tests and benchmarks. Do we need them?

[newpavlov]

Currently cargo includes tests and benches folders into a packaged crate. Do you know about any use-cases for keeping them or are they just a package dead-weight?

The question can be re-formulated as: should packaged tests work or can they fail? For example if tests use relatively heavy resources, it seems unreasonable to include them into package, but without them tests will fail.

[mbrubeck]

cargo test -p foo and cargo bench -p foo are not guaranteed to work for packages outside the current workspace, because Cargo won’t build any dev-dependencies of those packages. So in general, users should not rely on tests/benchmarks/examples in their packaged dependencies.

On the other hand, running packaged tests in automation tools like Crater can be useful for discovering regressions in published packages.

[newpavlov]

Ah, yes, I forgot about Crater.

[CAD97]

UNIC has been following a hybrid approach. We blacklist tests that rely on heavy UCD tables (which aren’t even packaged since they’re elsewhere in the workspace) in the original format but include the rest of them.

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.