This is just a quick strawman proposal about the unstable
feature. With the recent uptick in concern about the nightly channel
I’m feeling more urgency to knock down the unstable feature list.
When I look at that link the thing that stands out to me is the "test"
feature. This feature tends to be used to get access to either
Bencher. We’ve been reluctant to stabilize
these because they aren’t as good as they could be, prefering
to hold off for a more extensive custom test frameworks
In a crude grep through crates.io I see:
- 645 crates using block_box or Bencher
- 752 using
extern crate test
The discrepancy between the two is curious, possibly because I’m missing e.g. glob-imported black boxes, possibly because of other minor features people are using in the test crate.
There are only a few crates I see using
test::TestDesc (and thus
constructing their own test suites dynamically): the out-of-tree
compiletest and syntax, and (interestingly)
url. I see a few using
test::stats (I don’t even know what this is…), but that seems like
it can trivially be moved out of tree.
I suggest we stabilize
black_box as-is or nearly as-is.
I think this reasonable because:
- Even in unstable form, these features are low maintenance and likely to stay around forever as-is, they are so widely used. Stabilizing them, while distasteful, has relatively little maintenance impact.
- A stable, working implementation of custom test frameworks that can support benchmarking is far away. We don’t even have a design.
The primary reason for doing so is to quickly eliminate a major source of nightly dependence, not because we love the feature.
So here’s specifically what I suggest:
std::test, leaving unstable reexports.
- Actively migrate all existing crates from
That’s it. We can do it real fast. And I think it’s important that we actively do the ecosystem migration, not just wait for authors to get around to it. Let’s move the numbers.
Someday we can deprecate these if the tides change.
Why put these in
std::test instead of stabilizing
extern crate test? This is a tricky question. Today the test crate is an
implementation detail. No part of the hypothetical Rust spec requires
it. So it makes sense to hold that line and just move the few tiny
features people need into
std. That said, I am not confident the
test crate will always be an implementation detail, and I suspect in
the fullness of time it will go from de-facto stable to specced.
In particular, last I recall, the std-aware cargo RFC requires
cargo to know the test crate exists, though it does not require
the test crate to be importable by user crates.
The wrinkles to consider here are the impact on custom test frameworks and on no-std testing (which so far has recieved little thought). Neither of these implications I have thought through much, but I suspect stabilizing a few benchmarking types does not cause significantly worse problems for either effort.
Let me know what ya think.