Idea: mandate N independent uses before stabilizing a feature

[vorner]

Hello

I was pondering about an idea. Currently, the RFC is merged, then it is implemented, bakes there with a feature flag for a while and then gets stabilized (unless some problem is found on the road). While trying it out is recommended, it is not mandated and the feature could be stabilized without anyone actually using it, at least in theory. To forestall questions, I do have trust in the core team members, they do a great job but they are still humans and are prone to psychological effects/biases like WYSIATI.

So the idea is, before stabilizing a feature, would it make sense to mandate at least two (for some value of two) people posting a „I’ve tried it, it greatly improved my code/allowed me to do something I couldn’t before, it’s been great and here’s the code/commit: “ to the tracking issue. No more than one would count from the RFC authors. It would actually require a publicly visible code, so others could see how it looks like.

This would be a variation of the original RFC’s requirement for two independent implementation before the RFC moves from experimental to draft.

What I think this would bring:

• Raising the bar for features nobody cares enough to actually put the feature flag into their code to even try out. That would signify the feature is not really needed even if it looks nice, so maybe it’s not worth having it. The really important ones should not be hindered by this, I’d expect these would get a lot of such posts/links. I know a lot of unneeded features/ideas are shot down early, but if one slips through, it would be good to have a mechanism to catch it too.
• A place with an overview of the feedback would be easily available. I guess someone (the author?) tracks feedback from various places before it gets stabilized, but if I wanted to see what people thought about feature X before it got stabilized or for what I could actually use it, I’d have hard time finding it. Having it all there at one place helps to get more fact-driven decisions when deciding about stabilizing it too, instead of relying on impression from various forums, IRCs, gitters over the time.
• It would invite experimenting and sharing the findings. If I knew the feature needed these two sign-off comments, I’d be more inclined to actually take the time to use it in my code and to send the comment ‒ even if it turned out to be „I’ve tried it, and, well, no real improvement came out of it, so I removed it“.

I do see some downsides too, though:

• It makes the RFC more formal and heavy-weight.
• How to advertise the need of (new) „I’ve tried it“ comments?

What do you think of it? Does it make some sense (in this or some modified form)?

[scottmcm]

it’s been great and here’s the code/commit

One catch-22 we’ve hit as more and more projects stick to stable is that they won’t switch back to nightly for any feature, no matter how much they want it. This makes some things really hard to get experience with – particularly lints, like fn_must_use, where code that already exists and is tested and CRed well probably won’t hit it, and if it prevents a bug from being checked in by having it on, there’s no commit to reference, since the bad code was never checked in in the first place.

So I think I’m sympathetic to the intent, but worried about hard rules.

[vorner]

One catch-22 we’ve hit as more and more projects stick to stable is that they won’t switch back to nightly for any feature, no matter how much they want it.

I see, but I think this represents a deeper problem. Then the feature thing and nightly don’t really play the role it should.

Maybe if it was made clear someone has to experiment with it and show the code, not necessarily in the master branch, people would step up. Like, if a project would benefit from hte feature (or they believe they would), and they had the motivation, they could have a branch where they try it, show it off, but wait for merging before the feature stabilizes. Or maybe I’m naïve there .

As for the features that prevent bugs, maybe not being a commit but code snippet like „I had this bug and it identified it: <here comes the code>“ would work equally well.

I guess having it as a hard rule could turn out to be problematic. Any idea how to make it a soft rule or experiment with the idea (maybe just advertising the intent of gathering such kind of feedback?) would work, but I’m not sure how motivated the people would be if it wasn’t really needed. If it’s „You won’t get your feature unless you do something for it yourself“, then the motivation is a bit higher.

[pietroalbini]

There was a discussion about this in the loop_break_value tracking issue.

Edit: and another discussion related to this in internals.

[vorner]

These two talk about having #![feature(...)] in stable. While having to use nightly might be one inhibiting factor, I don’t feel that one much myself. I regularly develop on nightly, while having CI also on stable. If there’s a nice feature to play with, I don’t really mind adding a temporary #![feature(...)] in a branch.

What inhibits me is:

• There are too many things going on at once. I’m not paid core Rust developer (and if I was, it would kind of disqualify me from being the right person to experiment with it), so I have very limited amount of time to play with things. Picking the important ones just before they stabilize would require me to actively monitor a lot of tracking issues.
• While I was told the tracking issue is a good place to put feedback into, chiming in without anything very significant feels somewhat out of place. Nobody really does that.

Thinking about what I probably wanted in the proposal, it’s two-fold:

• Make sure there’s a motivation (or opportunity) people try the features out before they stabilize.
• Make it more explicit that the amount (not only that it’s overall positive) of feedback is also important for a feature to get stabilized, for two reasons: lack of feedback may mean nobody cares, and lack of feedback means the statistical value of the little feedback is kind of random.

Throwing some more brainstorming ideas to the table:

• Introduce an explicit „Final experimentation period“. Basically, as there’s final comment period before an RFC moves to accepted, this would be the same for moving to stable ‒ as there’s a check „Is this what we think we want?“, this would be a check „Did it turn out as we expected?“. As there’s list of RFCs in final comment period in This week in Rust (that prompts me sometimes to have a look at them and add a thumbs-up or something), there could be a list of tracking issues about to stabilize. Furthermore, if I experiment with it close to getting stabilized, I may keep the code in a separate branch, but there’s a high chance it’ll apply once it really stabilizes. If I play with the feature too early, the feature might change, my master branch might change, so the chance of the experiment being useful to me is lower.
• Have an ongoing poll/place to put the feedback into without spamming the tracking issues. For each open tracking issue I could put links to my code, or vote „I really need this, yesterday was too late for stabilization for me“, „Looks nice, but not really that important“, „I don’t care at all“, „Looks actively harmful“. I could change my mind later on (it could remember my votes) and if at least N people voted about a specific feature, it could show current statistics and trends.

[Ixrec]

One catch-22 we’ve hit as more and more projects stick to stable is that they won’t switch back to nightly for any feature, no matter how much they want it. This makes some things really hard to get experience with – particularly lints, like fn_must_use , where code that already exists and is tested and CRed well probably won’t hit it, and if it prevents a bug from being checked in by having it on, there’s no commit to reference, since the bad code was never checked in in the first place.

Possibly an insane idea, but over in Javascript-land the de facto and quasi-official solution to this problem is transpilation: anyone can use experimental flavors of Javascript “without fear” by adding a transpilation-to-“stable” step in their build/release process. I have no idea if there’s a realistic way to integrate this with cargo and crates.io (I guess we’d need to only transpile on publish, not on regular builds, or no one would be testing the error messages/compilation speeds/etc of the nightly feature…) but I assume there’s someone reading this thread who would know.

For things like try{} blocks, throw expressions, Ok-wrapping, auto-deref operators, and I have a hard time imagining any other way we’d ever get sufficient experience from large, important and stable projects like serde, diesel, etc to know if the feature is really working out in practice. Can features like that even be transpiled? Is it only practical in Javascript because Javascript has no static types, so a parser tells you all you need to know? Can a type-inference-aware transpiler be built atop compiler internals?

---

Introduce an explicit „Final experimentation period“. Basically, as there’s final comment period before an RFC moves to accepted, this would be the same for moving to stable

I do think announcing and advertising when a feature is ready for experimentation (i.e. fully implemented, more or less free of showstopping bugs, and no further changes planned) would be extremely valuable. But I don’t think there’s any need to impose a particular time limit on it, or call it “final” by analogy to FCP. At least, not until a feature is genuinely in danger of getting “unaccepted”.

In other words, I’d just like This Week in Rust to add a list of “nightly features we need feedback on” (presumably derived from some ready-for-experiment label we put on tracking issues). Whether any specific feature should wait X weeks for feedback, wait for N feedbacks, or be “unaccepted” if insufficient feedback appears is something that I’d rather argue on a case-by-case basis on each of the tracking issues. Especially since some features will probably want to go through multiple rounds of this.

P.S. I would like TWiR to also list tracking issues that are about to stabilize, but that’s sort of orthogonal to encouraging experimentation.

[MajorBreakfast]

I’d just like This Week in Rust to add a list of “nightly features we need feedback on”

Great idea! Maybe call the section “Mature nightly features”. That would suggest that the features work as intended and it’s time to play around with them.

[Tom-Phinney]

My primary concern here is that the proposal restricts the significant feedback to those projects which are publicly shareable. Many code authors who need, and thus try, a new feature during its development process may not be able to share their code, or not be willing to do so while that code is unfinished. I feel that anecdotal evidence from such developers is valuable, even when they are unable or unwilling to share real code that uses the feature.

[H2CO3]

downsides:

• It makes the RFC more formal and heavy-weight.

I wouldn’t count a somewhat higher barrier to entry as a downside

[vorner]

Ah, sorry if I sounded like having a too final proposal. That was more like an illustration of the direction in which I tried to think than actual wording, I assumed some like more concrete formulations would be finished later.

Anyway, while I would like to invite any kind of feedback, one with actual code that can be examined would indeed be more valuable.

[SheepKey]

Well, I don’t see how it “restricts” anything (assuming no hard rules about the feedback required). At least my understanding is that this proposal changes nothing for such devs.

Right now, proprietary devs who cannot share code can still post comments and discussions about their anecdotal experience. Nothing about this proposal says that they should no longer be able to do that.

However, the way I see it, this should be a proposal to actively encourage devs with open/public code to experiment more and share actual code. Seeing actual code examples would be really great for fleshing out experimental features. Devs who work on open-source projects should be able to share such feedback and this is a proposal for a way to actively encourage that.

It should change nothing for devs who work on secret code.

[scottmcm]

Possibly an insane idea, but over in Javascript-land the de facto and quasi-official solution to this problem is transpilation: anyone can use experimental flavors of Javascript “without fear” by adding a transpilation-to-“stable” step in their build/release process.

Well, Rust actually implements a bunch of things (like ?) as “lowering” internally, which is basically “transpiling”. The thing that’s awkward there is that those often need to call unstable things, like how ? needs to call https://doc.rust-lang.org/std/ops/trait.Try.html#tymethod.into_result. So making it work might take copying over traits and trait impls, which could potentially work, but not if those themselves use other language features that can’t be lowered to stable (like specialization).

[vorner]

It seems to me the idea to post features that want feedback (either at some important point in their life or before stabilization) is more or less generally considered a good idea. While there might be some more steps that could be taken, this one looks like a good first one of them.

The next question is then, how to make it happen? Should I write some kind of more official proposal or contact specific person?

[Centril]

The next question is then, how to make it happen? Should I write some kind of more official proposal or contact specific person?

You should write an RFC changing the process regarding stabilization. Depending on how broadly it applies, it would then be under the purview of the language team, the libs team, or maybe the core team. See the team structure for more info on the teams. See the README on the RFC repo for details on the process.

@scottmcm is on the language team, so you’ve essentially already contacted the language team by writing here

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.