Fortifying the process against feature bloat

[Centril]

Therefore, if something is written in prominent place in the RFC itself, it will be seen as stronger argument than something deep down, lost in hundreds of comments, or the thing in the comments really needs to be very impressive to balance that.

Here is what it takes for a language feature to go from a Pre-RFC to stable Rust (in my experience):

1. The Pre-RFC is filed (sometimes it goes to RFC directly without a Pre-RFC).
2. A bunch of discussion happens; and the proposal probably doesn’t make it past this point.
3. The RFC is filed
4. A bunch of discussion happens
5. At the earliest lang team meeting where we have time over for the RFC it is discussed and a shepherd / owner is assigned
6. A bunch of discussion happens
7. The shepherd / owner will read the RFC and more or less all comments.
8. The shepherd will eventually nominate the proposal when it has made sufficient progress (enough discussion) and it is discussed on the next lang team meeting. A lot of proposals get closed or postponed here.
9. PFCP merge
10. All but 2 team members must OK the proposal actively.
11. Some lang team member(s) might file concern(s) here after the meeting. It happens quite frequently.
12. If no concerns were filed, and all but 2 members OKed the proposal it goes to FCP.
13. It stays in FCP for 10 days. Any concern filed takes it immediately into PFCP again.
14. The RFC is merged and a tracking issue is filed.
15. The RFC is implemented and all unresolved questions get resolved. Changes do happen here.
16. Tests and documentation must be added.
17. It must bake for for some time in nightly
18. Someone must actively propose the feature to be stabilized. When this happens, the PFCP -> FCP procedure happens again for stabilization.

I might be overstating how lengthy this process is, but it seems to me that it is full of opportunities for important comments and objections to be raised.

But can you honestly say there was no „Huh, why not?“ RFC? An if not yet, that there will never be?

It is hard to think about this without specifics. Mistakes can be made, but our process is quite good at ensuring it happens extremely rarely.

EDIT: To clarify, the interesting question is if changes to the process would actually make mistakes more unlikely – we can tweak here and there, but I think the process is working well.

I saw the original idea in this direction (negative RFC) as being able to prevent feature bloat ‒ if once decided something is bloat, it would never get in.

A closed RFC means that the same idea shouldn’t be filed again. It’s up to the team to judge what “same” constitutes (how similar does it have to be…), and you’d have the same problem with negative RFCs in any case… A team would have to judge how similar a proposal is to the banned idea.

[scottmcm]

A closed RFC means that the same idea shouldn’t be filed again. It’s up to the team to judge what “same” constitutes

To elaborate on this, I think it could be, among other things,

• New motivation is found for the same feature
• An alternative solution is found for the same problem
• Over a substantial time period, the issue it addresses has become serious (probably at least a year, increasing as we get further from 1.0)

[vorner]

I might be overstating how lengthy this process is, but it seems to me that it is full of opportunities for important comments and objections to be raised.

To be clear, I fully agree with this. What I worry is point 8 and 10. If I imagine I’m in the position to decide: I’m looking at the RFC and deciding if I should OK it or not. I read the proposal. Then go through the comments. And I have to make a decision. Sometimes, it is clear-cut. But sometimes it feels like I’m not sure ‒ there were many comments, emotions, many people who want the proposal and many that don’t. How do I decide? I don’t know if you people in the teams have some way to handle this, but for me, my gut feeling would have a lot of say in the decision ‒ and my gut decides on quite many things, not all of them really relevant to the question ‒ like, what argument feels stronger, which is not always which one is better.

I’m trying to suggest to include things in the main RFC text (because it’s the part that influences the decision most) to include most of the relevant information ‒ to guide the subconsciousness to decide more on facts and less on feelings and random influences. Because, no matter the length of the process, this is the point where the actual decision is made.

It is hard to think about this without specifics.

With the risk someone will get too attached to specifics, the one I had in mind was 1925. The lesser problem is I don’t like it. The fact there were many thumbs-down is some indication about people’s feelings about it, but that’s also not my main concern. What I find most disturbing about it is (It’s always easy to criticize in hindsight, so I apologise for doing just that ‒ but I want to provide the specifics asked for):

• The RFC is missing several sections. Why wouldn’t they apply? And why nobody (if I scanned the comments correctly) even mentioned it and asked if there really are no downsides to that?
• The RFC was merged in confusion about which version people commented on, because there were two very different versions during the discussion.
• There was no strong argument for it. Making it easier for macros was mentioned in the comments, but nobody actually brought an example of a macro that showcases that. The RFC wasn’t updated to reflect that.
• The thumbs-down count was dismissed as unimportant, on the bases that they were probably for the leading+trailing version and without further investigation.
• It was stabilized without seeking further feedback from community, even though that was mentioned after merging it and even though there was the confusion.

I wouldn’t have counted it as an error in the process if the RFC was better written, or if ‒ upon the confusion ‒ the discussion was restarted (eg. the merge request closed and a new one started) to gather new thumbs-up/thumbs-down counts instead of just dismissing them (I know this is only one of the decision inputs, but still) and if it was clear how the decision was made. In that case I would still not like the feature, but would be content knowing that it was reached in a fair way and pros and cons properly weighted, so I’m just the one being weird. But looking at so many „oddities“ around it, it feels somewhat unfortunate, and some reactions from community suggest I’m not the only one. In other words, I believe in this case, the decisions were influenced more by some random noise than actual facts, because most of the discussion has a lot of interference. And I’m still not sure what problem was this meant to solve.

Now, I don’t want to overblow this one, because the feature itself is no big deal. But it shows there can be errors in the process. And I think it also somewhat undermines the trust in the process, which is unfortunate. I could probably come up with few more examples of something that felt like the consensus wasn’t really clear and strong, but I think it’d only be twisting knifes in wounds and I don’t want to do that.

To clarify, the interesting question is if changes to the process would actually make mistakes more unlikely – we can tweak here and there, but I think the process is working well.

I don’t really want to change the process itself ‒ as you say, it is working well most of the time. I just want it fed with better input. Currently I believe the best way would be to compose some micro-guide (or a checklist) for RFC authors. As I argue above, I think it should help (somewhat, it wouldn’t be a silver bullet, of course), but I guess nobody knows without trying. So in that sense, is there any risk in trying to compose such a short guide? I can give a shot at first version, but I’ve only read RFCs, not written them (I have the editor open right now, with the RFC for announcing things about to stabilize in TWIR, but that’s a tiny one and I have no idea yet how that’ll go ‒ but if there was a guide to help me with that, I’d definitely go and read it) ‒ so to have any validity, it’d need some help from others, who do write RFCs.

So, if I start writing it, are there people who would like to help? Or, is there some reason why I shouldn’t do that?

A closed RFC means that the same idea shouldn’t be filed again

That’s good to know, with that knowledge, the process does look more balanced. Are these things documented somewhere (eg. how the teams reach their decisions)?

[sfackler]

The thumbs-down count was dismissed as unimportant, on the bases that they were probably for the leading+trailing version and without further investigation.

Thumbs up/thumbs down counts are inherently unimportant. If someone want to have an impact on an RFC, they should read it and thoughtfully comment on it. A thumbs down provides no information other than someone clicked a button for some reason at some stage of an RFC. The Github emoji reaction feature didn’t exist for the first several years of the RFC process, and we weren’t stumbling around blindly during that time period.

It was stabilized without seeking further feedback from community, even though that was mentioned after merging it and even though there was the confusion.

This is not some secret process. Someone posted on the tracking issue in the rust-lang/rust repository proposing the feature be stabilized. This triggers a minimally week long process in a public forum where people who care about the issue can comment on what’s going on. In what way is this not seeking feedback from the community? Should we be emailing every single Rust user before stabilizing a feature or something?

[Centril]

If I imagine I’m in the position to decide: I’m looking at the RFC and deciding if I should OK it or not.

Oh; nomination doesn’t mean that you have to agree with the proposal; when I nominate a proposal it usually means that the discussion has slowed down, most discussion points have been made and the pros and cons are relatively clear. A nomination often results in postponing or closing an RFC. It just means that “the time is right” to discuss the RFC formally in the lang team meeting. I might raise some concerns, etc. but most lang team members will have also read the RFC as well.

I’m trying to suggest to include things in the main RFC text (because it’s the part that influences the decision most) to include most of the relevant information

I agree with this and I try to / usually do it with my own RFCs when I have the time, but it is rather time consuming (lots of busy work…) to add summaries of arguments in the RFC especially if it gets a lot of comments very quickly. Often, at least for my own RFCs, some arguments that are made in the RFC thread have already been discussed in the RFC itself, so the work is already done.

With the risk someone will get too attached to specifics, the one I had in mind was 1925.

Honestly, RFC 1925 is such a nano-RFC that I kinda don’t know what the fuss is about Personally I found @withoutboats’s arguments wrt. flexible grammar being part of Rust’s philosophy to be convincing. If there have been mistakes in the language design, I don’t think this one is it. I haven’t read that RFC in depth tho and I’ll take a deeper look wrt. the text itself (not the feature…).

to gather new thumbs-up/thumbs-down counts instead of just dismissing them (I know this is only one of the decision inputs, but still)

I think it is important to understand that we usually don’t discuss the GitHub reactions at all on team meetings, so they don’t factor into the language teams decision making process. If it is discussed on a meeting, it is merely to try to maintain community cohesion and good will. I.e: the team will feel uneasy about the number of negative reactions. It is barely a decision input.

I also think that RFC 1925 is used as an example too much; it’s importance is way overblown in my opinion.

but I’ve only read RFCs, not written them

Try it out, it is a nice experience

So, if I start writing it, are there people who would like to help? Or, is there some reason why I shouldn’t do that? Not sure I have the time atm tho. myself.

If you are up to the task, go for it. Maybe we’ll make some changes here and there.

[withoutboats]

the one I had in mind was 1925.

The response that this RFC has received from certain members of the community is really baffling to me, and seems to stem from ignorance about our policies on grammar. In the past we’ve merged changes to the grammar that in my opinion are at least as “bad” on the basis of the arguments against that RFC without even an RFC at all. In fact, I think our policy is that that kind of change simply doesn’t need an RFC, and the RFC was overkill.

Here are two prominent examples:

• Trailing +. T: 'static + is a grammatically correct bound.
• Turbofish in types: impl<T> Option::<T> is a grammatically correct impl header.

Here is a valid way to write the main function signature in Rust:

fn main<>() where for<'a> (): 'a +, { }

Neither of these are accidents of any kind; they were both conscious changes made within the last 12 months, without an RFC process. And they all stem from the same basic underlying language policy: Rust has a very flexible grammar, trailing or leading separator and unnecessary characters are supposed to be permitted liberally, and extensions to its flexibility are considered bugfixes (for example, fn main::<>() doesn’t parse, but probably should).

This policy was made with full awareness of the downsides (e.g. there are multiple, redundant syntaxes that have the same meaning), because we weighed the upsides as greater (especially the benefits to macros, but also often there is an underlying grammatical unification that a redundancy enables). Reiterating the downsides as applied to a particular case is not a compelling argument to make an exception to longstanding policy. There would have to be a very compelling counterargument which is specific to this case.

Its our ability to overrule the “votes” of the community (which have never been an actual part of the RFC process) that helps protect the longevity of the language, because we do better at pursuing a consistent vision than a shifting, temporary body made up of the people who happen to read a github issue would be able to. This RFC is, in my opinion, an excellent example of how our existing practices avoid the kind of negative outcomes you are worried about.

[vorner]

I agree this one received more than its share of beating, so I was reluctant to bring it up. But now I’m glad I did, because after what you say here it now makes much more sense (even the somewhat visible displeasure of the team members whenever this one is brought up). Each „side“ operated under very different understanding.

I mean, I didn’t know about this policy. The motivation in the RFC is somewhat inadequate to warrant any feature, even a small one. If I read just that, it makes me sad why anyone would add to the language just because of that, so I have a tendency to express my dissatisfaction (which, I guess, went a bit viral in the community). But if it wrote „Make the grammar more flexible in regards to | in patterns and make it more consistent with other separators which are allowed even at places where not necessary, in part to make life of macros easier“, it would give completely different impression. If one of the first comments at the merge request said „Oh, we usually allow these little details even without RFC, according to our policy [link]“, that would also look different. But reading just the information available there, it gives the impression of very „Oh, why not“ feature.

There were several interesting revelations in this thread regarding the working of the teams that explain a lot, but are not AFAIK commonly known ‒ at least I didn’t know them and I try to follow what happens around the language ‒ I regularly look at this forum, I have the subreddit in my RFC reader, I hang out in several rust gitter channels…. Therefore, I have to ask: are these things documented somewhere? Is there a checklist what to look into when you’re evaluating RFC? The policies like the one you just described? Would it make sense to make them more visible, so people don’t react out of ignorance? Are the meetings recorded somewhere (I’ve found rust-lang/minutes, but these seem to end about a year ago), if one is interested in following the discussions? I don’t want to say you try to operate behind closed doors, but sometimes it needs a thread like this to understand what is actually happening.

The thumbs-down thing: I understand these are not an input to either pass or sink a proposal and I don’t even think of different output of the team as „overruling“ ‒ there’s nothing to overrule. But I think it is a valuable mood-meter that can indicate some general unhappiness about something. It probably should be an input to investigate into what the something might be. The discussion raised the number of them, but found an excuse for them. In the light of „this doesn’t really need a full RFC anyway“, it makes sense to dismiss them, but it still gives impression you don’t care what the community thinks and that it feels unhappy. I guess more investigation into dissatisfaction would happen on a „full blown RFC“, but again, it wasn’t clear there’s actual reason to consider the RFC not being „full“ in the eyes of casual reader. Small, yes, but that would still deserve it’s due process.

If I try to restate the motivation and the missing sections in the RFC (without changing any of the technical parts) so it better explains the reasons and hopefully makes a better impression even without knowing all this background information and send a PR, will it have to go through a full-blown process, wasting your times, or can it be counted like a small fix that doesn’t change anything and could be merged in some lightweight manner? To avoid further discontent of people who read it ‒ after all, RFCs don’t act only as decision process, but as kind of a documentation.

[steven099]

Honestly, RFC 1925 is such a nano -RFC that I kinda don’t know what the fuss is about Personally I found @withoutboats’s arguments wrt. flexible grammar being part of Rust’s philosophy to be convincing. If there have been mistakes in the language design, I don’t think this one is it. I haven’t read that RFC in depth tho and I’ll take a deeper look wrt. the text itself (not the feature…).

I think one oddity of that proposal is that it opts to have a leading separator instead of a trailing one as would be more common in Rust. Both are equivalent as far as code generation is concerned, and there’s no parsing ambiguity being resolved by having it one way vs. the other. The argument in favour of a leading separator in this case is to allow a non-standard coding style imported from another language, which I think is what generated so much opposition (because people see “enable non-standard coding style” instead of “make syntax easier to generate, consistent with other existing syntax”). The argument against using a trailing separator like the rest of the language does wasn’t particularly substantive either, further highlighting that the proposal likely exists to support an aesthetic preference, which isn’t quite the goal of Rust’s syntactic flexibility.

[aturon]

There were several interesting revelations in this thread regarding the working of the teams that explain a lot, but are not AFAIK commonly known ‒ at least I didn’t know them and I try to follow what happens around the language ‒ I regularly look at this forum, I have the subreddit in my RFC reader, I hang out in several rust gitter channels…. Therefore, I have to ask: are these things documented somewhere? Is there a checklist what to look into when you’re evaluating RFC? The policies like the one you just described? Would it make sense to make them more visible, so people don’t react out of ignorance? Are the meetings recorded somewhere (I’ve found rust-lang/minutes, but these seem to end about a year ago), if one is interested in following the discussions? I don’t want to say you try to operate behind closed doors, but sometimes it needs a thread like this to understand what is actually happening.

This is really helpful feedback! It’s a perpetual difficulty with a project the size of Rust to “get the word out” about things (because there’s always so much happening), but I would love for the outcome of this thread to be a set of some concrete things we could communicate better.

To that end, it’d be helpful if you could enumerate what you think were the most eye-opening lessons here, and we can figure out where to document those.

re: meeting minutes, we’ve tried a few different approaches, but what we’ve found works best is that all the key points of discussion are taken back to a thread – either on an RFC, or as a new internals thread (if we’re talking about some new ideas). And we specifically encourage individuals on the team to write up their own comments with their own perspectives, rather than speaking “for the team”. But admittedly, all this isn’t done with perfect consistency, and it might be worth taking some steps there.

[mark-i-m]

This is not some secret process. Someone posted on the tracking issue in the rust-lang/rust repository proposing the feature be stabilized. This triggers a minimally week long process in a public forum where people who care about the issue can comment on what’s going on. In what way is this not seeking feedback from the community? Should we be emailing every single Rust user before stabilizing a feature or something?

It would be kind of nice to have a list of current FCPs published somewhere. I find the “Current RFCs” section of TwiR really useful. Something similar for stabilizations would be great. Personally, I don’t think this is a problem with the process, just with visibility.

[mark-i-m]

To that end, it’d be helpful if you could enumerate what you think were the most eye-opening lessons here, and we can figure out where to document those.

One seems to be policies for specific types of changes.

• What needs an RFC and what doesn’t (in a bit a more detail than what’s currently in the RFCs repo README).
• Policies on type of changes to various parts of the language. e.g.
  • Adding small syntactic changes/adjustments, e.g. 1925 => no RFC needed
  • Adding small helpers to library types => no RFC needed
  • New syntax construct => RFC needed
  • New API => RFC needed
• High-level rust design decisions
  • e.g. what @withoutboats said above, which was news to me too.
  • things like “immutability-by-default” or “avoid UB at all costs” or “make macro-authors’ lives easier” seem like they would also be good to include
  • Having a checklist of this sort would help macro authors because then they can point to some objective agreed-upon goals to measure an RFC by…

[scottmcm]

Policies on type of changes to various parts of the language. e.g.

Some of these are documented in the various parts of https://github.com/rust-lang/rfcs#sub-team-specific-guidelines; is there anything specific in there that’s outdated or should be improved?

[sfackler]

It would be kind of nice to have a list of current FCPs published somewhere.

You can always look at these at https://github.com/rust-lang/rust/labels/proposed-final-comment-period and https://github.com/rust-lang/rust/labels/final-comment-period but it seems pretty reasonable to stick the new ones in a new section in TWiR every week.

[withoutboats]

There were several interesting revelations in this thread regarding the working of the teams that explain a lot, but are not AFAIK commonly known ‒ at least I didn’t know them and I try to follow what happens around the language ‒ I regularly look at this forum, I have the subreddit in my RFC reader, I hang out in several rust gitter channels…. Therefore, I have to ask: are these things documented somewhere?

The answer is generally no (and I agree with Aaron that it would be good to make progress on this front) but I’d emphasize two things as explanations:

1. The project has been under a continuous experience of exponential growth through its entire history, resulting in a constant exponential growth in information to be distributed and people to distribute that information too, making it very difficult to distribute information adequately.
2. These kinds of policies are largely established through precedent. We never sat down and wrote up our rules about grammar, they emerged over time through examples until they became a norm.

[Centril]

To expand upon @sfackler’s answer, rfcbot applies the label disposition-merge to PFCPs which are newer than those filed after 1st of May 2018. Thus, you can see all to-merge FCPs in the rust repo with: https://github.com/rust-lang/rust/issues?utf8=✓&q=is%3Aopen+label%3Adisposition-merge+-label%3Afinished-final-comment-period (is:open label:disposition-merge -label:finished-final-comment-period).

I’ve retroactively applied disposition-merge to all relevant RFCs and cleaned up final-comment-period so applying finished-final-comment-period retroactively as well.

[vorner]

This was talked about in another thread and my feel is it was thought to be a good idea in general (adding a section for „close to stabilization“ features). This is the RFC I’m trying to write, but I have to find some time to actually do it ‒ probably over the weekend. And I plan to propose something as lightweight as possible (I guess the „changing the process regarding stabilization“ sounds much bigger than what I have actually in mind).

Idea: mandate N independent uses before stabilizing a feature

You should write an RFC changing the process regarding stabilization. Depending on how broadly it applies, it would then be under the purview of the language team, the libs team, or maybe the core team. See the team structure for more info on the teams. See the README on the RFC repo for details on the process. @scottmcm is on the language team, so you’ve essentially already contacted the language team by writing here

To that end, it’d be helpful if you could enumerate what you think were the most eye-opening lessons here, and we can figure out where to document those.

For me, it wasn’t outright eye-opening, just pieces to the puzzle, but I’ll go through it and find what exactly it was, also over the weekend.

[glaebhoerl]

It would be kind of nice to have a list of current FCPs published somewhere. I find the “Current RFCs” section of TwiR really useful. Something similar for stabilizations would be great.

Everything else aside this is an excellent and in-hindsight-obvious idea and I’m honestly surprised that they aren’t already published in the same place, and that I hadn’t even consciously realized that they aren’t.

[Ixrec]

I’ve seen this suggested at least half a dozen times by now (I suggested it myself several months ago), and so far it seems like everyone has exactly the same purely positive reaction.

Does anyone know who we need to talk to to make TwiR actually implement this?

[vorner]

As mentioned 2 comments above here, I’m writing an RFC in that regard (as suggested in another thread) and I hope to submit it soon. I hope that will make it moving :-). In other words, I don’t know who to talk to yet, but I decided to find out and push it through (unless there appears some reason why it shouldn’t be done).

[djc]

This is really helpful feedback! It’s a perpetual difficulty with a project the size of Rust to “get the word out” about things (because there’s always so much happening), but I would love for the outcome of this thread to be a set of some concrete things we could communicate better.

To that end, it’d be helpful if you could enumerate what you think were the most eye-opening lessons here, and we can figure out where to document those.

I’ve really been feeling the lack of communication from teams to their audiences, in particular the “older” teams. From my point of view with a long history working in open source projects, it’s been surprising to me that teams mostly operate out of sight and while notes are sometimes available if you go look, these aren’t always very accessible.

In contrast, in many open source projects that I worked with over the past two decades, there was a policy that everything substantial has to find it’s way to asynchronous communication so that it’s more accessible and everyone has a chance to weigh in. From the start, the Rust project has been oriented towards somewhat-closed-by-default teams (mostly invite-only membership) with a lot of the core discussion happening in synchronous communication, which makes it that much harder to keep up with what’s going on.

(I hypothesize that this is part of the culture of the research group at Mozilla that Rust sprung from, and that maybe there aren’t a lot of more old-school open source types involved in the core Rust leadership group.)

It sounds fine if someone consistently takes back the key decision points from a meeting back to a thread on a more personal note, but that should really happen way more consistently. In my opinion, this is a significant factor holding back the larger Rust community (and makes it harder, to reference your Rust 2018 post, to elicit technical leadership).

Personally I would like to see if every team or working group post a bi-weekly or maybe monthly announcement of what’s going on, and for the team/wg leads to be accountable for that process. It doesn’t have to be a long and it can be liberal in just linking to whatever asynchronous material is already available, but it should be published and advertised (for example in TWiR).