Fortifying the process against feature bloat

I must say I'm in awe of you people, it's the second time someone in this thread formulated some of my thoughts while I didn't even know I had them.

I do follow some RFCs so I know these are not true in the form you state them. But I believe there is a thing in the process that work in part as you describe. The thing is human subconsciousness. It has very important part on how we think ‒ and I believe this is one thing Rust as a language uses well ‒ for example, the ownership model (after getting used to it) feels much more natural and subconsciousness is able to grasp it.

I know you (as in, the teams) do research about the effects of the proposal. However, when deciding, arguments are not considered equal, they are weighted. Arguments that are more readily available are perceived as stronger (that's how human mind works). Therefore, if something is written in prominent place in the RFC itself, it will be seen as stronger argument than something deep down, lost in hundreds of comments, or the thing in the comments really needs to be very impressive to balance that.

Furthermore, you said you rely on the community to provide some of the feedback, and while not directly following it every time, you get influenced by it. You say that the RFC process is not a popularity contest, but the popularity has some influence. And even if you do full research about the effects of RFC, members of community usually don't have as much time to do that ‒ with day jobs that don't allocate time for Rust RFCs.

Therefore, I believe something written in the RFC has more effect on the decision in general than something not written there. Asking the author to write it there both saves the research of many people and gives the information equal(er) chances.

Again, written in this form it sounds somewhat ridiculous and obviously is not truth. And I believe it's more nuanced here than with above.

First, let's consider a vastly simplified model, where a proposal is either accepted or rejected (I know it gets modified during, but let's simplify for now) and that there's an universally good decision.

Every point on the process is human-driven. And humans err. So, there's a chance (let's say 5%) that a team member decides wrong. Depending on how many 95%-good checks are on the way, the percentage of bad decisions that reach the final stage will differ. But unless it's 100%, you get some percentage of bad ones on the output. So, if you are able to influence the input and turn some bad ones to good ones, you just get a better overall results. If you manage to lower the error rate by providing more accurate input (because the decision is partly based on expertise, partly on future guessing and, ‒ sorry, but everyone here is human ‒ on random effects like the amount of coffee), you get better overall results.

So, the question is, how many checks there are? And it would be inviting to count the members of corresponding team making the decision. But this would be the case only if you were deciding independently. You know each other, because you work together. Your decisions are going to converge over time slightly. Furthermore, you're all the best experts on Rust out there. In this case, it is a weakness (on the university, we joked about the professors not appreciating the depths of students' ignorance). If I selected a group of, say, C++ programmers, the sensitivity of the group towards what a bloat is would be somewhere else (my own experience would suggest an alergic reaction to bloat :innocent:). Neither is the correct one and I agree the input of the community helps in part ‒ nevertheless, you have the final say. Here I'm trying to arm you against the inherent bias brought by being experts.

Therefore, if there's a relatively low-cost way to improve both ratio of good ones and quality of the information about how good every specific RFC is, I think it might be worth trying. And if it saves time during the research phase for many other people, I guess the argument for low-cost is rather easy.

And there's a third thing. There are not only good/bad proposals. Even between the ones that make it, there are better and worse ones. When a proposal arrives, people try to improve it. However, the amount of energy people spend improving it is somehow limited ‒ not a hard limit, but depending on the motivation. If the proposal is good to start with, you're bound to have more motivation (because it's joy to work with) and as the starting point is better, the result will also be better. And, if there are too many „bad“ ones sucking energy, there's less energy to improve the good ones.

It this case, yes. And I think if you make a random selection out of the accepted RFCs, you're much likely to find the good cases, because, as pointed out, the process already does damn good job. But can you honestly say there was no „Huh, why not?“ RFC? An if not yet, that there will never be? That there's nothing to improve on? Of course you can argue that the error-rate is so low it doesn't pay back to try to fix it.

I guess this one is more about perception of the dange of bloat than the danger itself. I mean, if I read mostly the forum, my view on the future turns bleak after a time. I probably should read more of the „mature“ RFCs to have a happier outlook. Anyway, same argument as with „improving a better input gives better output“ applies here as well.

I saw the original idea in this direction (negative RFC) as being able to prevent feature bloat ‒ if once decided something is bloat, it would never get in. However, now I see passing a negative RFC while still in the controversial state would not be realistic ‒ so that wouldn't fly. Documenting the knowledge was a natural byproduct of that that doesn't address the original problem (at least, not in any direct way ‒ arguments about lowering the standards by seeing too many re-surfaced RFCs would be quite far fetched). But it seems to solve another problem :smiling_face:.

1 Like

Here is what it takes for a language feature to go from a Pre-RFC to stable Rust (in my experience):

  1. The Pre-RFC is filed (sometimes it goes to RFC directly without a Pre-RFC).
  2. A bunch of discussion happens; and the proposal probably doesn't make it past this point.
  3. The RFC is filed
  4. A bunch of discussion happens
  5. At the earliest lang team meeting where we have time over for the RFC it is discussed and a shepherd / owner is assigned
  6. A bunch of discussion happens
  7. The shepherd / owner will read the RFC and more or less all comments.
  8. The shepherd will eventually nominate the proposal when it has made sufficient progress (enough discussion) and it is discussed on the next lang team meeting. A lot of proposals get closed or postponed here.
  9. PFCP merge
  10. All but 2 team members must OK the proposal actively.
  11. Some lang team member(s) might file concern(s) here after the meeting. It happens quite frequently.
  12. If no concerns were filed, and all but 2 members OKed the proposal it goes to FCP.
  13. It stays in FCP for 10 days. Any concern filed takes it immediately into PFCP again.
  14. The RFC is merged and a tracking issue is filed.
  15. The RFC is implemented and all unresolved questions get resolved. Changes do happen here.
  16. Tests and documentation must be added.
  17. It must bake for for some time in nightly
  18. Someone must actively propose the feature to be stabilized. When this happens, the PFCP -> FCP procedure happens again for stabilization.

I might be overstating how lengthy this process is, but it seems to me that it is full of opportunities for important comments and objections to be raised.

It is hard to think about this without specifics. Mistakes can be made, but our process is quite good at ensuring it happens extremely rarely.

EDIT: To clarify, the interesting question is if changes to the process would actually make mistakes more unlikely -- we can tweak here and there, but I think the process is working well.

A closed RFC means that the same idea shouldn't be filed again. It's up to the team to judge what "same" constitutes (how similar does it have to be...), and you'd have the same problem with negative RFCs in any case... A team would have to judge how similar a proposal is to the banned idea.

10 Likes

To elaborate on this, I think it could be, among other things,

  • New motivation is found for the same feature
  • An alternative solution is found for the same problem
  • Over a substantial time period, the issue it addresses has become serious (probably at least a year, increasing as we get further from 1.0)
4 Likes

To be clear, I fully agree with this. What I worry is point 8 and 10. If I imagine I'm in the position to decide: I'm looking at the RFC and deciding if I should OK it or not. I read the proposal. Then go through the comments. And I have to make a decision. Sometimes, it is clear-cut. But sometimes it feels like I'm not sure ‒ there were many comments, emotions, many people who want the proposal and many that don't. How do I decide? I don't know if you people in the teams have some way to handle this, but for me, my gut feeling would have a lot of say in the decision ‒ and my gut decides on quite many things, not all of them really relevant to the question ‒ like, what argument feels stronger, which is not always which one is better.

I'm trying to suggest to include things in the main RFC text (because it's the part that influences the decision most) to include most of the relevant information ‒ to guide the subconsciousness to decide more on facts and less on feelings and random influences. Because, no matter the length of the process, this is the point where the actual decision is made.

With the risk someone will get too attached to specifics, the one I had in mind was 1925. The lesser problem is I don't like it. The fact there were many thumbs-down is some indication about people's feelings about it, but that's also not my main concern. What I find most disturbing about it is (It's always easy to criticize in hindsight, so I apologise for doing just that ‒ but I want to provide the specifics asked for):

  • The RFC is missing several sections. Why wouldn't they apply? And why nobody (if I scanned the comments correctly) even mentioned it and asked if there really are no downsides to that?
  • The RFC was merged in confusion about which version people commented on, because there were two very different versions during the discussion.
  • There was no strong argument for it. Making it easier for macros was mentioned in the comments, but nobody actually brought an example of a macro that showcases that. The RFC wasn't updated to reflect that.
  • The thumbs-down count was dismissed as unimportant, on the bases that they were probably for the leading+trailing version and without further investigation.
  • It was stabilized without seeking further feedback from community, even though that was mentioned after merging it and even though there was the confusion.

I wouldn't have counted it as an error in the process if the RFC was better written, or if ‒ upon the confusion ‒ the discussion was restarted (eg. the merge request closed and a new one started) to gather new thumbs-up/thumbs-down counts instead of just dismissing them (I know this is only one of the decision inputs, but still) and if it was clear how the decision was made. In that case I would still not like the feature, but would be content knowing that it was reached in a fair way and pros and cons properly weighted, so I'm just the one being weird. But looking at so many „oddities“ around it, it feels somewhat unfortunate, and some reactions from community suggest I'm not the only one. In other words, I believe in this case, the decisions were influenced more by some random noise than actual facts, because most of the discussion has a lot of interference. And I'm still not sure what problem was this meant to solve.

Now, I don't want to overblow this one, because the feature itself is no big deal. But it shows there can be errors in the process. And I think it also somewhat undermines the trust in the process, which is unfortunate. I could probably come up with few more examples of something that felt like the consensus wasn't really clear and strong, but I think it'd only be twisting knifes in wounds and I don't want to do that.

I don't really want to change the process itself ‒ as you say, it is working well most of the time. I just want it fed with better input. Currently I believe the best way would be to compose some micro-guide (or a checklist) for RFC authors. As I argue above, I think it should help (somewhat, it wouldn't be a silver bullet, of course), but I guess nobody knows without trying. So in that sense, is there any risk in trying to compose such a short guide? I can give a shot at first version, but I've only read RFCs, not written them (I have the editor open right now, with the RFC for announcing things about to stabilize in TWIR, but that's a tiny one and I have no idea yet how that'll go ‒ but if there was a guide to help me with that, I'd definitely go and read it) ‒ so to have any validity, it'd need some help from others, who do write RFCs.

So, if I start writing it, are there people who would like to help? Or, is there some reason why I shouldn't do that?

That's good to know, with that knowledge, the process does look more balanced. Are these things documented somewhere (eg. how the teams reach their decisions)?

8 Likes

Thumbs up/thumbs down counts are inherently unimportant. If someone want to have an impact on an RFC, they should read it and thoughtfully comment on it. A thumbs down provides no information other than someone clicked a button for some reason at some stage of an RFC. The Github emoji reaction feature didn't exist for the first several years of the RFC process, and we weren't stumbling around blindly during that time period.

This is not some secret process. Someone posted on the tracking issue in the rust-lang/rust repository proposing the feature be stabilized. This triggers a minimally week long process in a public forum where people who care about the issue can comment on what's going on. In what way is this not seeking feedback from the community? Should we be emailing every single Rust user before stabilizing a feature or something?

5 Likes

Oh; nomination doesn't mean that you have to agree with the proposal; when I nominate a proposal it usually means that the discussion has slowed down, most discussion points have been made and the pros and cons are relatively clear. A nomination often results in postponing or closing an RFC. It just means that "the time is right" to discuss the RFC formally in the lang team meeting. I might raise some concerns, etc. but most lang team members will have also read the RFC as well.

I agree with this and I try to / usually do it with my own RFCs when I have the time, but it is rather time consuming (lots of busy work...) to add summaries of arguments in the RFC especially if it gets a lot of comments very quickly. Often, at least for my own RFCs, some arguments that are made in the RFC thread have already been discussed in the RFC itself, so the work is already done.

Honestly, RFC 1925 is such a nano-RFC that I kinda don't know what the fuss is about :slight_smile: Personally I found @withoutboats's arguments wrt. flexible grammar being part of Rust's philosophy to be convincing. If there have been mistakes in the language design, I don't think this one is it. I haven't read that RFC in depth tho and I'll take a deeper look wrt. the text itself (not the feature..).

I think it is important to understand that we usually don't discuss the GitHub reactions at all on team meetings, so they don't factor into the language teams decision making process. If it is discussed on a meeting, it is merely to try to maintain community cohesion and good will. I.e: the team will feel uneasy about the number of negative reactions. It is barely a decision input.

I also think that RFC 1925 is used as an example too much; it's importance is way overblown in my opinion.

Try it out, it is a nice experience :slight_smile:

If you are up to the task, go for it. Maybe we'll make some changes here and there.

3 Likes

The response that this RFC has received from certain members of the community is really baffling to me, and seems to stem from ignorance about our policies on grammar. In the past we've merged changes to the grammar that in my opinion are at least as "bad" on the basis of the arguments against that RFC without even an RFC at all. In fact, I think our policy is that that kind of change simply doesn't need an RFC, and the RFC was overkill.

Here are two prominent examples:

  • Trailing +. T: 'static + is a grammatically correct bound.
  • Turbofish in types: impl<T> Option::<T> is a grammatically correct impl header.

Here is a valid way to write the main function signature in Rust:

fn main<>() where for<'a> (): 'a +, { }

Neither of these are accidents of any kind; they were both conscious changes made within the last 12 months, without an RFC process. And they all stem from the same basic underlying language policy: Rust has a very flexible grammar, trailing or leading separator and unnecessary characters are supposed to be permitted liberally, and extensions to its flexibility are considered bugfixes (for example, fn main::<>() doesn't parse, but probably should).

This policy was made with full awareness of the downsides (e.g. there are multiple, redundant syntaxes that have the same meaning), because we weighed the upsides as greater (especially the benefits to macros, but also often there is an underlying grammatical unification that a redundancy enables). Reiterating the downsides as applied to a particular case is not a compelling argument to make an exception to longstanding policy. There would have to be a very compelling counterargument which is specific to this case.

Its our ability to overrule the "votes" of the community (which have never been an actual part of the RFC process) that helps protect the longevity of the language, because we do better at pursuing a consistent vision than a shifting, temporary body made up of the people who happen to read a github issue would be able to. This RFC is, in my opinion, an excellent example of how our existing practices avoid the kind of negative outcomes you are worried about.

21 Likes

I agree this one received more than its share of beating, so I was reluctant to bring it up. But now I’m glad I did, because after what you say here it now makes much more sense (even the somewhat visible displeasure of the team members whenever this one is brought up). Each „side“ operated under very different understanding.

I mean, I didn’t know about this policy. The motivation in the RFC is somewhat inadequate to warrant any feature, even a small one. If I read just that, it makes me sad why anyone would add to the language just because of that, so I have a tendency to express my dissatisfaction (which, I guess, went a bit viral in the community). But if it wrote „Make the grammar more flexible in regards to | in patterns and make it more consistent with other separators which are allowed even at places where not necessary, in part to make life of macros easier“, it would give completely different impression. If one of the first comments at the merge request said „Oh, we usually allow these little details even without RFC, according to our policy [link]“, that would also look different. But reading just the information available there, it gives the impression of very „Oh, why not“ feature.

There were several interesting revelations in this thread regarding the working of the teams that explain a lot, but are not AFAIK commonly known ‒ at least I didn’t know them and I try to follow what happens around the language ‒ I regularly look at this forum, I have the subreddit in my RFC reader, I hang out in several rust gitter channels…. Therefore, I have to ask: are these things documented somewhere? Is there a checklist what to look into when you’re evaluating RFC? The policies like the one you just described? Would it make sense to make them more visible, so people don’t react out of ignorance? Are the meetings recorded somewhere (I’ve found rust-lang/minutes, but these seem to end about a year ago), if one is interested in following the discussions? I don’t want to say you try to operate behind closed doors, but sometimes it needs a thread like this to understand what is actually happening.

The thumbs-down thing: I understand these are not an input to either pass or sink a proposal and I don’t even think of different output of the team as „overruling“ ‒ there’s nothing to overrule. But I think it is a valuable mood-meter that can indicate some general unhappiness about something. It probably should be an input to investigate into what the something might be. The discussion raised the number of them, but found an excuse for them. In the light of „this doesn’t really need a full RFC anyway“, it makes sense to dismiss them, but it still gives impression you don’t care what the community thinks and that it feels unhappy. I guess more investigation into dissatisfaction would happen on a „full blown RFC“, but again, it wasn’t clear there’s actual reason to consider the RFC not being „full“ in the eyes of casual reader. Small, yes, but that would still deserve it’s due process.

If I try to restate the motivation and the missing sections in the RFC (without changing any of the technical parts) so it better explains the reasons and hopefully makes a better impression even without knowing all this background information and send a PR, will it have to go through a full-blown process, wasting your times, or can it be counted like a small fix that doesn’t change anything and could be merged in some lightweight manner? To avoid further discontent of people who read it ‒ after all, RFCs don’t act only as decision process, but as kind of a documentation.

14 Likes

I think one oddity of that proposal is that it opts to have a leading separator instead of a trailing one as would be more common in Rust. Both are equivalent as far as code generation is concerned, and there's no parsing ambiguity being resolved by having it one way vs. the other. The argument in favour of a leading separator in this case is to allow a non-standard coding style imported from another language, which I think is what generated so much opposition (because people see "enable non-standard coding style" instead of "make syntax easier to generate, consistent with other existing syntax"). The argument against using a trailing separator like the rest of the language does wasn't particularly substantive either, further highlighting that the proposal likely exists to support an aesthetic preference, which isn't quite the goal of Rust's syntactic flexibility.

6 Likes

This is really helpful feedback! It's a perpetual difficulty with a project the size of Rust to "get the word out" about things (because there's always so much happening), but I would love for the outcome of this thread to be a set of some concrete things we could communicate better.

To that end, it'd be helpful if you could enumerate what you think were the most eye-opening lessons here, and we can figure out where to document those.

re: meeting minutes, we've tried a few different approaches, but what we've found works best is that all the key points of discussion are taken back to a thread -- either on an RFC, or as a new internals thread (if we're talking about some new ideas). And we specifically encourage individuals on the team to write up their own comments with their own perspectives, rather than speaking "for the team". But admittedly, all this isn't done with perfect consistency, and it might be worth taking some steps there.

6 Likes

It would be kind of nice to have a list of current FCPs published somewhere. I find the "Current RFCs" section of TwiR really useful. Something similar for stabilizations would be great. Personally, I don't think this is a problem with the process, just with visibility.

5 Likes

One seems to be policies for specific types of changes.

  • What needs an RFC and what doesn't (in a bit a more detail than what's currently in the RFCs repo README).
  • Policies on type of changes to various parts of the language. e.g.
    • Adding small syntactic changes/adjustments, e.g. 1925 => no RFC needed
    • Adding small helpers to library types => no RFC needed
    • New syntax construct => RFC needed
    • New API => RFC needed
  • High-level rust design decisions
    • e.g. what @withoutboats said above, which was news to me too.
    • things like "immutability-by-default" or "avoid UB at all costs" or "make macro-authors' lives easier" seem like they would also be good to include
    • Having a checklist of this sort would help macro authors because then they can point to some objective agreed-upon goals to measure an RFC by...
2 Likes

Some of these are documented in the various parts of GitHub - rust-lang/rfcs: RFCs for changes to Rust; is there anything specific in there that's outdated or should be improved?

1 Like

You can always look at these at Issues · rust-lang/rust · GitHub and Issues · rust-lang/rust · GitHub but it seems pretty reasonable to stick the new ones in a new section in TWiR every week.

3 Likes

The answer is generally no (and I agree with Aaron that it would be good to make progress on this front) but I'd emphasize two things as explanations:

  1. The project has been under a continuous experience of exponential growth through its entire history, resulting in a constant exponential growth in information to be distributed and people to distribute that information too, making it very difficult to distribute information adequately.
  2. These kinds of policies are largely established through precedent. We never sat down and wrote up our rules about grammar, they emerged over time through examples until they became a norm.
9 Likes

To expand upon @sfackler’s answer, rfcbot applies the label disposition-merge to PFCPs which are newer than those filed after 1st of May 2018. Thus, you can see all to-merge FCPs in the rust repo with: https://github.com/rust-lang/rust/issues?utf8=✓&q=is%3Aopen+label%3Adisposition-merge+-label%3Afinished-final-comment-period (is:open label:disposition-merge -label:finished-final-comment-period).

I’ve retroactively applied disposition-merge to all relevant RFCs and cleaned up final-comment-period so applying finished-final-comment-period retroactively as well.

1 Like

This was talked about in another thread and my feel is it was thought to be a good idea in general (adding a section for „close to stabilization“ features). This is the RFC I'm trying to write, but I have to find some time to actually do it ‒ probably over the weekend. And I plan to propose something as lightweight as possible (I guess the „changing the process regarding stabilization“ sounds much bigger than what I have actually in mind).

For me, it wasn't outright eye-opening, just pieces to the puzzle, but I'll go through it and find what exactly it was, also over the weekend.

Everything else aside this is an excellent and in-hindsight-obvious idea and I'm honestly surprised that they aren't already published in the same place, and that I hadn't even consciously realized that they aren't.

2 Likes

I’ve seen this suggested at least half a dozen times by now (I suggested it myself several months ago), and so far it seems like everyone has exactly the same purely positive reaction.

Does anyone know who we need to talk to to make TwiR actually implement this?

2 Likes

As mentioned 2 comments above here, I’m writing an RFC in that regard (as suggested in another thread) and I hope to submit it soon. I hope that will make it moving :-). In other words, I don’t know who to talk to yet, but I decided to find out and push it through (unless there appears some reason why it shouldn’t be done).