Pre-meta-RFC: rate limiting the firehose

internals policy
1

Summary

Use the rfcbot to make Github-based design discussions more manageable. This proposal is the confluence of a few different parts. I know that this proposal will likely give people a “what?! you’re kidding!” reaction, so please hear me out.

I propose the following changes (motivated and explained further down):

  • As soon as an RFC PR is posted on the repo, rfcbot comments with a Table of Comments (ToC) (<-- heheh get it?) comment. The ToC is empty at the beginning and will be built up as the discussion continues.
  • rfcbot then asks the poster to identify a team to assign the RFC to. The RFC thread is locked until the OP responds with a team. The rfcbot then assigns the RFC to a member from the team to be the RFC’s shepherd.
  • A member of the Rust team (or the OP?) may issue a note command to rfcbot on that thread to add a summary of discussion to the ToC. I propose that this should be limited to 250 characters, plus links to appropriate comments and related other links.
  • At most C comments can occur between notes. If C comments occur and nobody has issued a summary note of activity since the last note, rfcbot locks the thread. I propose C = 30 because that’s about the most I can keep up with before I give up on following a thread.
  • Individuals and interest groups are encouraged to discuss proposals on their own repos or on internals before posting the RFCs repo.
  • The Code of Conduct is updated to explicitly state that it is considered poor etiquette to comment without making a good-faith effort to read and understand the ToC. Repeat offenders may be evicted.

Motivation

This proposal aims to address the following problems

  • RFC discussions can get quite lengthy, which is exhausting to read and participate in, both mentally and emotionally. Even highly dedicated individuals, such as Rust team members, have expressed this feeling.
  • Lengthy RFC discussions are hard to read on GitHub. This is because
    • GitHub collapses comments in the middle
    • GitHub marks all notifications read if you visit a thread once
    • Discussion on review comments don’t appear at the end of the thread, so it is easy to miss discussions because you are only following the end of the page.
    • Summary comments get lost easily in the flood.
  • Many RFCs get accepted without the necessary bandwidth to push them to completion. This leads to many people using Nightly rust because that’s where their favorite feature lives in pseudo-limbo for the foreseeable future.
  • Many people comment without reading. I can’t blame them. Catching up on a long thread is rather hard. However, this means that issues tend to get brought up multiple times, compounding the problem.
  • Many proposed features don’t fit clearly into the roadmap.

Explanation

At a high level, the idea is to make the Rust team the bottleneck in the RFC process. This is counterintuitive but intentional: if a Rust team member doesn’t have the bandwidth to keep up, I hypothesize that very few others in the Rust community will have the bandwidth, except those biased by a strong interest in the particular feature.

In contrast, by having a Rust team member there the whole time:

  • A proposal has somebody to shepherd it to the finish line (stabilization).
  • There is always a Rust team member up to date on the discussion
  • High-quality summary posts (the ToC) get generated as a side-effect of the discussion.
  • The RFC process is rate-limited to something that can be sustained.
  • Discussion of less complete proposals gets pushed to other media (e.g. internals or other repos).

Note that this proposal is not intended to produce more stress or workload for team members. Rather, we make it the new norm for team members to say “we don’t have bandwidth” or “this doesn’t fit into the roadmap”, which is already the case for many RFCs. If nobody on the team champions the proposal for 2 weeks, it is closed automatically by rfcbot.

Wow. This seems draconian.

Yes, I realize that. In part, I am trying to be controversial enough to start a discussion. But in part I think Rust as a project has reached a level of maturity where we need to slow down the pace a bit, and part of that is raising the bar for new proposals IMHO.

10 Likes
2

Half of your motivation is based on the point that “GitHub is bad on lengthy discussion”. As a potential option here, can we move RFCs to Discourse? Discourse is pretty good on moderation, continuing reading, and also my favorite feature, summarization. The like-based model seems to actually work, and that’s IMO better because no human intervention is required (which may be time consuming).

7 Likes
3

A PR is welcome to rfcbot; this doesn't require an RFC, just implementation work. Relevant issue:

What's the point to this delay? I do RFC triage usually within minutes of an RFC being filed. Furthermore, a poster might not know what team to assign and there may be multiple teams to assign.

In T-lang, we do RFC assignment more deliberately than this; in meetings where we have extra time after triage (issue/PR triage almost always comes first...) we assign members based on area of expertise and interest; I like to believe this has a better outcome than random assignment. The fact that we are limited by triage and team meeting time acts as a natural rate limiter on accepting proposals. For example, during the work on the edition we were doing a lot of triage and most accepted language design was driven by edition necessities.

Doesn't require an RFC; just implementation. I'd accept such a PR; I'm generally in favor of giving team members more commands in their toolbox to use at their volition.

Note sure I can accept this without T-core approval however. I would note that not all comments have the same density; for example, line comments that fix typos that get resolved quickly you don't need to keep up with.

This already happens?

I think this is more about the RFC than the discussion format. I think any proposal about error handling will for example be quite long. Meanwhile, an RFC like RFC: Associated type defaults by Centril · Pull Request #2532 · rust-lang/rfcs · GitHub gives the opposite feeling.

This is quite annoying yes; maybe we could solve it by reaching out to GitHub and ask for a setting?

This is not my experience; RFCs either a) get implemented quickly, b) are blocked on design questions, c) are blocked on larger architectural movements (e.g. chalk). Do you have concrete evidence for this claim?

I think that's fine -- and welcome even; these are usually small proposals that someone who's interested can implement on the side; for larger bits, they can be thought of as more long-term ideas that can be discussed over a whole year if necessary... small accepted proposals also provide a mechanism for onboarding new developers to the compiler. I don't think we should try to be too strict about conformance to the roadmap.

In what sense is it not already? It's my distinct feeling that the bottleneck for T-lang proposals is T-lang itself.

Is there an inherent value to this? I don't need to be fully up to date on proposals that we don't have time to consider right now... But I do try to read up on things before meetings if something is nominated.

How is an RFC championed? You noted above that rfcbot assigns a member... I think that for T-lang this will just mean that I will champion every RFC...

Also discussed in: Raising the bar for introducing new syntax I personally don't think we need to slow down or raise any bars; in fact, I'm trying to speed up language design to fix inconsistencies and to align with the work on Chalk.

4 Likes
4

The ToC sounds super helpful.

For me the worst thing is when the RFC text doesn’t reflect the consensus in the discussion thread, so people coming to the thread keep commenting about the same thing over and over.

6 Likes
5

Is that bad? :stuck_out_tongue:

This, this, this, a billion times.

Finishing off compiler or language infrastructure work is always welcome. What I think Mark refers to is that the constant influx of new features and the rate at which they are accepted is something we need to work on. There are many features requested daily, and some of them have a record of getting accepted without listening to opposing viewpoints. Some of the most controversial features aren't given too much design time nor time to mature. This definitely is a problem that we should manage somehow.

I don't believe OP is suggesting the team to stop working on the compiler or other infrastructure. Nor do I – in fact, I have stated this before, but I think it is quite "ironic" (for the lack of a better word) that while so many new features are being proposed and accepted, several much more fundamental ones have been pending for too long. For example:

  • Const generics (and in particular, arrays generic over the size parameter)
  • Trait objects of more than one non-marker trait
  • Implementing Fn* on user-defined types
  • Associated type defaults

Consequently, I think that implementing, polishing design-wise, iterating, testing, etc. these more basic features should be the primary goal while there are several such outstanding unstable features in the queue. We should come back to think about new features after this foundational work is complete.

There other issues that should have a higher priority IMO, too. For example, it was only a couple months ago that I've read in one of the discussions on GitHub that "the compiler has much long-standing technical debt". Certainly, I don't believe that more and more features and the consequent need for handling more and more special cases (e.g. feature gates) helps that. So why isn't more effort being focused on refactoring the compiler and getting rid of the technical debt instead, also fixing bugs on the way? After all, the compiler is one of the cornerstones of the language; I'd like to think that it's an exemplary piece of engineering work, which is not suffering from bad architectural decisions and hacks. The correctness and soundness of the generated code can depend on this fact, so it should also be a central issue, taking priority over often not-very-well-founded new ideas.

1 Like
6

The pain I feel most keenly in keeping up with RFC threads is definitely the lack of regular summaries (and even more so after the RFC has been completed). If nothing else, I think that someone (the RFC author, if no-one else volunteers) should be obliged as part of the RFC guidelines to write these regular summaries.

However, I think a warning message, rather than locking, should be enough. Locking is very detrimental: the RFC author is unable to unlock the thread, for instance, even to add the summary. An RFC should be considered "blocked on author" until a summary has been posted, which should make its status clear.

6 Likes
7

I tend to not participate in the RFC process exactly because (1) Github has awful UX (2) the IRLO/github split is obnoxious. Given that participating in Rust is not part of my full-time job, my participation is mostly limited to downtime when I’m running tests at work, and having all of the discussion happen in one place would be very, very valuable.

Unfortunately, we do lose the ability to (ab)use Github’s code-review features for commenting on specific parts of the RFC.

3 Likes
8

Doesn't Discourse's "Quote" functionality (as seen above) already kind of do that?

9

Not really the same =/. It doesn't update as the RFC text changes.

1 Like
10

Fixing inconsistencies in the language is not compiler infrastructure work, it language design. You first have to find the inconsistency, write a proposal, ...

I understood that and I disagree with Mark.

  1. Concretion would be helpful.
  2. Being on the language team I'm biased of course; I think we do a good job, so this is not my assessment.

This is the wrong team; T-Lang does not work on compiler infrastructure, T-Compiler does. My interpretation of what Mark and you are saying is that T-Lang should substantially reduce the number of accepted proposals (which means also reducing the work we do).

It is being worked on by @varkor et. al. From my understanding it also depends on work on Chalk which is being worked on by many people.

This is a new feature and also blocked on language design.

Blocked on language design.

Blocked on language design; I've proposed a solution in RFC: Associated type defaults by Centril · Pull Request #2532 · rust-lang/rfcs · GitHub.

To some degree I think it's possible to do both at the same time since it's not the same set of people working on both things.

Also, it's not clear what "these more basic features" are to me; For example, https://github.com/rust-lang/rfcs/pull/2535 is finishing the basic feature of or-patterns.

Furthermore, if you believe that RFC-accepted foundational aspects of the language need more work, you have the option of contributing to the compiler. I would be glad if you did as that would be the most helpful thing you could do to advance what you are saying.

I also noted before that T-Lang and T-Compiler have to a large extent been focused on shipping Rust 2018; to me, this is completing foundational and architectural work.

I certainly agree with spending time on shaving of technical debt and I agree there is substantial amount of that. However, as I noted above, fixing some of the features you enumerated requires more feature gates (which don't translate into "special cases", just unstable changes to the language -- indeed, a feature that removes special cases can be gated). Therefore I feel that what you are saying is internally inconsistent.

Work is being focused on doing this; why do you say otherwise? This is what WG-traits, and WG-NLL are doing: refactoring the compiler and getting rid of the technical debt.

I would be grateful if you would help out; in particular, I think that the compiler could use substantially more documentation of its various parts.

1 Like
11

So regarding GitHub vs Discord etc. I see a few major benefits to GitHub, which is why my leaning is toward trying to tame the madness rather than switch to something else:

  • Everyone is already on GitHub, whereas requiring a login to another platform will be a paper cut for new contributors. We already have GitHub, users.rlo, internals.rlo, discord, zulip... Keeping it on Github means less fragmentation
  • GitHub is good for record keeping. It is not a chat platform, whereas all the others are. This is a good thing IMHO because it means that discussions are easier to read after the fact (even if GitHub does have its own UI problems). Even the form of discussion will be slightly different. I'm worried that switching to an actual chat platform will cause RFC discussions to be come very chatty/noisy and impossible to follow after the fact.
  • GitHub allows cross-linking easily. This is useful for documentation purposes.

IMHO, GitHub is actually not that bad. We just need to find ways of keeping it productive and scaling it to larger discussions.

@Centril

I think @H2CO3 summarized my main point well. I'm not trying to stop RFCs -- just rate-limit them a bit to what we can actually manage. I do think we are going a bit fast at designing and not fast enough at polishing and stabilizing.

I am not convinced that having new features in every release is a good thing. I would like it to be more like every third release or something... not by artificially slowing down stabilizations, but simply by having fewer things to stabilize. I suspect we might just disagree on this point.

Let me clarify: my observation is that many features are blocked on (a) fcp merge for RFC; (b) implementation; or (c) fcp merge for stabilization. I propose we move this "blocking" up front: an RFC doesn't get even get discussed on the RFCs repo unless the appropriate team has allocated bandwidth to shepherd implementation all the way to stable. (Discussions are, of course, welcome elsewhere).

My point is mostly that there should be some human assigned to every RFC. How the teams do that is up to them. Random is the lowest-effort strategy I could think of, but it's just an idea.

I leave that up to discussion :stuck_out_tongue:

2 Likes
12

I think this is a valid point, but I argue that blocking has two advantages

  1. It means a team member will always be in the loop, which should be helpful for triage, nomination, etc.

  2. It does hard rate-limiting of discussions. There cannot be more than 30 comments to a ToC update. The ToC will always be reasonably up to date, and anyone coming in can easily catch up on the state of discussion.

1 Like
13

Also, cc-ing @steveklabnik after reading their Rust2019 post…

14

Yes, I sympathize with these objectives but want to solve them in a different way. We’ll see!

15

I do realize, perhaps not the best wording on my part. Isn't Chalk going to be the part of the compiler though? I see it as a multi-purpose effort: making the compiler smarter (this is where it affects language design) while improving the architecture of its code at the same time.

I also agree that fixing language-level inconsistencies is great. Small improvements and making existing constructs more powerful using incremental steps is fine with me, and they are explicitly not the issue I am talking about.

What I was referring to is large and mostly additive changes (which IMO often make inconsistency worse in fact). These are usually not the ones you propose; they are being scattered all over GitHub and IRLO…

sure, here are some recently-posted proposals which are mainly additive and I think not as important as the others I enumerated above:

As for the examples of features being accepted despite considerable controversy: default binding modes and impl trait in argument position.

Sure; I even explicitly wrote "compiler or other infrastructure" this time. I appreciate the work of both teams, and yes, "T-Lang should substantially reduce the number of accepted proposals". This doesn't contradict that other teams should continue with what they are doing – quite the opposite, in fact. Both compiler work and smaller-scale language design polishing is great – I would just like to see fewer radical and large-scale, high-impact changes to language design.

I don't understand why "blocked on language design" is being used as a counter-argument here; design is just as important in the lifetime of a feature as implementation is. Furthermore, it's not primarily the age of the individual features that bothers me. Rather, it is their much more fundamental and "simpler" (from the user's point of view) nature. In other words, it just feels wrong to get lost in discussions about sophisticated and complicated new ways of creating enums, writing loops, or overloading the assignment operator, when all I wanted is Box<Read + Write> (which is something quite natural to want IMO).

That's a perfect example: it's a small change which improves consistency by allowing something to be arbitrarily nested whereas previously it was a top-level-only construct. I think RFCs like this are always welcome; this is exactly the kind of polishing I was referring to earlier. I'm not against small improvements like this, because they do drive the given language feature towards completion, and they don't focus disproportionately much effort and/or attention.

I'm only against big, overly ambitious changes while there's still lots more to flesh out. I realize that we might have different definitions of what's "fundamental", so the disagreement might stem from this.

We agree on this completely.

Sorry, it was indeed not the best way of putting this (let me change it in a moment). I wanted to ask why isn't more work being focused on it? I realize that the speed of getting things done doesn't increase linearly with the number of people working on them, but what I was trying to suggest is: isn't it possible for more people to (temporarily) join the anti-tech-debt special task force? Of course the motivation being that I think this is more important than continuously acquiring new features, which you disagree with.

16

For what it's worth, I think most people just use their github logins for IRLO/URLO (though... I don't use URLO, but that's just because I haven't gotten around to pressing the "link" button =P).

17

First off, in the space of molding GitHub to better support Rust RFC's use of GitHub issues, I think this is a worthy entry! Keeping space reserved for summaries and slowing down the pace to force people to contribute summaries seems like an interesting way of iterating on some of the weakest points of the current process.

On the other hand, I'd like to respond to this:

I very much disagree with your arguments here, to wit:

  • Discourse already supports GitHub authentication, so the extra work for maintaining an account is minimal.
  • Discourse also allows cross-linking easily, so this doesn't seem a differentiating feature.

But mostly, I would argue that what RFC discussions need is actually more conversation style than what GitHub offers, and so where GitHub optimizes for what I would maybe call an exchange of facts (in issues or PRs), RFCs need something closer to exchanges of ideas. In some ways, by your own admission of advocating for table of contents/summaries as an important addition, it seems you are arguing (and I would agree) that perhaps the more important feature of RFC discussion is not the record keeping, but rather the establishment of a shared understanding of the design space for a given RFC.

18

We just disagree; I think we can manage the current amount, and if we cannot manage, designs just don't get accepted. The RFC process is therefore self-adaptive in the sense that it is rate limited by T-Lang's bandwidth. We also prioritize designs based on size and what we believe is most needed right now. At least that is what I usually do when trying to nominate things for the agenda.

Aside: I've been trying hard to stabilize things the last month; I've looked at what can be stabilized and proposed stabilization for those; for all the rest it appeared to me that they were blocked on implementation work or open design questions.

I just feel that this already happens; but our time-frames around how long "shepherd"ing may take may vary. Along the way some proposals just get blocked by design concerns that were either unforeseeable during the RFC's discussion or we thought we had a solution but it turned out it wasn't good enough. Specialization and associated type defaults are examples of this. const fn is another example, it was blocked a long time until I, Oliver and Eduard came to the realization that we should stabilize a smaller subset.

I agree with your point that some team member should be assigned to every RFC. I cannot speak for other teams such as T-Libs or so, but my point is that for T-Lang this already happens on T-Lang meetings to the extent we have time; sometimes it may take a week or two to assign a member, but I think that's fine.

Yep.

Well... The thing is that the foundational work on Chalk enables not just GATs, but from my understanding it also enables for<T: Debug> (and by extension generic closures), possibly where clauses on enum variants, fixing type aliases, getting inherent associated types, quantification over traits and associated traits, ...

I would also say that something like for<T: Debug> is a large additive change from a user-end perspective, but once you have Chalk it's not that big implementation wise I think. In lieu of already having for<'a> it could also be seen as fixing an inconsistency.

Sure, but just because they get proposed does not mean they will be accepted.

Doesn't seem additive to me; seems like it fixes a syntactic inconsistency; mostly a bug-fix imo. As you said it hasn't been accepted yet.

I don't want to rule out anything (because I don't want to prematurely shut down debate) but all except the first of these seem unlikely to be accepted.

Sure, there was controversy, and for arg: impl Trait it was primarily after stabilizing; however, I don't agree that insufficient design time was given to these or that T-Lang made a decision without thinking it through. These both seem like decisions T-Lang made in full view of the trade-offs.

Sure; I understand, I just disagree. Unlike some people who have been arguing that "Rust is complete", I think there are substantial areas where it is incomplete. A language feature like for<T: Debug> or generic closures is a good example imo.

Sure, my point is just that things like associated type defaults, specialization, or multi-trait-objects are not blocked on implementation work, but by design which require RFCs and reaching consensus. Thus it's not a matter of finishing.

I agree that Box<dyn Read + Write> is a natural thing to want, but the design work needs to be done by someone, an RFC needs to be filed, and T-Lang needs to agree before implementation work can begin. My main point here is that someone needs to do the work, and if you can do it, that is great.

I also think that for the people proposing new ways of writing loops or overloading, they probably have the same feeling as you that "it's something quite natural to want IMO". I don't want to prejudge in what areas proposals may and may not be filed.

I think so. For example, one could be forgiven for thinking that default { ... } is a massive change to the language. Indeed, I am considering scaling back the RFC to make more incremental progress... More on this later!

:+1:

Not only does it not increase linearly with the number of people working on them, we also have bors in the way making sure that PRs don't get merged as quickly as we'd like. Even our CI infrastructure might not be able to cope.

Another problem is that not everyone is knowledgeable in the area of compiler development. for example, I'm not a compiler dev... I have been trying to implement one of my own RFCs, but it takes time to learn the compiler's infrastructure.

I don't see accepting features as an end in itself, I just think there are many areas where the language is incomplete and needs improvement to need the needs of people using it. Language design is one of the areas I feel I am most competent in (as opposed to working on rustc), so that is where I try to help out.

19

It's a mixed bag; T-Lang RFCs are both about ideas, but also about the technical details, especially in the reference sections. I think the way we use GitHub can be adjusted and tweaked to fit the needs of the RFC process, but we need some work on bots, and someone needs to do that work.

20

I concur that Github is better suited to this than Discord, mostly for the edit-ability and keeping things up to date, along with the inline file support (well, commenting on commit files…). Although getting to conclusions takes a lot of discussion most of the time consuming or reviewing an RFC you want the structured conclusions and easy visibility of the reasoning behind such things. The conversations are the unorganized organic source. I think the we should prioritize reading - write once, read a ton.

I’ll go out on a limb and say while I think GitHub is better than Discord, I think it’s okay, not great. We can shore it up with a bot, and shore it up more with writing special tools to stick in the appropriate metadata in a well formed state. Part of me wonders though it it wouldn’t just be better to write a separate tool and just store data in GitHub as the backend (because of the wonderful linking / file preview support / being able to stick other things as well in there). But maybe that’s what you mean the bot suggestion.