In his epic blog post on the RFC process, @aturon described some challenges in the current RFC process. In particular, when a new RFC appears today, it couples together a couple of things:
- a description of the problem(s) to be solved;
- a high-level summary of the idea;
- detailed description.
This often results in large, sprawling threads. The thread can easily get diverted on a small detail of the implementation, often even before it's been established whether the problem to be solved is really a problem, or if the approach makes basic sense. @aturon proposed address this by introducing more explicit notions of RFC staging; later, we discussed his specific proposal in this internals thread. Unfortunately, that proposal was perhaps a bit ambitious, and the thread never reached a firm conclusion.
In this thread, I want to propose a more limited version of this staging idea. The high-level concept is simple: Landing an RFC will become a two-step process: First, you open a PR with a motivation; if that is approved, you open a second PR with the details.
To get a bit more specific:
- Step 1. Open a PR with RFC text that primarily describes your motivation. In other words, what problem(s) is this proposal solving?
- The RFC should include a high-level summary of how the solution works, but it should skip the "detailed design" section.
- Summarizing alternatives is highly recommended.
- For this first PR, discussion in the thread will focus on two questions:
- is this problem worth solving right now? (this can reference the roadmap for the year, for example, to help determine prioritization)
- is this the general shape of the solution that we want?
- If the answer is yes to both of those questions, then the RFC will be merged into a
pending directory (e.g.,
- Step 2. Open a second PR that amends that
pending/12345-my-wonderful-rfc.md file with a detailed design.
- For this second PR, discussion will focus on the details of the design.
- It may be that in the course of this discussion, we decide to revisit the motivation or basic solution -- that's ok. The original RFC text is not frozen in stone or anything like that.
- Once everyone is happy with the design, we will merge the PR and move the file to
- Of course, if the idea doesn't seem to be working out, we may also close the PR. At this point, we can either remove the
pending file (if we decided against the idea overall), or else take another stab at a different solution.
Note that this is intentionally a "maximally minimal" proposal that just has the goal of ensuring that every RFC has a firm (and well-fleshed out) motivation. I think that there are further extensions we might consider, but it makes sense to take things a step at a time.
Why include any solution at all?
In the past, we've talked about separating out a "pure motivation" from any outline of the solution at all. But whenever we've tried to imagine what this would look like in practice we got stuck. It feels like the motivation and overview is often heavily influenced by the solution you have in mind. So we decided to modify slightly to include an overview of the solution as part of the initial PR.
It seems like there is also some use to having a mechanism to discuss whether it is even worth trying to solve a given problem now. I'm not sure if RFCs are that mechanism -- perhaps internals threads are better?
BTW, we may want to modify the template to include a section for "high-level overview", though my inclination is just to include that material in the "motivation" section -- precisely for the reasons outlined here. It's often natural to discuss the problem and bits of the solution adjacent to one another.
Why have two PRs?
Good question. It's not clear that this is the right thing. We could instead just have some labels that we move through to indicate the "stage" of discussion. However, it does seem nice to have multiple PRs:
- it helps to frame the two discussions, making it easier to revisit and re-read (GH doesn't scale well as discussions get longer);
- it means that whenever we reach a decision, we follow a similar chain of events:
- move to merge/close (via rfcbot)
- have a final comment period
- close / merge the PR
That said, I'd love to hash this out here! I like for example the idea of having just one PR, but not having discussion on the PR itself. Instead, we open threads on internals and link to them from the PR (along with summaries and other key points). The PR could be closed for comments to enforce this.
What about summarizing threads?
When threads get really long, it's hard to catch up. In the past we've talked about the importance of summaries. I think this problem is somewhat orthogonal, though adopting techniques like moving threads onto internals (and putting only summaries in the RFC PR itself) may help. We've also been talking about a very simple step: adding a bot that tags RFCs that have had a lot of comments without a summary as
summary-needed, and thus encouraging people to go through and summarize what has been said. They can then say
@rfcbot summarize ... in order to tell the RFC bot that this is a summary post. This also makes it possible to link to those summaries and so forth. It may be worth pulling this proposal itself into a separate thread.
What about abandoned, multi-author, or very complex RFCs?
One limitation of the current RFC process is that every RFC has a single "author" (the person who opened the pull request). In practice, a lot of RFCs go through iteration and incorporate suggestions from a lot of people. If the author is responsive, this is generally fine -- but the author can easily become a bottleneck, either because they've drifted onto other things or because they just don't have enough time to keep up. GitHub's new ability to support more collaborate PRs helps here. In general, though, we decided not to try to address this yet.
What should we do with existing RFCs?
They get grandfathered in under the old process, I guess -- though in some cases, it might make sense to close out an RFC and re-introduce it using this process, particularly if we'd like to focus on the motivation.
Ok, enough from me! Thoughts?