I’ve recently written a blog post enumerating several ideas for improving Rust’s RFC process. The post contains three separate (but interdependent) proposal sketches, which I’m eager for feedback and discussion on. I’m carving out each proposal as a standalone thread here on internals.
At the heart of Rust’s open development is the RFC process. Every major change to the language, compiler, core libraries, tooling, and policy go through an RFC writeup and consensus-building process. The process served us incredibly well in clarifying our technical direction on the road to 1.0, and has continued to be highly active since then, with on average about 2 RFCs merged every week.
But it’s not all roses. There’s been a growing sense among both Rust leadership and the broader community that the RFC process needs some further refinement as we continue to grow the community. I want to lay out my view of the problems and sketch some possible solutions, based on extensive discussion and brainstorming with many others on the team.
Each idea operates at a different scale (from big-picture to low-level mechanics), but they are intended to fit together into a whole; each one supports the others. Ultimately, these should result in a single RFC, but in the meantime I’ll start a discuss thread for each proposal.
There is a clear common theme to all of the problems I want to raise: communication. We need to find ways to better scale up lines of communication around the RFC process, and for Rust core development in general. There is also a cross-cutting concern: a need to increase our focus on mentoring and the path to team membership. @wycats has a great saying about measuring the health of the team structure:
Being a very active contributor who is not yet on a subteam should feel very close to actually being on that subteam.
Shooting for such a state of affairs has many benefits, not least of which is increasing the scalability of our community.
This thread is for the “RFC staging” part of the proposal.
Proposal: RFC staging
RFCs are hard to keep up with in part because reading a full design – and all the commentary around it – can be a lot of work, and there tend to be a large number of active RFCs in flight at any time. RFC discussions are often hard to follow, due to the overwhelming number of comments, sometimes stretching over multiple forums. Naturally, this problem is exacerbated by "controversial" RFCs, which is where we most need broad input and careful discussion. It can also be hard to track RFCs that are in some sense “competing” (offering alternative proposals for a common problem), or to correlate discussion between the discuss forum and github.
It’s also problematic to start off with a full proposal. What we really want is to get the community on the same page first about the importance of the problem being solved, and then to proceed to the design phase, perhaps considering multiple competing designs.
Finally, RFCs are sometimes closed as “postponed”, but ideally that should not simply terminate the discussion; instead, the discussion should simply continue elsewhere, or somehow be marked as being at a different stage.
Idea: introduce stages into the RFC process, including one for reaching consensus on motivation prior to considering a design.
Idea: move the focus away from an RFC PR as the primary venue for RFC discussion.
Put differently, the idea is to orient the RFC process around problems first, and solutions second.
The rough phases I have in mind are:
- Problem consensus
- RFC drafting
- RFC PR(s)
- RFC merged
Concretely, what this would look like is having some venue for tracking the problems we might want to solve, perhaps a revamped version of the RFC issue tracker. Whatever this venue is, it would track the progression through all of the phases. Let’s call this venue the “Problem Tracker”.
Phase 1: Problem consensus. The initial discussion is essentially about reaching consensus on the motivation section of an RFC, which should include examples and make a compelling case that solving the problem is important enough to warrant expending energy and potential complexity. The subteam would sign off on that motivation, at which point there is some level of commitment to solve the problem. That puts the focus where it should be – solving problems – and should make it much easier for subteam members to engage early on in the RFC lifecycle.
Phase 2: RFC drafting. This phase can proceed in parallel with the previous one. During this phase, people sketch designs and work toward one or more full RFC drafts. Brainstorming and discussion on specific drafts would happen within dedicated “pre-RFC” discuss posts, which are linked from the Problem Tracker. In particular, newly-opened RFC PRs today often get an avalanche of comments and early revisions, making it very hard to join the discussion even a week later. Pushing early feedback to our forum instead will make the eventual RFC PR discussion more focused and easier to participate in.
Phase 3: RFC PR(s). At some point, a shepherd (see below) can determine that an RFC draft is of sufficiently high quality and steady state that a PR should be opened, at which point discussion proceeds as it does today. Multiple RFC PRs might be open for the same basic problem – and indeed, this is a good way to take the “Alternatives” section more seriously. All open PRs would be linked from the Problem Tracker.
Phases 4 and 5 work just as today.
One interesting aspect of this phasing: it’s possible to approve a Motivation section, then get all the way to RFC PR, only to close out the PR for one reason or another. In such cases, it should be possible to go back to the Problem Tracker and build a new, alternative RFC draft with the same Motivation section.
Note that, in this regime, you don’t ever open an RFC PR out of hand – it must go through the earlier phases, including the pre-RFC discuss post. While this may feel like more process, I think that globally it will make the whole thing more efficient, by weeding out poorly motivated RFCs earlier, by focusing attention on the problem, by producing higher quality RFC PRs, and (as we’ll see) by decentralizing the process a bit more. In addition, it makes it easier to cope with the problem of “Does this need an RFC?”
As part of this proposal, I think we should “reboot” the notion of a shepherd. The idea would be to create a broader network of people around a subteam who are empowered to help move the RFC process along in various ways, but aren’t necessarily responsible for the final decision. So, for example, we would have a larger set of “lang shepherds” who help lang RFCs progress. The powers and responsibilities of shepherds would include:
“Calling to question” – that is, proposing that the subteam move to make a decision on problem consensus or moving to FCP.
Working with the community to help brainstorm, draft, and revise pre-RFCs.
Moving to from pre-RFC to RFC PR phase.
Acting as the “scribe” for the RFC process, by keeping the Problem Tracker up to date. In particular, the subteams currently attempt to provide "summary" comments for contentious RFCs, to help people track the discussion. This proposal would give those comments more formal status, as something that would go directly on the Problem Tracker, and that any shepherd could provide at any point.
All subteam members can act as shepherds as well.
In general, I envision the Problem Tracker as the go-to place to see where things stand for a given problem/set of proposals, including summarization of discussion and pros/cons for the proposals. The shepherds would play a special role in establishing that official record.
I think these changes make the RFC process both more accessible and more scalable. More accessible because it’s easier to get involved and get quick feedback in lightweight ways (before writing up an entire design). More scalable because of increased parallelism, and because the big decision points happen at either an easier stage (establishing motivation) or with many fewer proposals in flight (the RFC PR stage).
Details to be worked out:
- What happens to current RFC PRs? Are they grandfathered in, or moved into this new process?
- Where does the “problem tracker” live?
- What are good guidelines around an initial “motivation”?
- How and where can we keep an “official record” of the progression of a problem, including links to (and summaries of) pre-RFC and RFC PR threads?