Two-stage RFCs: motivation+overview, then details

1

Hello all,

In his epic blog post on the RFC process, @aturon described some challenges in the current RFC process. In particular, when a new RFC appears today, it couples together a couple of things:

  • a description of the problem(s) to be solved;
  • a high-level summary of the idea;
  • detailed description.

This often results in large, sprawling threads. The thread can easily get diverted on a small detail of the implementation, often even before it’s been established whether the problem to be solved is really a problem, or if the approach makes basic sense. @aturon proposed address this by introducing more explicit notions of RFC staging; later, we discussed his specific proposal in this internals thread. Unfortunately, that proposal was perhaps a bit ambitious, and the thread never reached a firm conclusion.

In this thread, I want to propose a more limited version of this staging idea. The high-level concept is simple: Landing an RFC will become a two-step process: First, you open a PR with a motivation; if that is approved, you open a second PR with the details.

To get a bit more specific:

  • Step 1. Open a PR with RFC text that primarily describes your motivation. In other words, what problem(s) is this proposal solving?
    • The RFC should include a high-level summary of how the solution works, but it should skip the “detailed design” section.
    • Summarizing alternatives is highly recommended.
    • For this first PR, discussion in the thread will focus on two questions:
      • is this problem worth solving right now? (this can reference the roadmap for the year, for example, to help determine prioritization)
      • is this the general shape of the solution that we want?
    • If the answer is yes to both of those questions, then the RFC will be merged into a pending directory (e.g., pending/12345-my-wonderful-rfc.md).
  • Step 2. Open a second PR that amends that pending/12345-my-wonderful-rfc.md file with a detailed design.
    • For this second PR, discussion will focus on the details of the design.
    • It may be that in the course of this discussion, we decide to revisit the motivation or basic solution – that’s ok. The original RFC text is not frozen in stone or anything like that.
    • Once everyone is happy with the design, we will merge the PR and move the file to text/12345-my-wonderful-rfc.md.
    • Of course, if the idea doesn’t seem to be working out, we may also close the PR. At this point, we can either remove the pending file (if we decided against the idea overall), or else take another stab at a different solution.

Note that this is intentionally a “maximally minimal” proposal that just has the goal of ensuring that every RFC has a firm (and well-fleshed out) motivation. I think that there are further extensions we might consider, but it makes sense to take things a step at a time.

Some “FAQ”:

Why include any solution at all?

In the past, we’ve talked about separating out a “pure motivation” from any outline of the solution at all. But whenever we’ve tried to imagine what this would look like in practice we got stuck. It feels like the motivation and overview is often heavily influenced by the solution you have in mind. So we decided to modify slightly to include an overview of the solution as part of the initial PR.

It seems like there is also some use to having a mechanism to discuss whether it is even worth trying to solve a given problem now. I’m not sure if RFCs are that mechanism – perhaps internals threads are better?

BTW, we may want to modify the template to include a section for “high-level overview”, though my inclination is just to include that material in the “motivation” section – precisely for the reasons outlined here. It’s often natural to discuss the problem and bits of the solution adjacent to one another.

Why have two PRs?

Good question. It’s not clear that this is the right thing. We could instead just have some labels that we move through to indicate the “stage” of discussion. However, it does seem nice to have multiple PRs:

  • it helps to frame the two discussions, making it easier to revisit and re-read (GH doesn’t scale well as discussions get longer);
  • it means that whenever we reach a decision, we follow a similar chain of events:
    • move to merge/close (via rfcbot)
    • have a final comment period
    • close / merge the PR

That said, I’d love to hash this out here! I like for example the idea of having just one PR, but not having discussion on the PR itself. Instead, we open threads on internals and link to them from the PR (along with summaries and other key points). The PR could be closed for comments to enforce this.

What about summarizing threads?

When threads get really long, it’s hard to catch up. In the past we’ve talked about the importance of summaries. I think this problem is somewhat orthogonal, though adopting techniques like moving threads onto internals (and putting only summaries in the RFC PR itself) may help. We’ve also been talking about a very simple step: adding a bot that tags RFCs that have had a lot of comments without a summary as summary-needed, and thus encouraging people to go through and summarize what has been said. They can then say @rfcbot summarize ... in order to tell the RFC bot that this is a summary post. This also makes it possible to link to those summaries and so forth. It may be worth pulling this proposal itself into a separate thread.

ping @dikaiosune

What about abandoned, multi-author, or very complex RFCs?

One limitation of the current RFC process is that every RFC has a single “author” (the person who opened the pull request). In practice, a lot of RFCs go through iteration and incorporate suggestions from a lot of people. If the author is responsive, this is generally fine – but the author can easily become a bottleneck, either because they’ve drifted onto other things or because they just don’t have enough time to keep up. GitHub’s new ability to support more collaborate PRs helps here. In general, though, we decided not to try to address this yet.

What should we do with existing RFCs?

They get grandfathered in under the old process, I guess – though in some cases, it might make sense to close out an RFC and re-introduce it using this process, particularly if we’d like to focus on the motivation.

Ok, enough from me! Thoughts?

12 Likes
2

cc @eddyb @scott @brson @pnkfelix @nrc @withoutboats @aturon @wycats @josh – all people involved in the design and/or conversation in the past. Sorry if I left anyone out. =)

3

This effectively seems like turning the pre-RFC convention into a defined part of the RFC process, which seems like an excellent idea. That first PR sounds exactly like a pre-RFC.

4

I think this is definitely feasible to include in rfcbot. Fair warning: I'm pretty behind on rfcbot things owing to a few factors, so I'm not sure what the turnaround would look like.

My two cents: even with the bot's help I would imagine that summaries of the thread will be useful but probably not enough to replace some kind of scope-reduction strategy (i.e. the two PRs you've proposed here).

5

Yes, to a large extent. I think that part of the goal here should also be that we can better manage expectations -- in particular, once an RFC passes the first level, we can hopefully have quicker turnaround for finishing out the detailed version. (That is, there will be fewer things in that area.)

6

What I like about this proposal is that it lowers the bar for contributing an RFC about "what we should maybe do" - without the author needing to have the know-how or free time to do a full detailed design. And then, once we have a set of accepted motivation RFCs, others have a stepping off point for proposing detailed designs for features that already have general acceptance, so they don't have to deal with two problems at once.

I agree. I also see the motivation RFCs as an easier way to make progress on some of the non-PR issues on the rfcs repo, which tend to just sit around.

2 Likes
7

Speaking of which, aren't the issues in the rfcs repo already kind of like the "stage 1" RFCs proposed here?

2 Likes
8

If both the motivation stage and the design stage both happen in the same repo I would like all PRs to be tagged as one or the other.

I’m worried that this change might actually compound the problem it is trying to solve. While it is disinviting half-baked and unwanted RFCs it is actually inviting new discussion of pre-RFC ideas into the official RFC process, in the same forum as the RFCs. A big part of the existing problem is that so many RFCs are simply not worth devoting precious time to because they are poorly-conceived, underbaked, etc. Presumably it will be easier to evaluate and turn away these small RFCs before they get too far, but they’ve still got to be evaluated by somebody with the authority to say yes or no. There may end up being more decisions to be made, not fewer.

9

2c from a Discourse team member: Perhaps (1) would be better served by an Internals discussion? I'm merely a casual observer of Rust so I don't have a complete picture of your process, but Step 1 strikes me as primarily conversational, which is specifically the type of interaction Discourse is built for. Also:

  • We now support Webhooks which could facilitate all sorts of nifty GitHub integrations.

  • We support GitHub linkbacks.

  • Although not perfect, our automated summarizations of topics can help skim-readers get reasonably up to speed. I think manual recaps - which @dikaiosune has set the gold standard for - is still an invaluable resource that can't be automated, but would at least always show up in these summaries thanks to a high like count.

  • I've previously specced out a plugin proposal for "Manually curated topic summaries". This isn't something the Discourse team is likely to work on, but if someone from the Rust/Discourse community makes it there's a good chance we'll support it.

  • Similarly, other custom workflow features like for example RFC voting might make their way into Discourse if such a feature is applicable to software development forums in general.

I welcome all Rust enthusiasts to engage with us in #feature on Meta to make a case for more software development-oriented features in Discourse. And we welcome the core Rust team to talk to us directly if there's anything we can do together in this space as a joint effort.

7 Likes
10

Yes, this is certainly possible. I guess we have to be clearer, perhaps, about what we aim to achieve. I think I am possibly holding two contradictory goals in my head at once.

I think what I want are balanced RFCs. That is, I want to have lots of good details worked out, but I want to first discuss the higher-level questions: Do we want this feature? What will we use it for? Is there a problem that needs solving? And is this the right time to solve it? (After all, even if there are people motivated to do the coding, reviewing, thinking, and incorporating RFCs is a non-trivial time investment on the part of subteam members.)

I can certainly see though that having a separate PR for the details encourages people to open the initial PR earlier. This seems overall like a good thing, but it will likely mean more PRs, as you predict.

An alternative might be to have people provide the full RFC, but to have "stages" of discussion. Maybe we can make these clearer. For example, I could imagine a bot that, whenever an RFC PR is opened, immediately:

  1. Opens an internals thread labeled "Motivation for RFC PR #123: XXX"
    • this thread can have an introductory post clearly spelling out what we intend to discuss there
  2. Locks the original PR and adds a comment like
    • "Thanks for the RFC! A thread has been opened on internals for the discussion of the motivation."

Then lang-team members can later leave a comment (does this require unlocking?) or otherwise direct the bot that the motivation has been agreed upon, in which case it will open a thread for discussing internals -- and maybe at this point we can have ways to branch more threads, helping to keep the conversation organized (summary posts might also be reflected in the PR itself).

This idea is rather more ambitious in terms of coding support required to make it happen, and rather more invasive to our process, though. Probably too much of both.

2 Likes
11

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.