[blog post] Proposal for a staged RFC Process

One of the goals with this proposal is to create sort of a "funnel" of involvement by the teams. The initial stage represents zero team involvement; anyone can start a repo. But to move onto the next stage, someone from the team has to sign off on it: that's your champion. But it only requires one team member. Later stages require the entire team.

So it's not so much "speak for itself" as it is making sure that, as things progress, they fit into the team's goals, while also not bogging every single step of the way down in requiring all team members to sign off immediately.

As far as finding a champion goes, I imagine this is a good use of the team email aliases; once you think you're ready to move from spitballing to designing, you email the team and ask for a champion. If nobody on the team wants to sign off, that's a good sign that this either doesn't fit into the roadmap, needs to wait until someone's time is freed up (which isn't inherently a bad thing; it's part of prioritization), or something else. Ideally this is actual feedback given to the RFC author.

That said, that's just what I'm imagining; you're right that this is an important aspect of the whole thing, and so we should have a good story sorted out here.

3 Likes

You may have looked at this already, but in case you haven’t and for the sake of others reading this who haven’t—

The “prior art” here in the TC39 world actually works really well. Some of them are indeed small, and it’s a slightly too-high level of overhead, but it shines for more complex language design issues—not least because you can open discussion issues for different parts of the design rather than it all having to occur in a single thread. A prime example of that is the discussion level on some of the pipe operator and partial application proposals there. There is a lot happening in those proposals, and they’d be nearly impossible to manage as a single thread; as a repo it’s much more comprehensible.

It also makes it much easier to tune your level of engagement with the process in my experience (all as an interested observer, to date).

I do agree we’ll need some tooling improvements to support the change, but I strongly recommend to anyone who hasn’t done it yet to spend some time looking at the various TC39 proposal repositories.

5 Likes

It may be more clear to present the "team decision points" as gates or transitions rather than phases. In terms of state machines, they act more like inputs or events than states. This also lends itself naturally to describing the "off ramps" mentioned by @chriskrycho and @lambda

This seems like a natural fit with the role of the champion, as I envision it. The champion already serves as a gatekeeper. One of their responsibilities should be to evaluate the acceptance criteria of the RFC, which includes cohesion. As the conduit between the RFC author(s) and the related Rust team, they would also be responsible for ensuring feedback related to the central vision finds its way back to the RFC.


Similar feedback on reddit


This could be built in to the web interface proposed on the blog:

The repositories will live in the rust-rfcs organization. There will be a convenient webpage for creating them; it will create a repo that has an appropriate template and which is owned by the appropriate Rust team, with the creator also having full permissions.

I imagine there would also be a way to provide feedback via the main rfc repo PR via rfcbot.

1 Like

I agree that this feels overly focused on the giant RFCs which receive hundreds or thousands of comments of discussion. Most RFCs don't fall into that category, but they do stick out in memory. RFC per repo sounds like a reasonable idea, but it sounds like it's mostly just an attempt to solve a need for threaded discussion to me. From the comments here it sounds like it hasn't worked out as well as we'd like on RFCs that have tried it in the past.

I don't like the forced spitballing phase, personally. I'd like to see it made more explicit how smaller RFCs would skip through that period. For the majority of the RFCs I've written, I've not felt a need to write a pre-RFC (2 of them would have benefitted from that phase for sure). This feels like it should be left up to the author, rather than requiring team approval to move past it.

I am really torn on the entire idea around champions in general. I very much understand the issue of prioritization and making sure there is bandwidth available. One thing that is not clear to me from this proposal is whether the champions (who seem like they're essentially just what shepherds are/were supposed to be today) are required to be on the teams implementing the feature or not. To put it another way, is the problem we're trying to solve related to bandwidth of reviewing RFCs or bandwidth of implementing?

My biggest concern with requiring a champion is that it unfairly benefits those who have close relations with folks on the teams in question. Arguably those folks are most likely to write RFCs that could be accepted, but this feels like an arbitrary barrier that we should not be introducing.

You touched several times on the need to grow the team, but I don't think it's been given enough attention. Honestly I think that is the main problem we should be focusing on, rather than the RFC process itself. The biggest issue I see is that there's no clear path to start taking on these sorts of roles beyond getting involved in the implementation.

As you know, I personally tried to get more involved recently -- And while it was clear I didn't have the bandwidth to fully understand how the trait system is implemented and meaningfully contribute there, I'm certainly still qualified to help review RFCs, discuss them with the lang team, etc (and likely have bandwidth in that regard). But there's no clear path for me to do so. I suspect there are many out there like me in that regard.

5 Likes

One thing which worries me in “one repo” approach is that it becomes harder to write an RFC review and read them in one place. Yes, blogpost leaves place for summary FCP comments from the team, but I don’t think that it’s enough. For example when I read RFCs I often just briefly read motivation and summary and after that skim through comments in search for opinions and highlights of RFC’s weak/strong points. This is why here I’ve proposed to limit top-level RFC comments by “review” format.

We could add “reviews” issue, but how should reviews be discussed then? We either make this issue review-only (then we can’t see replies to it) or thread again becomes filled with noisy discussions. Or every review can go to the separate issue marked by tag, but then it becomes harder to read them in sequence.

The RFC process, especially the early parts, is inherently chatty. GitHub excels at reviewing and pushing code, but that doesn't mean a discussion about code that might one day materialise ought to live there from day 1. I've suggested using Discourse to a larger extent in the Rust RFC process before. Since it received some quiet approval and no pushback I hope you don't mind if I bring it up again.

I really like the flow of the new process, but I'm not a fan of the separate repository per-RFC proposal. By committing hard to GitHub so early in the discussion process you're setting yourself up for a lot of busywork like keeping track of all these disparate repos via some mix of manual curation and GitHub API hacks. Good points have been raised about how this workflow is primarily designed for "epics" rather than minor RFCs that are adequately covered in a single topic of discussion.

Here's how I would structure the early parts of the RFC process in a way that scales from small to big RFCs with minimal friction:

Spitballing (Optional)

  • Create Pre-RFC topics on internals.rust-lang.org with a #pre-rfc tag to casually discuss any idea you might have. Minimal formalities.

Designing / Prototyping

  • Create an RFC in the #rfc category. (Any related pre-RFC should be closed with a link to this RFC).

    • Every RFC topic must have a descriptor tag attached to it, like #semicolons-as-items. Discourse allows you to require this.

    • Branching discussions about a complex RFC can easily be created by starting a linked topic.

      These new topics will automatically have the appropriate tag attached to them.

    • For extremely complex/controversial RFCs you might consider creating a whole new sub-category for discussion, e.g. #rfc:rust-epochs.

All of this categorisation is completely fluid and won't create any breaking links when changed.

Implementing

  • The bulk of the discussion moves to GitHub as code starts to be written.

Discourse will cover this part of the discussion process:

The way I see it, keeping everything in one place is a pipe dream. Before an RFC comes into existence there will always be informal discussions happening that can live just about anywhere. Better to use different platforms best suited for the different stages of discussion.

I see the stages of an RFC more like:

  1. Fragmented discussion (chat, forums, reddit, blogs)
  2. Centralised discussion (#rfc category)
  3. Code proposals & review (GitHub PRs)
  4. FCP (final GitHub PR)

During the early discussion-heavy phase you want to facilitate discussion as best you can, and Discourse does discussion much better than GitHub.

  • Split out or merge in posts
  • Moderate users and their posts
  • Easily quote or reply to specific posts
  • Granular notifications
  • Duplicate content warnings

And the open source nature of Discourse allows you to better formalise the RFC process, complete with UI conveniences, e.g.

  • Allow moderators to highlight certain posts, e.g. marking a post as "summary", which would be clearly called out in the sticky page counter on the right.
  • Visually highlight the posts of an RFC's champion, if one exists.

A lot of these are feature we'd be happy to sponsor.

13 Likes

This is the key point. The mechanisms on Discourse are far superior to what is available on GitHUB.

7 Likes

These are features GitHub sorely lacks. It's also been pointed out that github thread collapsing often hides valuable feedback.

As a thread gets long, it typically starts to fragment. This is where splitting and merging would be useful, but that requires active management of the thread. As far as I've seen, this is not typical of the internals forum. Wouldn't this increase the burden of the moderation team? Are there any tools provided by Discourse that would allow the "Pre-RFC" author to prune the threads?

One benefit of the one-repo-per-RFC approach is it groups all conversations related to a feature. The try-related discussions (try fn, throw, pass, and various bikesheds), for example, would all live under one roof. What features does discourse have to facilitate this that are available to any member? It doesn't appear, for example, that #hastags work unless a category is set up, and I doubt we'd like to add a category for each "umbrella" RFC.

3 Likes

As a proposal to change the process so much, it is lacking Prior Art other than TC39 (I get that this is a draft :smile:).

Most other languages do not use an entire new GitHub repository for a new proposal. TC39 is a rare exception. Some projects like Elixir, TypeScript and Kotlin don’t even have a formal process.

Swift uses Discourse as the RFC discussion medium, similar to the suggestion above.

  1. Author creates a Pitch (Pre-RFC) on the forum.
  2. Author submit the proposal document to the apple/swift-evolution repository with a link to the Pitch.
  3. Review manager merge the PR, enqueuing it to the review queue.
  4. Review manager creates a Proposal Review (RFC) on the forum which lasts for ~1 week.

The D process is also interesting in how it can forces a summary to long discussions by timeboxing:

  1. Author submits a draft to the dlang/DIPs repository.
  2. DIP manager merge the PR.
  3. DIP manager creates “Community Review Round N” thread on the forum which expires after 15 days.
  4. (Repeat step 3 until steady state)
  5. DIP manager creates “Final Review” thread which is mutually-exclusive and expires after 15 days.

(Note that unlike Swift or D, Rust has an additional stabilization phase which we need to account for.)

Comparison of feature proposal process among major languages with explicit proposal process

C++

Ref: https://isocpp.org/std/submit-a-proposal

Stage Discussion medium
Float Newsgroup at std-proposals
Draft Newsgroup at std-proposals
Submit Email to WG21 LWG
Face-to-face meeting

Changes to language will be exclusively relying on face-to-face meeting.

C#

Ref: https://github.com/dotnet/csharplang

Stage Discussion medium
Discussion GitHub issue on dotnet/csharplang labeled Discussion
Proposal GitHub issue on dotnet/csharplang labeled Proposal champion

The remaining stages are driven primarily within the C# Language Design Meetings (LDM).

D (DIP)

Ref: https://github.com/dlang/DIPs/blob/master/PROCEDURE.md

Stage Discussion medium
Point of Contact -
Development Stage (optional) -
Draft Review GitHub PR to dlang/DIPs
Community Review Forum https://forum.dlang.org/group/general
Final Review Forum
Formal Assessment Private GitHub gist (final edits)

ECMAScript (TC39)

Ref: https://github.com/tc39/ecma262/blob/master/CONTRIBUTING.md#new-feature-proposals

TC39 will hold a face-to-face meeting every 2 months to promote or demote a proposal’s stage.

Stage Discussion medium
0 (Strawman) Mailing list, IRC
1 (Proposal) New GitHub repository (can be hosted anywhere)
2 (Draft) (the repository above)
3 (Candidate) (the repository above)
4 (Finished) (the repository above) + GitHub PR to tc39/ecma262

Go

Design doc are submited as Gerrit codereview mail to https://go.googlesource.com/proposal. A final discussion at end of design doc stage will decide whether to accept or reject the proposal.

Stage Discussion medium
Proposal GitHub issue on golang/go
Design doc (same issue as above)

Haskell (GHC proposal)

Ref: https://github.com/ghc-proposals/ghc-proposals

Stage Discussion medium
Community discussion GitHub PR to ghc-proposals/ghc-proposals
Committee process Mailing list ghc-steering-committee

Java (JSR)

Ref: https://jcp.org/en/resources/guide

No thanks.

Python (PEP)

Ref: https://www.python.org/dev/peps/pep-0001/

The core developers (“BDFL” and “PEP czars”) will approve, reject or defer a PEP at the end of a review.

Stage Discussion medium
Presentation Mailing list python-ideas
Draft GitHub PR or direct commit to python/peps (acquire PEP number)
Discussion (anywhere; authors are encouraged to set up SIG mailing list or wiki pages for large PEPs)
Review Usually mailing list python-dev

Rust 2015 (RFC)

An FCP vote will be raised directly in the GitHub PR/issue to let team members give consensus whether to advance to next stage.

Stage Discussion medium
0 (Pre-RFC) (optional) Discourse https://internals.rust-lang.org/
1 (RFC) GitHub PR to rust-lang/rfcs
2 (Tracking) GitHub issue on rust-lang/rust
3 (Stabilized) (the issue above)

Rust 2018 (Staged RFC i.e. this proposal)

The team champion will open GitHub PR on rust-rfcs/rfcs to update the index; FCP will be raised there.

Stage Discussion medium
0 (:bulb: Spitballing) New GitHub repository under rust-rfcs/
1 (✎ Designing) (the repository above?)
2 (:hammer_and_wrench: Implementing) (the repository above?) + GitHub PR to rust-lang/rust
3 (:microscope: Evaluating) (the repository above?)
4 (:100: Stabilized) (the repository above?)

Swift (SE)

Ref: https://github.com/apple/swift-evolution/blob/master/process.md

Stage Discussion medium
Pitch Discourse https://forums.swift.org/c/evolution/pitches
Proposal GitHub PR to apple/swift-evolution (acquire SE number)
Review Discourse https://forums.swift.org/c/evolution/proposal-reviews
15 Likes

I think making the stages more explicit is probably a very good idea.

I still have qualms about this use of GitHub. I feel that even the old-school pipermail email archives that for example python-dev has largely provide a better interface to reading these kinds of technical discussions, due to:

  • Threading
  • Overview of new messages in particular sub-threads
  • Overview that allows me to quickly scan for particular authors

GitHub issues (and even Discourse) really don’t do so well in these departments, and I’m skeptical that splitting up into many issues (in separate repos) will be the right improvement.

1 Like

Overall, I like it a lot, because it tries to address some problems I’ve been seeing. Few of these I’d like to point out:

  • It tries to accommodate Rust getting „bigger“ (eg. more people participating).
  • It tries to make the „mess“ more organized and explicit. I like the flow-chart (though I’d like to see the back-edges and failure modes there as well, in the final version).
  • It makes it more useful for after-the-fact historical study. If I look through some features and try to get a grasp how and why a specific decision was made, it takes huge amount of effort, especially with features through several RFCs, tracking issues (shared with some other features), and one never knows if some important part is missing. The process shouldn’t be just to make a decision, but to be able to go back to it and learn from it, for those who came later.
  • I see the increased overhead of introducing and pushing an RFC through as a benefit (or at least not as a downside). As Rust is getting more adoption and gains maturity, it should „calm down“ in its development ‒ the cost of introducing a change to the language, in the amount of affected code and people, is growing with the number of code and users, so the cost to introduce the change to it should too, gradually.

Unlike some, I do think an explicit spitballing stage is a good thing (except the word „spitballing“ itself ‒ as a non-native english speaker, this is the first time I’ve seen it). It requires to stop and think even with „small“ features. If anything, I’d say it could be possible to move two stages in one team session if the result of the first stage is good enough than to officially skip the first one.

I’m a bit sceptical about the champion thing. It makes sense mandating the teams to participate (I guess it can’t really work without that). But, should there be a mandate for the community to participate too? Which is probably a bit related to a philosophical question, who „owns“ Rust? Is it the team, with community just helping out, or is it the community with the team just coordinating? I guess it is something in between, but despite the proposal stating it would like more community involvement, the whole champion thing seems to be pushing towards the other direction. But that could be just my private impression.

As for the format of discussion ‒ I don’t see either Discourse or Github as optimal platform. Github is not very good for discussion, Discourse seems not to be an excellent archeological source for the future generations. And both would need some amount of indexing to find the way around.

6 Likes

Concerning the Discourse vs GitHub discussion, if neither of these fully fit the bill in isolation maybe the right thing to do would be to set up some kind of tighter integration between the two? Just throwing an idea up in the air, not sure what that should look like or how much work it would involve.

1 Like

Definitely. That's why I mentioned Discourse's customisable nature as a big plus. With GitHub you're limited to its API, whereas with Discourse many parts of the standard UI can be changed as needed. We already have a working version of GitHub linkbacks, which is one of many integration features that could be further refined:

https://meta.discourse.org/t/the-github-linkback-plugin/66081

2 Likes

Wow, thanks for the thoughtful feedback. Lots of good thoughts here. I don’t have time for a detailed response just now so let me just jot down some thoughts:

First, regarding the name “spitballing”, I think perhaps “brainstorming” would be a more widely understood choice (I’ve now seen two non-native speakers complain about it, so it seems like spitballing is clearly a confusing term).

Second, regarding the other detailed points, it occurs to me that the RFC process is probably not dissimilar from a lot of other design discussions, in that it is a bit hard to tell in advance just how everything is going to feel. I think it would probably make sense, before we convert wholesale to any new process, to do some kind of “trial run”, where we experiment with the new mechanisms. This would also give us time to try a variety of RFCs (both simple and complex, for example) and work out the kinks. I suspect we’ll immediately find a desire to make adjustments, no matter what system we wind up with.

7 Likes

It's really important to use language that native, as well as non-native, English speakers can relate to. With that in mind, "spit-balling" and "brainstorming" don't really mean the same thing. Generally, when I think "brainstorming" I think of coming up with lots of different ideas and looking at them all for any gems (i.e. "Let it Rain (ideas)!"). When I think of "spit-balling" I think of proposing a particular idea and seeing if I can "sneak a win" with it (i.e. "Get this spit-ball past the batter'). I think the idea we're going for here is definitely more of "Brainstorming" than "Spit-Balling" anyways (even if it weren't difficult for non-English speakers to relate to).

2 Likes

Discourse has support for ad-hoc tagging; it's just turned off on Internals, and would need to be turned on for this.

Also, when you just link to a thread, Discourse does linkbacks the same way GitHub Issues do, so even without tagging, split discussions are about as manageable as a GitHub repository would be.

Add "view wiki post with quote-bearing replies as inline annotations" to that list, like what GitHub does with the "Files Changed" tab showing each thread. I'd be happy to help with that, if I thought it'd get used.

2 Likes

As a native English speaker from the UK - I have no idea what spitballing is, and have just taken it to be equivalent to ‘brainstorming’. Regardless of the nuances between them, I would expect the latter to be much more familiar to native speakers outside the US, so I support that position

2 Likes

I think we all agree we should change the term “spitballing” here to one that is more widely familiar =) But for those who are curious, here is an interesting article about its origins, and the various meanings that it has had leading up to the one that I intended ( “to suggest ideas, especially those that are jocular, improbable, or impractical”):

(I was not aware of the connection to baseball or softball)

Against “Each RFC gets its own repository”, some RFCs are big, some are small, this will also cause the documents too fragmented. I think we should not depend too much on the github toolset, better to have a dedicated website for RFC process and still maintain the RFC docs in a single repository.

1 Like

I’m not 100% against a separate website, but:

  • The ability to edit the documents locally and offline, in a real editor and not in a browser is definitely a plus.
  • If you were designing such website, do you know how it would look like and what its features were? I believe the problem here is not exactly missing tooling, but not knowing what we want from the tooling in the first place.
2 Likes