When an RFC is posted, we start with a “deliberation period” or “quiet reading time” about the length of FCP (10 days) during which no public comments are posted on the RFC itself. Everyone gets a bit time to read the RFC and send their personal thoughts to either the RFC author or someone in a moderator/shepherd role. At the end of the deliberation period, the RFC author/moderator/shepherd posts all the comments they received along with who made them, and a summary of the themes present in the comments. Then discussion opens as usual.
The Rust community is growing, and the RFC process is starting to feel some growing pains. It’s becoming more and more difficult for people to get involved in the deliberation part of the RFC process without feeling overwhelmed with content. We’re missing out on feedback from folks who look at a recently posted RFC, see that it already has 50, 100, 200 comments, and close the tab (I have done this). The people who do comment have often not read all of the other comments, which leads to repetitive discussions that still expect a response but that all the people who are following the discussion have to experience over and over. A too-large number of comments on RFC threads are discussions that spin out of control and off topic. This may even be discouraging folks from submitting RFCs.
I’d like us to work on ways that we can get more feedback from more people without everyone getting overwhelmed. My proposal here is but one possible idea; this is at very early stages in my thought process and I’d like to see what others think.
In order to explain the concepts of my idea, I would like to present it in the context of a thought experiment where we are all physically in the same room. This will hopefully keep us (myself included) focused on the concepts rather than the concrete implementation in GitHub or a custom web app, and which dropdowns/radio buttons/textareas go where, etc. I think it’s worthwhile to decide if this would be a good direction to try going in before we invest effort in thinking about how we’d implement it.
Ok, so we’re all in a room-- Rust team members, Rust contributors, Rust users, any interested party. The RFC author brings in a copy of the RFC for each person and hands them out. Then we start the deliberation period. The room is quiet so that we can each concentrate on reading the RFC for ourselves. Each person has some number of Post-it® Notes on which they can write any questions or comments they have while reading. In the front of the room where everyone can see, there are these prompts for people to think about:
- What is your general opinion of this RFC and why? (in favor, agree on the problem but not the proposed solution, disagree with the problem premise, not sure, etc)
- What use cases aren’t addressed in the text of this RFC?
- What parts of this RFC are confusing and why?
- More prompts along these lines, might vary per-RFC depending on what kinds of feedback the RFC author is most interested in
We can’t prevent people from leaving the quiet room and discussing the RFC amongst themselves in another venue, but people who walk into this room will be presented with the text of the RFC, blank sticky notes, and quiet. (If my metaphor is falling apart: we clearly can’t stop people from talking about an RFC on other forums like internals, users, r/rust, IRC, Twitter, email, etc. However, if we turn off comments on the RFC during this time, anyone who visits the RFC during the deliberation period will only feel like they need to read the RFC in order to participate, not the RFC + the comments made so far).
When each person is done reading the RFC and writing up their comments, they put their name on their notes and hand the notes to the summarizer. The role of the summarizer could be played by the RFC author, an RFC shepherd, or another volunteer. The summarizer reads all the comments and groups them into topic areas. If the summarizer has clarification questions about anyone’s comments, they can have a quiet, private discussion until they both are satisfied the comments are clear. The summarizer does have to experience the potential repetition of the comments, unfortunately.
At the end of the deliberation period, all notes are posted on a wall for anyone to read to continue to ensure transparency. They’re organized into groups of similar comments that the summarizer has created. The summarizer also posts a short description of each group of comments along with how many comments are in the group and who made the comments in the group. Counting the number of similar responses, while repetitive, will actually give us a sense of how many people hold which views, that we don’t get today since some people DON’T want to repeat what has already been said. Anyone can choose to read all the comments or just the summaries of the groups, thus eliminating the repetition of reading many similar comments unless you so choose to do so.
At this point, normal RFC discussion resumes as it proceeds today. If anyone feels like their comments weren’t represented accurately, they can provide more clarification. People can ask each other questions about their comments. The RFC author can respond via comments or via RFC modifications.
Deliberately omitted at this stage; I would expect to flesh this out into actual implementation details if people like the ideas.
- There will still be a lot of repetition for the summarizer to read through, potentially more if people can’t see each others’ comments. This proposal doesn’t eliminate the repetition completely, but it does eliminate the experience of reading the same comments and discussions over and over (except for the summarizer) unless you decide to opt in to reading the full comments.
- Summarizing is a lot of work. People sometimes provide summaries today, but as comments that also get lost in the sea of comments. We’re also not consistent with how often or whether there are people to summarize on every RFC.
- Still might not encourage the kind of feedback we’re missing out on today
- This may actually, and will definitely feel like it, slow down the progress of an RFC. This may not entirely be a bad thing!
Rationale and alternatives
I think the proposal here will take away some of the overwhelming aspects of participating in the RFC process that exist today without unduly hampering the benefits we get from the RFC comment process, which include:
- Getting community buy-in before implementing an idea
- Discovering missing use cases
- Discovering new solutions that are better than the proposed solutions
Alternative: better summarizing of discussions/acknowledgement of points/answering of questions on a regular basis in the RFC PR description
- Like some of svenfuchs’ comments in this issue, but with links to the original comments and posted in a single, prominent position
Alternative suggested by natboehm:
- Have multiple deliberation periods: one round of comment submission, then one comment summary, then another round of comment submission, then another comment summary.
- Could make it easier to be inspired by others’ comments, have a discussion but slowly
- If these are 10 days each, could make more opportunities for people to have time to get involved
Optional addition or orthogonal thing:
- Today, RFC authors solicit feedback from people who represent important stakeholders and who might not participate organically. This process isn’t advertised very much, however. This could happen during the deliberation period and be explicitly called out in the summary. Example: “[RFC author] asked wycats, as a production Rust user, how this proposal would affect their workflow. wycats had this to say: […]”
I would like to hear about more alternatives!
- Should the RFC author be allowed to revise the RFC text in response to questions/comments during the deliberation period, or should all modifications be held until the end of the initial period?
- Should the summarizer be the RFC author? Or deliberately not the RFC author?
- Is 10 days enough time for people who do Rust work as their job and people who do Rust work in their spare time to read and submit their comments?
- When we open up the RFC for comments, will the discourse be worse than it is today?