Refining RFCs part 2: RFC staging


#26

I too think that a custom web app is a great idea.

I think that one of the issues here is the format of RFCs - a text document is a more centralized format which worked great so far for Rust. However, as the community grows and we tackle bigger and more complex issues together as a community, i think the process becomes a joint brainstorming more than a single person’s idea/design.

Therefore, I think we need a graphical format such as a mind-map at least for the initial stages. the RFC text IMO is just the final stage - it is the proposed spec doc of the proposed final design and it could be extracted semi-automatically from the mind-map.


#27

This is a popular model for TC39 proposals.


#28

I wonder if we can successfully predict whether an RFC would be better suited for a PR or repo? I would think we can, but maybe that’s just hindsight bias


#29

Another question about repo-per-RFC: how would discoverability be handled? Right now it’s very easy to go to the RFC repo and see the status of a variety of conversations, and it’d be a shame to fragment all of the conversations without some process for indexing and tracking the status of the various repositories.


#30

Seems likely. I think it makes sense to assume we’ll have a “lightweight” process for smaller things – if someone erroneously submits an RFC to the lightweight process, or if an RFC turns out to have more complications than originally thought, we could always move it over to the “heavierweight” process. One might imagine giving these two distinct names. Something like a proposal (the lightweight version) versus an RFC (the heavierweight version).

Anyway, I think for now it makes sense to focus on “heavyweight” proposals (such as most, though not all, language changes) since I think that is where the problems are most acute.


#31

Summary

Implementation Details:

  • Consider non GitHub/Discourse RFC tracking system 1
  • RFC PRs should be an RFC tracking system with links 1
  • API-type access to relavent links, CMS-style summaries 1
  • Summary points could have logical/topological relationships plotted 1
  • Github/Discourse may not be sufficient to hold long conversations 1 / 2
  • Heavy RFC get own repo? 1 / 2 / 3

Additions/Suggestions/Concerns:

  • Alternative/Supplement: distinct problem forums and solution forums 1 / 2
  • Incorporate clear/concise summaries on some cadence 1 / 2
  • Too much work for shepherds? 1
  • How to increase team/author communication? 1 / 2

#32

So one issue with the summaries is that it’s somewhat subjective and requires a good deal of work. At first you would think it’s probably best to give the responsibility of summaries to the authors, but that may be asking for a lot. Is it possible to automate these somehow? What I do think is important, is that perhaps we could somehow require a single sentence summary of your post to be included in the summary (to help out whoever is doing the summary), and your post can be referenced for more details if need be. I am a little uncomfortable if we can find a good automated system, so I have a proposal:

Require the author of a response during the RFC discussion to include a concise summary in their post (if they feel that the post is significant enough to be in the summary, as opposed to say, questions for clarifications on someone else post). I feel that this will make the job of summarizing the current status of an RFC significantly easier, and it will make scrolling through the posts significantly easier while trying to keep track of certain topics in the linear structure that GitHub provides.

edit: Clarify post; forgot to include my own one sentence summary! :blush:

Topic: Incorporate clear/concise summaries on some cadence. Suggestion: Require concise summaries for (most) posts by author of post.


#33

Agreed, for a busy RFC, it does! But it’s vital, I think, and all the more reason to find ways to spread the work around.


#34

@nikomatsakis I think I wasn’t clear when I wrote the post you responded to, since “author” is ambiguous in that context. I was trying to propose a way to spread the work around by making it significantly easier to construct summaries of an RFC’s current standing for whoever (most likely the RFC author). So I wasn’t sure if you were just giving a :+1: or that I didn’t make that clear in my post :slight_smile:


#35

So, stepping back a bit, I’m a bit nervous about this proposed change escalating into a lot more complexity. It seems like there are a lot of things being discussed here – modeling problems, structuring conversations, etc. I’d like to try and dial it back a bit though, and try to find what is a small, simple set of things we can do as a first step.

To me there are a few key motivations:

  1. Encourage more discussion on motivation:
    • Try to get consensus that on what precisely the problem is that is being solved first and on the most promising approach
    • Also get commitment from sub-team members to be more involved detailed design
    • Avoid RFCs that languish for a long time, awaiting attention
  2. Structure conversations:
    • It can be really hard to follow a long RFC thread, particularly on GitHub
  3. Increase bandwidth and mentorship:
    • Shepherd role for helping to summarize conversations and bring people to consensus
    • Makes a better path towards (e.g.) lang team membership

It’s interesting ponder what is a small set of changes that can work towards those goals. One thing I saw that seemed of interest also was D’s new RFC process, which definitely incorporates an explicit notion of “staging”.


#36

What exactly is a long team?


#37

Sorry, I meant “long time


#38

More related work: https://github.com/golang/proposal


#39

Unsurprisingly, given the unusual degree of overlap the two projects seem to have… it seems we might want to find a way to collaborate/share common tooling with the Ember team if possible. One of their learning team members noted relatively recently:

We (the learning team) have been slowly working on ideas for a “features” dashboard to help with this type of tracking ability [for the status of RFCs]. Nothing to show on it yet though …

Maybe there’s something to reuse, maybe not, but at the least it’s worth knowing what they’re building and seeing what might be reusable from it. No point reinventing the wheel if we don’t have to.

Edit: adding some relevant links:


#40

I have indeed been talking with @wycats a lot about these ideas. Actually, he and I were just talking yesterday, and I thought we came up with an interesting thought. We want to find better ways to structure RFC discussions that make them easier to follow, especially after the fact. GH’s “big bag of comments” model basically means you have to read every single thing to follow along, teasing out subthreads and so forth as you go (it doesn’t help that GH finds a lot of the comments from you). This is very tedious and time-consuming, particularly since a lot of those comments don’t even apply to the most recent draft (since they prompted subsequent changes).

But there is another goal. We want to empower people to shepherd and facilitate discussions on the topics that they personally care about. One RFC that made a big impression on me was the union RFC, where @josh did a great job of keeping the conversation on track and bringing us to consensus. One of the most important bits here are summary messages (like this one) that helped to give structure to the ongoing conversation.

So here is what I am thinking. What I would like to see is that when you arrive at an RFC page, you see in the “top comment” not only the rough motivation, but a guide to the conversation that follows. For example, this might include a list of subthreads on particular topics (ideally hosted on internals and not github), as well as links to summaries of important points in the conversation. The aforementioned union RFC actually has a primitive example of this: you can see that @josh edited the RFC summary as he went, including notable points in its development, and also included links to some prior discussion.

One problem is that, right now, that top comment can only be edited by people who own the repository and the RFC author themselves. This makes a rather limited set of people who can curate. Moreover, there is no standard form for the top comment, so you wind up with ad-hoc edits and so forth. Moreover, maintaining all these links and so forth can make the whole curation job even more difficult.

I was imagining that we might make a bot that can help in this regard. The bot might recognize specially formatted comments that claim to do things like “summarize the discussion from here to here”. It could then insert a link to that comment up in the top-comment, under a bullet list of “summaries”. There might also be a way to spawn subthreads devoted to a particular topic on internals: then when you visit the RFC, you would see the link to the appropriate subthread where some point is being debated.

Finally, we could use the dashboard to monitor discussions that are in need of this sort of work. If a discussion has been going too long, say, without a summary, the bot might add a “needs-summary” tag – or if a subthread goes dormant (hasn’t had any comments in a while), the bot might notice and indicate that the subthread seems to have reached conclusion, perhaps its time to report back the (current) consensus.

I can imagine that this might be an interesting way to be involved in Rust, as well – if you don’t feel like coding, you can put in some time trying to help guide the RFC and design process move along. And of course you might find something you want to get more involved in while you are at it.

Anyway, this is very rough, but I wanted to throw it out and see what people thought.


#41

@nikomatsakis Having somebody curate the discussion is a noble idea, but I suspect that enforcement of that policy would be hard. But maybe the difficulty is a good lever for making under-motivated RFCs go away more quickly. If nobody is willing to maintain the summary then it’s clearly not important enough, and a bot can auto-close it.

I also though think that the summary of the discussion belongs in the RFC. When I come to an RFC PR with hundreds of comments I expect to be able to read the RFC to understand the state of the proposal. The RFC should contain an up-to-date consensus design, and all the dissent should be recognized in the alternatives and open questions sections.


#42

Hmm, interesting. It seems like we could just move the summaries into the RFC at the end, though, as well. The harder part is getting them written. =)


#43

Just a quick elaboration on my previous comment. I might expect that after each ‘round’ of debate - where the discussion tends to be overwhelming and only interesting to those most deeply involved - somebody must update the RFC text to reflect everything notable from the debate. Then people less involved, including many of those who have to make a decision, just have to re-read the RFC.

It bugs me a lot to read RFCs where the design is obviously underspecified, and the alternatives and open questions sections are neglected.


#44

I definitely feel like the “alternatives” section seems like one of the most critical in the RFC, and it should document any remotely credible idea for the record, along with pros and cons compared to the proposal in the RFC. Adding summary comments in the RFC discussion on Github helps clarify the state of the discussion, and gives people an opportunity to say “no, I meant something different” or “I have a different issue”. However, the consensus record of the discussion absolutely belongs in the RFC.


#45

See Two-stage RFCs: motivation+overview, then details for a new take on this idea.