Refining RFCs part 2: RFC staging

  • to the issues about the problems being solved;
  • to the issue about the motivation for this RFC;
  • to a discuss thread (or multiple such threads) for discussing the RFC itself;
  • perhaps multiple threads, with summaries given where relevant;
  • if the RFC is accepted, a link to the final text in the repo itself;
  • if the RFC is not accepted, a link to the final rational

To chime in from an automation perspective -- making sure this follows a standardized format would be very useful for providing some visibility into the statuses of various RFCs.

Yes. This reminds me that @brson and I have kicked around the idea of making a dashboard page for this summary instead.

  • to the issues about the problems being solved;
  • to the issue about the motivation for this RFC;
  • to a discuss thread (or multiple such threads) for discussing the RFC itself;
    • perhaps multiple threads, with summaries given where relevant;
  • if the RFC is accepted, a link to the final text in the repo itself;
  • if the RFC is not accepted, a link to the final rational

While in general I sympathize very much with all of this, I will say: some kind of central user interface as suggested above needs to exist, and the design of such an interface needs to be very carefully considered, or it will be even harder to understand RFCs than it is today. The downsides to GitHub-driven RFCs are fairly well-known in general and are well-summarized by @aturon at the top of the thread; but the upside is that there is one canonical source of truth for that RFC.

That said, if we have API-type access to all the relevant places where the discussion happens (Discourse threads, relevant GitHub issues indicating problems, etc.) in such a problem tracker, along with (perhaps?) a CMS-style interface for the shepherds to use for summarizing status, that could be a real win.

1 Like

Just wanted to mention a couple more current “pain points” which are sort of complementary to the “it’s hard to get caught up with RFC discussions due to their length” problem:

  • When discussion of an idea or proposal keeps moving to new threads (discourse, issue tracker, RFC postponements followed by new RFCs), thoughts I’d expressed in one of the earlier iterations are often still just as relevant, but now essentially lost as most people don’t go back to read earlier discussions (if they even know about them). Having to re-express the same thoughts all over again every time a new thread is started is exhausting, and starts to make it feel like a contest of stamina rather than a discussion about ideas. (The most egregious example of this was/is the “impl Trait” feature.)

  • When a discussion thread gets long enough, it often happens that someone doesn’t have time to read the whole thread, but still wants to propose an idea or ask a question that seems “obvious” to them. Which just happens to be the same idea that’s been brought up in that thread, in the same fashion, 50 times beforehand. (Examples: “why can’t we just use fn foo<T>() -> T for abstract output parameters”; “why can’t we just optimize Box::new instead of adding a placement operator”.) Which then exacerbates the original issue by making the thread even longer. (Under the existing regime, all solutions to this problem are bad: either you erect an unreasonable barrier to entry for discussing RFCs by forcing everyone to read miles of discussion before commenting themselves, or you burden everyone else with the consequences when people don’t and then keep independently bringing the same ideas up over and over.)

All of these problems (including the originally-mentioned ones) seem to motivate the same kind of solution: having some kind of persistent document/page/site where the ideas, arguments, and counterarguments are mapped out, according to their logical or topological relationships, “once and for all”; and this would be grown and fleshed out as discussion progresses. (Which sounds broadly similar to the “Problem Tracker” described in the blog post.) But doing this “right”, or even “good enough” seems nontrivial: Who gets the right to edit these pages? How are the ideas even laid out – as a linear text document (with its attendant drawbacks), or as some kind of nonlinear tree or graph structure (laid out how)? You can’t even reliably silo off particular “big-ticket items” or “problem areas” from each other, the problems and/or their solutions are very often interrelated. (For example.) (Maybe the best solution to this last problem would be to employ a sophisticated device, known as the “hyperlink”; like a wiki, if not actually a wiki.)

2 Likes

Some thoughts on staging:

  • separate motivation RFCs seems like a really excellent idea!
  • also very much like the idea of expanding the pool of shepherds
  • I’m a bit worried by the role of shepherds as scribes - shepheds are already overworked and not properly fulfilling their duties. I’d also like to encourage authors to be more responsible for RFCs and this seems a bit like babysitting. In the future I’d like writing and ‘maintaining’ an RFC to be more work, but much better supported, rather than less work, per se.
  • others have mentioned collaborative authorship of RFCs, this sounds good to me and we have the tools (GitHub) and culture (OSS) to make this work.
  • I’m not keen about moving more work to Discourse, I like it for conversation, but not for serious editing, etc. (Not that GitHub is perfect by a long shot, but still).
  • The summary/state of the art conversation seems like a good candidate for being a living document/wiki on the dashboard.
1 Like

I would love for authors to be more included in RFC processes, and one issue I’ve seen as an RFC author is that it often feels pretty hopeless; there’s absolutely zero communication between the teams and you. I didn’t even know that there was such a thing as a “shepherd” until last week, despite writing… four RFCs over the past year.

I think this is really important to fix. 'Better communication' is something that I think would improve a lot of the friction around some of the process-ey stuff in the community. And you are right that communication between the sub-teams and the wider community (and in particular RFC authors) has sometimes been poor.

I'd like to see suggestions in this round of process updates for ways (formal and informal) to improve communication. I hope the increased pool of shepherds and increased level of interaction required to maintain the rolling updates will help. More ways would be better too.

1 Like

In my opinion, the biggest problems here is that github’s (and discourse’s) interface is not meant to be used for long conversations.

When commenting on an RFC, most people don’t read the entire conversation and bring up the same points that were already raised earlier in the conversation.

But how can you blame them when it’s so freaking long? For example I got interested in an RFC this morning, but the comments are 36 pages of text long! Either I read everything and it take me the whole morning, or I don’t read it and I’ll be “that guy” who brings up a point that was already discussed earlier. The end result is that I won’t participate in that discussion at all, even though I think I have an important point. I’ll wait until the RFC is approved, then if necessary raise my point in the tracking issue, which is usually less flooded.

The same problem arises if you open your own RFC. As you are supposed to defend your RFC, you are also logically supposed to read all the comments. But instead of reading few constructive comments, you get flooded by walls of text. I personally have two RFCs in mind that I’d like to propose, but that I won’t because of this point alone.

Compare this to reddit’s interface. Sometimes some RFCs have been linked on reddit and discussed on reddit, and it makes it pretty clear that their interface is much more readable for long discussions. Obviously I’m not saying that we should move the discussions to reddit, but I think it shows a clear problem.

4 Likes

This is a good point too... and this somewhat characterizes my own behavior as well. (It hadn't even occurred to me that it could be otherwise...)

The problem I find with Reddit is that in can be hard to find new posts during the discussion, since one has to figure out under which subthread it appears. If one is interested in following the entire conversation, Discourse’s linear format works better. Newsgroups seemed to have the best of both worlds (easy to identify new messages in any subthread, but the conversation tree was clear and it was easy to ignore parts of the conversation that went off the rails). I’m still surprised none of the current web-based options seem on par.

1 Like

The real problem is that they provide this, but only for people with Gold :confused:

That said, I would pretty vehemently oppose Reddit being a part of any official Rust process; its format isn't the problem though.

2 Likes

Another appropriate format for example, is that if each RFC was a repository we would open an issue for each point to discuss.

Again, this is just an idea. But in my opinion, the current state where the discussion is linear is the worst possible system.

I've thought about this from time to time. I think another factor here is that some RFCs require more comments than others, and making a fresh repo for smaller RFCs feels a bit like overkill. Maybe the answer is that such things shouldn't be RFCs at all, but something else, I don't know.

It seems to me like a custom web app for discussing RFCs could be a worthwhile project, since there’s no prebuilt platform with the communication flows we want. Delegate auth to GitHub and probably have it automatically maintain the RFCs repo as well.

1 Like

The one repo per RFC model seems like a good fit for more complex RFCs, in particular ones where we expect collaborative change and thus multiple PRs (PRs against PRs don’t seem very popular). However, I fear the details of this idea will kill it - who would manage the repo, where would it be hosted, how do we arrange the right admin rights, etc.

I too think that a custom web app is a great idea.

I think that one of the issues here is the format of RFCs - a text document is a more centralized format which worked great so far for Rust. However, as the community grows and we tackle bigger and more complex issues together as a community, i think the process becomes a joint brainstorming more than a single person’s idea/design.

Therefore, I think we need a graphical format such as a mind-map at least for the initial stages. the RFC text IMO is just the final stage - it is the proposed spec doc of the proposed final design and it could be extracted semi-automatically from the mind-map.

This is a popular model for TC39 proposals.

I wonder if we can successfully predict whether an RFC would be better suited for a PR or repo? I would think we can, but maybe that’s just hindsight bias

Another question about repo-per-RFC: how would discoverability be handled? Right now it's very easy to go to the RFC repo and see the status of a variety of conversations, and it'd be a shame to fragment all of the conversations without some process for indexing and tracking the status of the various repositories.

Seems likely. I think it makes sense to assume we'll have a "lightweight" process for smaller things -- if someone erroneously submits an RFC to the lightweight process, or if an RFC turns out to have more complications than originally thought, we could always move it over to the "heavierweight" process. One might imagine giving these two distinct names. Something like a proposal (the lightweight version) versus an RFC (the heavierweight version).

Anyway, I think for now it makes sense to focus on "heavyweight" proposals (such as most, though not all, language changes) since I think that is where the problems are most acute.