AiC: Adventures in consensus

1

I just posted the first post of what I expect to be an "irregular series" of posts, entitled Adventures in consensus:

In the talk I gave at Rust LATAM, I said that the Rust project has always emphasized finding the best solution, rather than winning the argument. I think this is one of our deepest values. It's also one of the hardest for us to uphold.

Let's face it -- when you're having a conversation, it's easy to get attached to specific proposals. It's easy to have those proposals change from "Option A" vs "Option B" to "my option" and "their option". Once this happens, it can be very hard to let them "win" -- even if you know that both options are quite reasonable.

This is a problem I've been thinking a lot about lately. So I wanted to start an irregular series of blog posts entitled "Adventures in consensus", or AiC for short. These posts are my way of exploring the topic, and hopefully getting some feedback from all of you while I'm at it.

This thread is a place to collect feedback for this post and future posts (I'll update with links as they are posted).

Post links

29 Likes
2

Worse, I think the current process often starts with a particular solution . This encourages people to react to that solution .

I think this is key; and I think it would be worth splitting the RFCs in two documents:

  • A problem document, meant to explore the problem space: core issue, linked issues, usecases, stakeholders, assumptions which may have to be challenged...
  • Multiple "solution" documents, meant to explore the solution space: attacking the problem, identifying the trade-offs of the particular solution.

I think it may also be worth enriching the problem document with some sort of weighted objectives, but I am somewhat wary about it muddying the waters and starting looking like the embryo of a solution.

And I should also specify that I do not think this should be a waterfall; I'd expect all documents to evolve concurrently, with explorations of the solution space leading to realizations about the problem space, and to challenging assumptions that were made.

The idea behind the split, instead, is two-fold:

  • It cleanly separates the discussion space between problem and potential solutions.
  • It makes it easier to explore multiple solutions in parallel, by having a central place (the problem) linking all alternatives.

The latter is particularly powerful for archiving purposes, whether to document the living process of shaping Rust, understanding how certain decisions came to be and possibly challenging some, or to come back to a problem that was tabled for lack of time/resources later.

As a side effect, I think it also lightens the RFC process a bit: now, the start is posting a problem, not a solution, which the community will help shaping up. It seems like this would help avoid the bitter feeling of having poured a lot of work into a potential solution only to have it tabled or torn to pieces.

And ironically, I am here presenting a potential solution before even defining the problem in detail.

14 Likes
3

Bryan Cantrill has given a couple of talks on software as a reflection of values (here’s one that includes Rust). The premise is that a piece of software is a manifestation of the values that its creators and its users care deeply about, and are ready to put above the values that are espoused by other projects or people. Values are in tension (e.g., it’s hard to have a system with that is both approachable for beginners and powerful for experienced users), and if a project and a person share no common value, it’s pretty clear that the person won’t want to use that particular software.

So what are the values of Rust? Safety, performance, and expressivity are three that come to mind; are there others? (Cantrill identifies quite a few more.) I think that the community should think about this question and write down somewhere—ideally in big bold letters—the values of Rust. Then, when a proposal comes along, software as a reflection of values can be an evaluation criteria that is used in addition to the ones proposed by Niko. Where software as a reflection of value would particularly shine (I think) is when a proposal (a) is in line with the values of Rust, but (b) is in conflict with one or many values of a participant.

Let’s look at an example, async. Async makes it easier to write fast systems software without compromising safety, what’s not to like? Well, those improvements come at the expense of simplicity; though async is an embodiment of the Rust values (at least, the ones I identified above), it might conflict with the values of a participant. When this happens, the participant may want to push back against the proposal, but not for technical reasons: for moral reasons. But if that participant can realize that async is in line with the values of Rust, they may decide to put their unease aside and contribute to discussing the technical merits of the proposal.

12 Likes
4

I have a talk at QCon London based on Brian’s framing here, with my personal views about what Rust values. I don’t think the video is out yet though! I think it’s a fantastic way to think about things.

7 Likes
5

I think that RFC discussions could be better structured, so that the same arguments aren’t repeated over and over.

Maybe we should consider other platforms that are better suited to reaching consensus. GitHub Issues and message threads don’t scale well with hundreds of contributors. For example, I heard good things about pol.is.

1 Like
6

I posted something new =)

1 Like
7

I think there's something here. RFCs can be big, monolithic documents, which can make them daunting to read and, for many people, actually writing one seems like an impossible task. So we end up with one RFC that proposes a solution, with a ton of comments that no one has time to read. Sometimes a new RFC is written with a different proposal, but this happens infrequently. Maybe splitting up RFCs into problem and multiple solution documents could help make those giant comment threads smaller, by splitting them up across different pull requests, and by giving people an outlet to propose and discuss their own solution in a new PR rather than in the comments on the single main PR.

8

I agree with the sentiments that we can improve the process by improving how we handle the different steps of the process, adding more smaller incremental steps, each with discussions at its appropriate level, allowing concurrent discussions, etc.

But one thing I want to call out is that the more we do this, the more we should IMO be thinking about how to summarize each of these discussions and merge them in a cohesive way somewhere, such that ideally people just read that, and are able to jump into the discussions without, e.g., repeating arguments.

I think we are currently failing hard at this.

For example, @matthieum called out that one monolithic document might not be a great idea, but I want to argue that currently we don’t even have that.

I was recently scouting for rationale from RFCs for the UCGs new inline rationale sections, and the current process requires a full-time an internet archeologist for that. The search must include:

  • internals post, rust-lang/rust/issues, rust-lang/rfcs/issues
  • previos closed RFCs
  • pre-RFCs
  • RFCs Rationale section: which often only contains rationale about why the alternatives were not picked, but the rationale about why was the merged solution picked is scattered
  • RFC Motivation / other sections for rationale
  • RFC thread for both arguments / inline discussions that were never updated to the rationale, and summary comments, which are never merged into the RFC rationale, as well as raised concerns by team members, etc which often result in changes to the RFC
  • Tracking issue, and summary comments there
  • Stabilization PRs, summary comments there
  • Bug reports for the feature, breaking changes, fixes, reverts, …

Most of what happens there is not summarized, and if it is, the summary does not land anywhere, and if it would, chances are that it would be changed later.

I think that a single source of information for a language feature that’s kept in sync would already be an improvement, even if it ends up being a huge document. Having multiple documents, more threads of discussions at different levels for the different stages, etc. would then be an improvement over that, but gathering, filtering, and sync’ing the relevant information, might end up being more work and we should keep that in mind.

10 Likes
9

There’s a new post in this series:

http://smallcultfollowing.com/babysteps/blog/2019/07/10/aic-unbounded-queues-and-lang-design/

It reminded me a lot of the first section of my Rust 2018 post:

https://dirkjan.ochtman.nl/writing/2018/01/14/rust-in-2018.html

2 Likes
10

Thanks for bringing that up! Re-reading again I definitely found myself nodding along a fair bit. I never did read the epic treatise on bug triage that you cited, I've added that to my queue.

11

It comes highly recommended. I am dealing with exactly that kind of “unused inventory” problem at the moment. The public & private dependencies RFC was accepted and languished for … a long time. In the meantime several other changes make the RFC no longer completely designed. Those other changes should have included an explanation of how they interact with this RFC, but I don’t blame them for not looking through all the languished RFCs to see “if this was ever implemented would this change still work”.

3 Likes
12

I think you captured the core tradeoff (focus vs. serendipity) & most meaningful constraint (reviewer bandwidth) very well, and at least intuitively I agree with both of them!

one of the things we’re going to have to figure out is how to draw good boundaries so that we can push out a useful subset of a feature (an “MVP”, if you will) and then leave the rest for later

I'm sure you've also thought about this, and the post was just nuanced enough as it is, but - although this is very possibly the least-bad approach - it also has tradeoffs and hazards. The failure mode here is that in terms of the end-user experience, the language ends up feeling like it's full of jagged edges, with users frequently running into errors or limitations whenever they stray from the "happy path" of the design, and, upon asking about it, learn "yeah, a solution to that was discussed on the RFC repo at some point". Since it helps to have a memorable term, I like to think of this failure mode as "Procrustean design".

A specific (hypothetical but plausible!) dynamic which could exacerbate this: a given feature X is considered a high priority, so by extension, the MVP of X is also a high priority, but once X is shipped, the chopped-off pieces no longer qualify as particularly urgent, and languish.

(Of course, "whole features" usually don't have clear & simple boundaries, so in some sense this is all subsumed by your suggestion that we need to find "good boundaries".)

1 Like
13

I have indeed thought about this. I was talking to @wycats about this recently and he elaborated a useful rule of thumb: it often makes sense to try and reach the "80-20" point (point of diminishing returns, etc) as quickly as you can, but that once you think you've reached that point, we should try to avoid piecemeal extensions and instead tackle the problem holistically. I think that impl Trait is a good example of a feature that reached its 80-20 point, and where we should aim to put the whole picture together at this point (which reminds me that I still want to try and pull more of that together).

2 Likes
14

New post: Shepherds 3.0

http://smallcultfollowing.com/babysteps/blog/2019/09/11/aic-shepherds-3-0/

First thoughts: this feels like the wrong way around somehow. To make progress with the unbounded queues problem, it seems to me that the first thing the lang team should do is come to a roughly prioritized list of things that they are actively working on. The notion of shepherds seems useful mainly/only as a way of setting an initial size of that list (roughly one per active lang team member).

The notion of allowing anyone to be a shepherd like this seems interesting but mostly orthogonal, and, with the risk of sounding like a broken record, cutting down on the number of in-progress things seems much more important.

2 Likes
15

I'm not sure we actually have an unbounded queue problem... I would only agree that we have one in the sense that anyone can make an RFC or so but we are generally not sifting through those RFCs at the moment because of the roadmap goals to focus on finishing what we started. Achieving this is largely a compiler engineering matter. New language design being slowed down to just a few urgent items (e.g. &raw and the unwind) is a good thing.

...What I think we really need is more compiler engineers. To help out, I've reoriented myself towards working on accepted RFCs both in terms of implementation and stabilization reports.

I also think we largely do have a "shepherd system" as is described in the document but instead of the github repo we use dropbox paper and agenda documents, recording the implementation history on tracking issues (at least that's what I do), and feature gate labels instead. I think things are mostly working well and are improving. We have shipped lotsa things. I guess we could also distribute the work a bit better across the language team and other folks who want to help out write good reports; I've tried to do so recently and it seems to work well.

1 Like
16

Response/followup by James Munns on the Shepherds 3.0 post: https://jamesmunns.com/blog/shepherding-3-1/

17

Hey all -- a few notes on what's been happening in the meantime. It seems like we're starting to consolidate on some vocabulary and common concepts across teams.

First of all, James Munns (don't think he's on internals?) wrote a nice piece on shepherding, and also opened an embedded wg RFC based on these ideas.

I've started trying to restructure how the lang team works to create a simple version of shepherding. If nothing else, we are now tracking ongoing projects and checking back in on them, and in some cases assigning shepherds.

One thing I realized recently is that we don't really have much documentation for how the "feature process" works today -- at least I couldn't find anything that included all the stages. e.g., pre-rfc, rfc, tracking issue, stabilization report, etc.

I also started drafting a "shepherded project proposal" -- this aims to bring together shepherding, stages, and a few other things besides. I am looking to "dogfood" the ideas by moving those notes into a repository and establishing a rhythm, in conjunction with the Lang team and (I think) Governance WG, to discuss these things regularly and refine the proposal.

(I'm also looking to try out the process on the ffi-unwinding topic, which @BatmanAoD and acfolzer have bravely agreed to do as well. =)

1 Like
18

I think we're dealing with the problem -- i.e., the fact that we don't spend a lot of time reviewing RFCs. This may not be the best solution, though, as it's not very clear which RFCs are progressing and why.

Similarly, I think the impl backlog is one of the symptoms of unbounded queues -- that you get an imbalance in one area, but we keep proceeding in other areas despite that.

I agree! One of my big goals here is not to "reinvent the wheel" but to better document and expose the system that we have today. For example, I believe we already have a staged feature development system that proceeds like so:

  • Exploration: Pre-RFC
  • Initial design: RFC
  • Implementation: Tracking issue
  • Stabilization and stabilization report

but I think this full system is not very well documented, and it's often quite hard to follow. This ranges from mundane things like (a) no one place to watch (moves from github repo to repo) or (b) RFC threads that are full of out of date comments to more complex things like (c) we sometimes let blocking concerns and things last forever without great follow-through and (d) it's not always clear what kinds of concerns are appropriate to raise at each time.

This is why I'd like to see us do a better job of describing the system we have and then thinking it over to see if it can be improved. I also think we should keep in mind the needs of folks who arrive with little context -- what can we do to make it as obvious to them as possible what's going on? I think that's very much in our interest, not the least because it'll help us get precisely the feedback we want.

Anyway I don't want to go overboard on process. I don't for example think that "every shepherded project needs a github repo" -- but I do think a dedicated repo is a great way to organize complex projects like ffi-unwind. (I don't think that those repos should necessarily be the place where conversation takes place.)

3 Likes
19

I think not spending a lot of time reviewing RFCs when we don't have time is how it should be. In my view, the language team should act as a funnel through which proposals have to pass before they even reach the compiler team and through not reviewing RFCs I think we also put a limit on the amount of work that is unfinished. If you think this means we have unbounded queues then I guess we do, but I think that's fine, and I don't see how it would be possible not to have unbounded queues if so. This also biases towards quality rather than quantity in feature development, which is also how I think it should be.

I think the aforementioned practice of recording the implementation history and keeping tracking issues forcused on tracking the implementation (and not stabilization proposals or in-depth) design discussions are a good way to do that. Generally keeping things well-organized and well-linked seems to help.

1 Like
20

New post:

I want to write about an idea that Josh Triplett and I have been iterating on to revamp the lang team RFC process. I have written a draft of an RFC already, but this blog post aims to introduce the idea and some of the motivations. The key idea of the RFC is formalize the steps leading up to an RFC, as well as to capture the lang team operations around project groups. The hope is that, if this process works well, it can apply to teams beyond the lang team as well.

3 Likes