Last week in prep for the Rust All Hands I wrote down a bunch of thoughts about governance in Rust, specifically focusing on how we conduct online discussions. Sharing this a bit more widely now:
23 Likes
This is a thought-provoking read. I definitely agree with your statement of the problem, and I’ve also noticed that having two individuals go chat out-of-band and report their consensus works really well (e.g. you and josh on the Non-ascii idents thread (kudos)).
My main question is how to build a large group of facilitators. In particular, it seems that facilitators would need to either (a) follow all discussions everywhere always or (b) always have the time to catch up on threads that are about to get heated and head them off. (a) sounds just plain impossible. (b) sounds plausible but exhausting. Moreover, it would be better if discussions never got to that point at all… if the discussion process/format was changed to just encourage doing the right thing in the first place.
It seems like one thing that might help keep discussions concise and useful is an easy way to spin off side-discussions. Currently, with GitHub threads, the easiest way to respond to someone is in-band, so that’s what people do by default. But if it was easier for two people (or a small group of interested parties) to branch off the discussion, have a conversation, and merge back to the main discussion with their conclusions, then perhaps that would become the default. I can see facilitators encouraging this kind of discussion pattern effectively.
In my head, I keep thinking of Slack’s threads. TBH, I don’t really like Slacks threads, but they are great at keeping the main thread clutter-free and they do make it convenient to have side discussions without the need for everyone to see all comments. I’m not necessarily advocating for moving away from Github; I think there are other benefits to github that we should consider. On the other hand, perhaps we should change forums? Or perhaps we just need a bot that makes it easier to spin off out-of-band chats somehow?
2 Likes
My main question is how to build a large group of facilitators
Well, the way I was envisioning it was that facilitation would kind of be pull-based, teams ask for a facilitator (and block on the existence of a facilitator) for threads they feel will get contentious. Most threads don't need facilitators. Facilitators can also be called in when stuff has unexpectedly gotten contentious -- this form of defusing has worked in the past -- but ideally it's more proactive than that. Facilitators can also assign themselves to threads they are interested in -- usually threads they feel they have very little opinion on, but are on a topic they wish to see progress on. We can also treat this somewhat like an ops position with early metrics informing facilitators what's going on (@anp has a bunch of ideas on this)
When I was talking to folks there was a lot of interest in filling in this role at least partially -- I feel that a large chunk (if not a majority?) of the community doesn't have strong opinions about most bits of contention on discussions, but often does want to see these things go through and would like to help cut through the bickering smooth the discussion so that it can progress. This role can be designed without a hard time commitment, too, making it easier to be a part of.
It seems like one thing that might help keep discussions concise and useful is an easy way to spin off side-discussions
yeah, I've brought this up in the post but this is something others have suggested too -- making this a first class part of our process (one-repo-per-rfc is one commonly brought up format)
fundamentally i consider this a social problem, not a technical one, so I personally don't think that a technical fix (changing the discussion format) is sufficient.
(probably should explicitly mention this in the post, I've found myself saying it often enough)
2 Likes
I’ll write more later, but “Do we see a role for active moderation for certain venues in helping our community cope with growth right now?” is a much more interesting question to me than “What tactical changes can we make to get further along?” It might be that the former question has an obvious “no” answer, but we should think about shoring up our governance process for the potential eventual outcomes of the (up-to-now exponential?) growth we’ve seen.
1 Like
Some more interesting questions:
“What insight does an SREish approach offer for helping communities reach consensus?”
“Does anyone want to be an RFC gardener?”
1 Like
That's what I immediately thought of, too : maybe something along the lines of reddit. That's one automatic way to scale well [EDIT] way to help with scaling, but of course the facilitators idea is great. Just my 2 (or 5 or 50) cents.
Pure technical fixes aren't going to cut it. This includes the Reddit threading model, because that model has its own problems:
-
Because it makes tangents easy and relatively harmless, it also makes tangents more common. Even when it's possible to distinguish the garbage from the gold without having to literally read the entire thing, it still makes the reading experience marginally less pleasant.
-
The way it's presented encourages "tactical threading," a degenerate state where most of the conversation is registered as a reply to the top comments because that gives it more visibility, even when a more objective taxonomy would make it its own top comment.
-
Sort by anonymous vote encourages meme writing over detailed back-and-forth. I assume by "something along the lines of it", you mean to replace vote sorting with something else, since the "circlejerk" phenomenon is probably Reddit's most famous failure mode.
Which isn't to say that we couldn't employ a model like that. I am saying that it wouldn't really reduce the need for someone like a facilitator, because changing the discussion format really just brings in its own set of trade offs.
6 Likes
I've been on forums where full threading is used for Serious Discussions (e.g. Wikipedia) and the discussion burnout problem is worse there.
Reddit's model sometimes works for low-stakes discussions because people just leave when it gets too much but can't scale to handle high-stakes discussions where you also now have to sort between irrelevant tangents and relevant important stuff all buried in the threads.
Curated threading -- facilitators proactively splitting out important and involved subdiscussions into their own thread -- is something that can be useful here, however.
1 Like
You've been reading about Discourse's fourth trust level, haven't you? I should warn you that there are known weaknesses in its approach.
-
One of the obvious ones is that a TL4 user can't break a mega-post in half. The only way I know of is for a moderator to edit the off-topic chunk out, make a new post elsewhere, and reassign the post to the OP. And only moderators can do that.
-
The other big downside, inherent in any curated approach, is a feeling of lost autonomy. Kind of unavoidable, since giving you power is synonymous with giving you a system that you can game, but feelings of powerlessness are exactly one of the things you said you wanted to fix.
-
Taxonomies are not neutral. This includes whatever rationale the leader in question uses to decide if something is on-topic.
In spite of the fact that I think this is better than Reddit's approach (notice that I have TL4 on here, while my Reddit account lies deleted and burned), it's not going to completely fix the growing sense that individuals don't have the ability to shape rust's direction any more. Because they don't. And any process, even a manual one, is going to be dominated by whoever puts in the most time to deal with it.
2 Likes
I don’t think either a social solution or a technical solution is sufficient on it’s own; both are necessary.
I think the key here is that just creating more threads doesn’t help if those threads also become unproductive; you’ve just created more noise. Rather I observe that the most productive side-conversations I have seen are two people discussing one-on-one somewhere else in good faith and coming back. This suggests the following properties:
- The discussion should not be visible to others by default. It should be out-of-band, but linkable.
- The discussions should have a fixed and well-defined purpose: they should be spawned to discuss some particular comment or view point, rather than the whole proposal in general. Once that discussion is done, the thread is done.
- The discussions should be strictly invitation-only. The creator of the thread should invite one or two select individuals to discuss. It is strongly discouraged for anyone else to join the conversation.
- Once the thread is over, the conclusions drawn from it are posted back at the main thread.
Assuming we agreed on the above (bare with me), none of the existing forums really seem to meet these criteria currently. It seems to me that the github might be the closest, but I think we would still need some automation to make it work.
2 Likes
Oh, no, this is something I've been talking about for at least two years now.
but feelings of powerlessness are exactly one of the things you said you wanted to fix.
Having nonpartisan entities do the actual curation really helps with this.
one of the things I suggest in the post is precisely this: the facilitator proactively encourages people to have a discussion elsewhere (perhaps creating a space for them, e.g. a github issue), and links it back. Technical tools can help make this nicer (especially for moving existing comments) but this is already doable with the tools we have. Whether or not this discussion is invite only depends on the situation, for involved discussions on specific points it makes sense for this to be an open forum (just elsewhere), whereas for individual disagreements encouraging people to use private chat seems enough.
2 Likes
One thing that is (to me) conspicuously absent from all of these blog posts I’ve read on Rust governance is any statement that large spike in discussion on proposals and RFCs is resulting in better outcomes. In fact, it seems that there is a near unanimous view that outcomes are unchanged or worse degraded by these endless debates. If this is the case, then it would seem to follow that it would be better if the vast majority of posts in contentious threads were never made.
1 Like
I agree with that, but I'd like to express it with a little more nuance.
Most comments in high-volume RFCs don't add much new - they're likely either contentious or not saying anything new. Those are the comment we want to less of. And we have two main ways of reducing that flow:
- Resolve the motivation for posting (e.g. in this thread we talk about resolving conflict offline, ergo reducing conflict and contentious comments). People who would have commented in this scenario but didn't are satisfied.
- Placing barriers to commenting. Could be an artificial comment cap, locking the discussion to certain people, or holding the bulk of the conversation elsewhere and only posting the conclusion, not the arguments that lead to it, etc. People who would have commented in this scenario but didn't are not satisfied.
I don't think you were endorsing #2 (or that anybody in this thread did), but I just wanted to note that in addition to aiming for less comments made, we want to make sure to aim for less comments made because the people who didn't comment feel like their potential comment has been addressed.
And I think this thread has a lot of good suggestions and considerations made in that regard.
3 Likes
FWIW, I think some form of #2 is inevitable if we really want design discussions to scale. Maybe we don't quite need it yet, and we should certainly try all "type-1" solutions first so we keep type-2 solutions/barriers to a minimum, but I can't see us never needing a barrier of some kind. But I don't think that barriers to commenting necessarily mean leaving people unsatisfied.
The key thing is to distinguish between "comments" that everyone is effectively required to read before participating in the discussion at all, and "comments" that are optional to read or respond to. I think we can all agree that there should always be an official place (covered by the CoC, team members watching, etc) where anyone can comment on any RFC with no barrier other than the ability to write English. But we don't need to make all of those comments automatically go in the "you must read this before commenting" pile the way github RFC threads implicitly do today.
For example (this is a strawman just to make my thoughts concrete enough to avoid misunderstanding the general idea, don't worry about the details), we could lock comments on the github RFC pull request thread to just team members and facilitators because it's their job to summarize discussion and ultimately accept/reject the RFC. Then have this or some other Discourse instance be the official place where anyone can post opinions, comments or questions related to the RFC, including disagreeing with past summaries, and it'd naturally be just as well-threaded as irlo is today (which is to say, not perfectly, but still asymptotically better than smushing everyone into one thread).
In addition to the many obvious threading benefits that every suggestion here hopes for, I like to think something like this would lead to more people participating, not less, because you no longer have to worry about your comment or question potentially wasting everyone's time (it'd certainly have that effect on me), and you'd have far fewer "mandatory comments" to read before participating.
6 Likes
I like that idea a lot actually. Specifically, I like the idea of having a continuous stream (where anybody can participate) vs an ongoing summary.
I had an idea similar to yours, but instead of two locations, the summary is inline at the top - a moderator asks people to edit/delete their comment saying that it’s been summarized in the first comment, after said moderate has made sure that is in fact summarized up there.
But I really like the two separate locations you suggested because it’d be easy to a) keep the github RFC clean and official status update and b) still allow people to add there two cents (in the irlo). I actually really like it.
I know you said it was a strawman, but I really the direction it’s going. You could go so far as to ask people to strike through their comments on irlo if they feel like it’s been well addressed/summarized in the github RFC. Thinking more, the downside is that it requires all people who previously have participated in github discussions to now also have a irlo account (not a huge barrier, but still a barrier). For sake of ux continuity, we could have both threads as separate github issues.
I’d have to think more about the downsides of two separate locations, but I really like that idea for things that are going to be huge.
Part of the reason I threw out that particular github + irlo strawman is to show that I do not believe we need to invent any brand new discussion management software or convince existing platforms to make major changes in order to solve these problems. I don’t think anyone’s explicitly said that, but a lot of speculative comments on this subject seem to drift in that direction, which lends to a sense of “this would be nice but obviously won’t ever happen” that I’ve been feeling in some of these threads.
I don’t think I have any strong preference about what the two “locations” should ideally be. Having some kind of special summary thing at the top of a github discussion thread would be cool, but obviously that falls under “convince existing platforms to make major changes”. But we could get awfully close even with current tech, by repurposing the initial PR comment as such a summary, and giving team members/facilitators the admin powers to edit that comment for that purpose. However, having “both locations” be on github makes it a lot harder to make proper threading a reality, so we might want a separate forum anyway.
The two-thread solution is a form of gatekeeping. Basically every common criticism of gatekeeping will, thus, apply. The summary will be seen as propaganda, especially since the gatekeepers and the decision makers are part of the same organization. It also still requires someone in the gatekeeper class to read all of the public thread.
1 Like
I do feel like that’s mostly mitigated by having people in the free-flowing area striking out their post as soon as they feel like it’s been addressed in the summary (ergo you can look at the freeflow thread to easily see what hasn’t been summarized well). But you are right, it’ll still lead to contention is my guess.
Also, sorry for derailing this thread - it was mostly about the article and then I led us off into a tangent about categorizing ways to reduce comments… worth discussing, but maybe not here.
I might be wrong, but I think this is the right place for it because the post is about scaling empathy, and the volume of comments (especially unhelpful ones) makes it far harder to scale.
I also kind of liked the strawman, but I think the primary danger is that the second thread (e.g. irlo) becomes the unproductive thread because anyone can post there. Effectively, the barrier for posting junk in that thread becomes even lower, except now some poor facilitator must read it.
I do really like this observation, though:
I'm a bit worried about equating organizational empathy with decreasing the number of people represented in processes. It doesn't necessarily equate to number of perspectives lost but it's not too far removed. Further, IMO if we add tiers of participation we should do so in a way that is prepared for the iteration needed for participation to grow 10x in the coming years. I don't think that incrementally closing the process off is in the spirit of our community, nor will it ensure the needs of everyone are represented as the mix of community participants changes to include more representatives of "socioeconomic power" (companies, governments, etc) over time. To put it a bit strongly, I'm convinced that we need to find actions we can take to stay as open a community as possible while also succeeding to grow the community and overall adoption.
Another way to put this is that we cannot put the genie back in the bottle. Sharding decision processes only works when there is good coordination and stakeholders feel adequately represented. If possible we should be discussing how to give ourselves tools for ensuring good coordination and representation ("empathy") rather than ways to keep our discussions small.
This forum is also subject to some selection bias - irlo is generally a place for those who perceive themselves as contributors in some way already. Would additional layers of threading help us maintain access to the perspectives of new users of the language as we grow and the language changes? How will a more closed process hear about the needs of people who have not previously been in scope for our marketing, like first year CS students?
This post is rambling quite badly right now, but there's another question I'd offer: "how can we avoid pulling the ladder up after ourselves on rusts process inclusiveness?"
4 Likes