Thoughts on RFC/stabilization reform in 2019

@Centril Thanks for point me to that reddit thread.

I would say that I also don’t fully agree with the everything Graydon says about negative space. I think we should make sure that Rust is relevant to whatever the current state/needs of technology are, especially in such a fast-moving field as computer science

However, I do see a need for temporary negative space: ruling out an area of exploration for a time (e.g. an edition) to be reevaluated later. In particular, I think that would allow us to focus community efforts and improve scalability.

We have tons of work in progress and not nearly enough bandwidth to keep up; tons of stuff is semi-perma-unstable because it’s just not getting worked on. We have 344 open tracking issues (https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3AC-tracking-issue), and as of now, all but about 30 haven’t been touched since October 2018. Half haven’t even been touched since 2017.

I don’t think temporary negative space will hamper Rust’s growth in the future, but I do think we need to make some hard choices about where to focus efforts. I think that necessarily means we will need to make some sacrifices and some will be disappointed (I’ve been on the receiving end of that too sometimes).

See, I don't think it will; rather, it will just waste time on deciding on what to not consider for now and so on. We also have mechanisms today that allow for a measure of negative space: closing RFCs, postponement, and the roadmap. The latter already states what the major priorities are, and other things are by definition not major priorities (there will always be small things that don't take a lot of time that we can work on...). I think this works well. It's better to positively state what we believe is important right now rather than stating what isn't.

At the time of writing, there are 124 open T-lang C-tracking-issues (I cannot speak for other teams... not my business...). I have reviewed all of these 124 issues several times. Many of them are small things, others are old crap we want to remove but cannot yet because e.g. servo depends on it. Others are waiting on architectural changes like Chalk or on making NLL the default. Some things were made very recently to make tracking more fine grained. Some things are really small issues that cannot and should not be compared with bigger things. Other things are done or close to completion.

This is all to say that using this list as a measure of things to do is a extremely blunt instrument that doesn't account for differences and the weight of various items in the list. Many things in this list are being actively and quickly worked on. Negative RFCs and whatnot will not help, it will just give me more work and take time away that I would like to devote to this list instead.

The nice thing about our current process is that it deals with priorities, backlog, and bandwidth organically. Language team meetings start with going through triage (nominated issues, PRs, bugs, etc.); we then proceed to RFCs... If we don't have time for RFCs, it will just wait until we have time. This happens a lot; most meetings we don't discuss new RFCs.

3 Likes

Community efforts include more than just the efforts of the official teams.

Aside: I'm going to be use quotes (") in this post to differentiate between people belonging to teams ("team") and those who aren't ("community") - in practice team members are part of the community, and it's more about differentiating those keeping very close tabs on wide part of rust development/RFCs vs those who are less acquainted with what's going on.

When I hear "focus community effort" I don't think of the teams, I think of "community" attention - involvement in RFCs from people not on the official rust teams: reading the huge comment threads, adding comments, critiques and feedback, trying out the unstable features, etc.

The organic attention of the "community" often does not line up with the attention of the "teams". Consider RFC's written and postponed, or RFCs that garner lots of comments without team attention. I think in part the drift between "community" and "team" focus has gradually grown as the community has gotten bigger. It's easier to follow what's going on when less is going on.

I think negative RFCs would be useful for "community" involvement - I think that with negative RFCs/space there would be a fair number of RFCs that would be unwritten, instead of written and then postponed. Knowing something won't get much attention beforehand is pretty valuable. And it's hard from a community perspective to know what that is.

I think negative RFCs / negative space is only one approach to this though, so don't take this as endorsement.

I do think in these discussions it's useful to keep in mind that that what helps "team" members might actually inconvenience "community" members, and vice-versa.

TL;DR - "team" members and "community" members interact differently - I think the negative space RFC stuff was for the benefit of "team" members, not "community" members. I think it's important to consider both though.


Personally, I'd go for more tooling making it easier for team members to mark where their attention is, and aggregating it into an index / portal that makes it easier for "community" members to see where attention is allocated.

Also, I'd gladly accept a slower pace of rust language development for better rust community involvement.

2 Likes

Also, @Centril, thank you for your comment. I got a better insight into how at least one team operates, and it gave me lots of food for thought.

1 Like

I agree that it handles priorities, but I’m not convinced that it handles backlog/tech debt, at least not as much as I would like to see (and others seem to agree judging from Rust2019). Also, a huge portion of technical debt is from non-T-Lang work, but T-Lang work still consumes community attention/effort/resources that otherwise go to other tech debt; there’s only so much that can happen in parallel. The systems we have should consider the resources of the entire community, rather than being on a team-by-team basis IMHO.

Also, to be clear, I’m not bashing on T-Lang or anything. I’m super appreciative of the team’s efforts and work and your persistent triaging.

I think an interesting alternative to negative space might be “positive space with guard rails”. For example, we define a roadmap and only accept RFCs for roadmap items.

1 Like

What I'm not convinced of is that it could go much faster than it is right now. Some of the tech debt is from all the way back in 2015. It won't be solved unless we land Chalk and such things and only so many people can work on that at the same time; the speed with which we remove technical debt doesn't scale linearly with the number of people willing to do that work. We are also limited purely by the bors queue, even if we had more people working on it the PRs might not get merged any faster. I want to see concrete evidence (not just abstract arguments) that refocusing will substantially solve the technical (design) debt quicker.

T-Lang work, for example on designing associated type defaults, type ascription, already is focused on fixing technical debt to a large extent. Some technical debt or unfinished things can only be solved with new language design (e.g. I'd be uncomfortable with stabilizing much of implied bounds without where clauses on enum variants, or trait aliases without quantification over traits...). There is other types of debt, e.g. lack of documentation about the compiler -- but that is not something I could substantially help with since I'm not a compiler dev and don't know all the components. Having new designs to implement are also a way to get people into compiler work and moreover, I think that a consequence of less design is just fewer people working on things -- it won't result in all language designers suddenly doing more compiler work.

Thanks for sharing this reddit thread @Centril, it was a very interesting read. I really appreciate all the conversation going on here.


I'm not completely convinced that the language is growing too complex, but I can understand those concerns and I think that it's important that this is always considered and that the language team consists of individuals who have different opinions on where the line is (this isn't to suggest that isn't the case currently, I'm very appreciative of all the hard work that T-Lang do).

However, I do think there is a problem with how challenging it is to follow the changes being proposed to the language (even at a high-level) and how quickly these discussions become draining (even just to follow along, I can't imagine trying to participate). I spend probably an above-average amount of time following Rust's development with my semi-regular contributions to the compiler and if there's something I'm interested in keeping a tab on, it just isn't feasible (eg. I couldn't keep on top of the await syntax discussion this weekend). I think it's important for the community to be able to keep up (with some reasonable time investment).

I don't think that the issue is that there are too many RFCs. I think the issue is that the discussion on these proposals often end up large, sprawling and unproductive.

In my opinon, the discussions on RFCs quickly become unprofessional (definitely not everyone), propose often inpractical solutions (eg. things would need RFCs themselves) or attempt to relitigate already settled discussions.

One of the great things about Rust is how transparent and open the processes are, and how the community around the language can get involved - but I do feel the bar for participation in RFC discussion needs to be higher - there's a really high cost to commenting (which is why I very rarely participate on RFC discussions myself - I'm in no way qualified and don't want to introduce noise).

I would love to be able to look at an RFC proposal and see mostly discussion from the Rust teams, some regular contributors and the occasional community member where all comments are professional, considerate and contribute to the discussion meaningfully. I'd like to walk away understanding what different members of the team think about a proposal and having read a really good discussion (like that happening in the previously linked reddit thread or in this internals topic).

There's just too much noise right now, there should be much less discussion than there currently is - that isn't to say that the topics aren't important and don't warrant discussion - they absolutely do - but a handful of productive and professional individuals could discuss a topic as thoroughly in thirty comments as we currently do in hundreds.

I'm concerned that the team members doing a great job could get burnt out and stop enjoying the work they do on Rust if some changes aren't made to these processes - and I think that's a concern shared by many given all the Rust 2019 posts that hit on these issues (although they may not see things exactly as I do, that's fine).


That's just my 2c, some (ironically) half-baked thoughts - it's not intended to be a criticism of any team or individuals, or to suggest that we do anything radical to limit discussion and shut down any opinions. I also recognize that this post isn't particularly useful as I've not proposed any particular solutions, just enumerated the problems as I see them.

9 Likes

This is fair. I can't really give authoritative statements for most things, but I can give my observations for the parts of the project I know.

I firmly believe that there is lots of work that just hasn't gotten any love recently due to a mix of bandwidth constraints and lack of attention. Refocusing community attention and effort at these would accelerate development.

Here are some examples that are closer to me (and I suspect others could point out more work):

Note: I'm not complaining. Just pointing out that we already have lots of stuff we can work on that is just wanting community attention.

  1. Documentation:

    • The rustc-guide has tons of available work and not enough helpers. Most of this work could happen in parallel. Also, some of the last missing chapters (e.g. LLVM) kind of require a knowledgeable/experienced compiler dev to write them or mentor someone writing them, but those people tend to be way oversubscribed and too busy to help (and understandably so).
    • RBE has "languished"
    • The cookbook was not finished
  2. Design and implementation of old unstable features that we already know we want: much of this work is undone due to bandwidth, as well.

    • Things like inline asm, custom testing/benchmarking frameworks have all had some discussions 1 2 3 4, but my strong impression is that there are not many people available to work on them.
  3. Much tooling could use more love.

    • cargo and rustup have been frozen for the time being due to bandwidth constraints and planning needs. cargo has 712 open issues, 84 of which are labeled C-bug, along with 18 PRs, the oldest of which was opened in March 2018. The rustup repo doesn't even appear to be triaged regularly, and my own experience of trying to add a useful note to the README saw the PR left untouched from Nov 2017 to May 2018.

    • RLS/IDE integration is a field where a lot of work is going on, but my impression is that it is largely due to the heroic efforts of a few dedicated individuals like nrc and matklad. My own experience with RLS is that it's not quite there yet, and many of the Rust2019 posts also mention it 1 2 3 4 (<- there are others, but these are the first few on Read Rust)...

    • There have been some interesting experiments 1 2 aimed at improving rust documentation and code search, but these seem to be primarily the work of a small handful of people. rustdoc itself seems to be mainly managed by a handful of people.

  4. I would also like to point to the impl Period of 2017 as a demonstration of what might be possible if the community attention is directed in a given direction. It was highly successful: the ergonomics initiative, libzblitz, and working groups made significant progress during that time 1 2.

For the compiler I can't call the "impl period" 2017 successful at all.
Do you remember what happened due to the pre-impl-period RFC deadline? I remember.

Last month(s) before the impl period:

  • Compiler contributors - work less than usual due to constant distraction by the flood of pre-deadline RFCs.
  • Compiler non-contributors - write RFCs as if it's the last day of their life.

The impl period itself:

  • Compiler contributors - work as usual.
  • Compiler non-contributors - relax on the pile of RFCs.
2 Likes

Ideally, negative-space would prevent the flood of pre-deadline stuff.

The other point about the impl period itself seems more salient - as Centril as said about himself and his team, (and according to you applies to most (if not all) compiler contributors): compiler contributors already know what they’re working on and what comes next (and they don’t want more busy work).

But that’s just the compiler team. I think guard rails or guidance of some sort could help direct people to areas of rust that are more … in need of love.

Well, I definitely remember that there were too many RFCs running at the same time, but I see that more as a reflection about setting hard deadlines for things, rather than doing them when they are ready. I think a similar observation can be made about the Edition 2018 rush (e.g. the website stuff).

1 Like

I’m not very convinced that negative RFCs would be received well by the community. In order to serve their intended purpose they have to be respected. And there hasn’t been a great track record of that happening. The crates.io team made a “negative policy decision” regarding crate squatting, but since then there have still been a bunch of contentious debates on that topic…

4 Likes

You mean that it would be perceived as a dismissal of some sub-community’s concerns?

Do you think a positive roadmap with guard rails would alleviate some of your worries?

What the language team can help with here, and in particular with respect to the reference, is to require more elaborate stabilization reports so that it becomes easier for documentation writers to know what they are writing about. This is something I know the docs team feels and as a T-lang member I want to help facilitate things here to make parallel work possible.

The theme here is that I firmly believe that bandwidth and work done is not a linearly scaling resource -- you cannot just refocus community attention arbitrarily to fit your desires; people have different skills, know different things, are interested in different things, etc. We should try to unblock and allow more people work on more things rather than to stop working on language features. In fact, new language features may onboard new compiler developers.

I would also say that from the POV of making rollups, the PRs I see are mostly about improving diagnostics and improving perf.. I don't think implementing new features is the lionshare of where review time is being spent.

  • https://github.com/rust-lang/rfcs/pull/2360 -- this just needs someone from T-libs or T-compiler to take the initiative and fcp-merge or something; the RFC did see ~recent activity and moved away from a position which would have required T-lang approval to not requiring that... Implementation-wise this one is pretty easy... it mostly requires exposing an intrinsic + adding some docs (no more than the usual smallish libstd feature). This is currently blocked on "do we want this?".

  • inline assembly -- this is not accepted as an RFC and definitely not something "we already know we want" and the language team will likely be cautious here with any RFC.

  • custom test frameworks -- I think your strong impression is wrong; in fact, the internals link shows otherwise imo. Rather, I think what has been missing here is frameworks such as quickcheck and proptest actually trying these out to ensure the design is up to snuff.


My view is that a positive roadmap is good since it says what we want to do; for the other things, we should communicate more clearly that they won't be priorities, but also not forbidden. In my view, it is fine for an RFC to not see much action (e.g. a few comments and no team review) for a whole year -- when our priorities change, it is there and more polished.

I agree with this, but my conclusion is opposite: because resources don’t scale linearly, unblocking more work doesn’t get more done; it just creates more tech debt.

I’m not arguing that we should make everyone work on the same thing or that everyone has the same strengths/interests. I am saying that I feel the community is stretched too thin and would benefit greatly from focusing on a smaller set of work items.

I also suspect that we have a difference of experiences (possibly because I’m not on a rust team?) that leads to different impressions about capacity and effectiveness of the community. I’m not really sure how to reconcile these views. Perhaps the existing processes work fine and what’s missing is some way of giving everyone else the same perspective as the rust teams? It doesn’t seem that way to me, though...

I think part of the disconnect is a perception of what’s “in progress” from “inside” versus “outside”.

Just as a minimal example, the impression of the atmosphere of interest is definitely different if you primarily consider internals, the RFC repo, the bug tracker, or just TWIR. Then there’s the slightly more closed (feeling) team processes: obviously not everyone’s present for steering/triage meetings so what’s covered there, while potentially guiding what that team considers “active”, has little to no effect on “outside” atmosphere.

It’s similar to the “pre-RFC flood” that can be felt here on irlo. A lot of medium-baked (or less) ideas get thrown around here but never make it to the RFCs repo, due in part to both a gating of thought provided here and a lack of interest in putting forth the effort to push it forward. RFC (issues) that don’t see team interaction are similar in that if they’re open and “community effort” goes into them, the “atmosphere” is that they’re “open” and taking some “cost” to host (even if that’s just real-estate on a digital issue tracker) even if the team isn’t going to look at it for a year due to other priorities.

This isn’t the entire reason for this “flooded” atmosphere, definitely. But I think at the very least it is a part of the disconnect leading one party to say “the system is working” and the other that it isn’t in respect to managing the flood of opinions that public consensus-driven design is.

(off-topic EDIT: I had no idea irlo had cake day icons, this is the first time I’ve seen them (on me).)

6 Likes

Summary

Things appear to have settled. I wanted to summarize the discussion a bit, since it would be ironic if I didn’t.

Meta-note: I tried to be “complete” and “concise”, erring on the side of “complete”. It’s kind of hard. Let me know what you think…

OP

  1. In the OP, I proposed a number of ideas. These don’t seem to have generated much controversy. My impression is that many people agree with them. Some also seem like incremental enough changes to be done without an RFC.
    • Centril points out closed/postponed RFCs are already easy to find; it’s just a matter of expending the effort to read them.
    • skade argues that adding automation without other changes will cause problems to get worse.

Thread summaries

  1. cartesiancat Posted some thoughts on how to implement good thread summaries for long discussions. Here are the high-level points (please point out if I missed anything):

    • Summaries should be “complete, clear, concise, and non-redundant”; “optimize for reading”
      • There was some discussion about what “complete” and “concise” means. scottmcm suggests framing it similarly to the Agile Manifesto, so I proposed “While we value concise and complete summaries, in situations where they conflict, I think we should completeness.”
    • cartesiancat suggests that if a summary comment would be too long, a sentence with a link to the original comment would be acceptable.
    • rfcbot used to update the top post or make a new post, ensuring everyone gets emailed. rfcbot collapses comments between summaries.
      • pietroalbini says there is not GitHub API for collapsing comments at the moment.
    • Frequency of summaries: we should define and enforce how frequent they should be.
      • I originally proposed every 30 comments, but there are other metrics, e.g. once per unit of time or once per unit in time in which there are at least N comments.
      • Another problem (pointed out elsewhere) is that initially there are usually a bunch of typo comments.
    • Who writes summaries
      • In a previous thread, I proposed that it should be a lang team member. cartesiancat proposes it should be the RFC author.
    • “Rules about comments should be strict and enforced”. The idea is to reduce noise and redundant commenting.
  2. I implemented a prototype in rfcbot…

    • varkor points out that it might be cleaner to have rfcbot make the first post to each issue and update that as the summary comment.
  3. CAD97 points out that summarizing a bikeshed is hard, especially when going for completeness.

Unintentional Bias

There were a few discussions that arose around how to reduce bias.

  1. cartesiancat proposed a “proponent/opponent” model, in which each summary should be written by two people: one in favor of the RFC (presumably the author), and one against the RFC, playing devil’s advocate.

  2. josh points out that proposing an FCP with a given disposition at the beginning gives that disposition “inevitable momentum”. Instead, they suggest that each team member should list their disposition and let discussion come to a consensus from there.

Deciding on Problems

  1. matthieum suggests that we should amend the RFC process to first formally acknowledge a problem using a “Request for Proposals” process. The intent is to reduce discussion of problems that may not be widely felt to be a problem.
    • Ixrec responds that “in practice it’s often impossible to decide “this problem isn’t serious enough to warrant a language change” without knowing what the language change to solve it would actually look like”, but agrees that avoiding debates that don’t need to be had would be good.
    • fintelia worries that such an RFP process would identify difficult or impossible problems and devolve into discussions about the existence of some unlikely solution.
  2. I raise the question of how eRFCs and roadmaps tie into this problem. In a sense, they are about defining problems that the community finds important.

Negative Space, Exhaustion, and Resources

There were much discussion on exhaustion and negative space. skade first brings up the topic with regards to heated discussions. I later turned more towards prioritization and the use of community resources and effort.

  1. skade brings up the topic of “negative space”. (To my knowledge, this was first brought up by Graydon in their Rust2019 blog post).

    • (@skade: I tried to distill the main points, please feel free to correct me if I missed anything)
    • They point to the recent “Turbofish” and “await syntax” RFC discussions to illustrate some points.
    • They suggest “more roadmapping and sticking to it”. They also suggest creating mini-roadmaps to elaborate more on specific areas.
    • They suggest some ways of managing discussion threads: “call project members out more harshly if they aggressively discuss”, “actively pulling in people from relevant groups early”, “Frequent rechecking if we don’t go off track by absolutely wanting to solve a hard problem just for the sake of it”
    • In the case of heated syntax bikesheds, like the await one, the design space has become enormous and the discussion is exhausting.
      • Debate includes ideas that would need their own RFCs.
      • Pressure is high as the feature is in high demand.
      • The implementors often feel unappreciated because of the high rate of new ideas.
      • skade suggests listing the available options up front and limiting debate to those options. They also suggest raising effort the bar for commenting and prohibiting proposals that would require their own RFCs. They suggest that arbitrage and credit assignments should be made up front.
  2. skade points out the need to deal with “emotional real estate” (as defined by anp).

    • They point to “turbofish” and “await” syntax as examples.
  3. I raised a few questions about how we would implement negative space:

    • how to evaluate if a proposal falls into the negative space and should thus be closed?
    • how to prevent a flood of negative RFCs and thus waste time anyway?
    • how to prevent stagnation?
  4. Centril points to this useful discussion about negative space between Graydon, Centril, and others.

  5. Centril and I discuss temporary negative space and focusing of community resources in these comments: 1 2 3 4 5 6 7 8. A few interesting realizations came out of this discussion for me. Perhaps @Centril can point out if I missed anything or mischaracterized their views.

    • I feel that I spend above-average time on rust stuff, but still feel that the pace of change is overwhelming and that much technical debt has accumulated.
    • Centril (as a member of the Language team) finds that technical debt related to that team is triaged and handled regularly and not a source for concern. Moreover, much of the technical debt is blocked on items that are actively receiving attention.
    • I expressed that work on T-Lang items still consumes community time and attention away from other items and that refocusing community attention on a smaller set of things would improve productivity.
    • Centril disagrees that work could proceed much faster and that refocusing attention would have little effect because of difference in skill sets.
    • Centril argues that negative RFCs would just create more work for the language team.
    • Centril argues that much of the design work happening is technical debt and simplifies the language overall.
  6. davidtwco suggests that the pace of change is not the problem. Rather, the ability of the community to keep up is the problem.

    • They suggest that this is due to “large, sprawling, and unproductive” discussions and that " the discussions on RFCs quickly become unprofessional (definitely not everyone), propose often impractical solutions (eg. things would need RFCs themselves) or attempt to relitigate already settled discussions." and that “There’s just too much noise right now”.
    • They suggest that the solution is to raise the bar because “there’s a really high cost to commenting” and that “a handful of productive and professional individuals could discuss a topic as thoroughly in thirty comments as we currently do in hundreds”
  7. fintelia suggests that a negative RFC would not be well received and would generate even more contentious debates.

  8. Postitive roadmapping and sticking closer to the roadmap or some for of “guard rails” to prevent discussion from straying to far seem to be suggested often as an alternative to negative RFCs: 1 2 3

  9. Regarding refocusing effort, I suggested something like the impl Period of 2017.

    • petrochenkov recalls that this led to a pre-deadline flood of RFCs which was hard to handle. I claimed that this is a reflection on setting hard deadlines more than the impl Period itself.
    • samsieber suggests that negative RFCs might mitigate the pre-deadline flood.

Team and Community Perspectives

Another (unexpected, to me) theme was the difference in perspectives from the language team and the rest of the community. In particular, these seem to indicate to me some communication disconnect.

  1. Centril’s comments in the discussion above seem to focus largely on the view of the Language team (understandably). My impression is that team is pretty organized and does not feel an excess of tech debt. (TBH, I wasn’t really sure if this was the view of the whole team or just Centril as a member of the team?)

  2. samsieber suggests that “The organic attention of the “community” often does not line up with the attention of the “teams”.” and that negative RFCs might help with that but aren’t the only way to accomplish it.

  3. CAD97 notes “part of the disconnect is a perception of what’s “in progress” from “inside” versus “outside””. Specifically, team triages and decisions affect what’s worked on, but not what the community perceives as “in progress”:

    • “obviously not everyone’s present for steering/triage meetings so what’s covered there, while potentially guiding what that team considers “active”, has little to no effect on “outside” atmosphere.”
    • “the “atmosphere” is that they’re “open” and taking some “cost” to host (even if that’s just real-estate on a digital issue tracker) even if the team isn’t going to look at it for a year due to other priorities.”
  4. samsieber suggests that part of bridging the disconnect might be better tooling: “I’d go for more tooling making it easier for team members to mark where their attention is, and aggregating it into an index / portal that makes it easier for “community” members to see where attention is allocated.”

Unresolved Questions

  1. How to enact changes? One big reform or incremental progress?

  2. Do we want negative RFCs? My impression is that the leaning on this thread is “probably not” but we want more “guard rails” or something similar, though there are several who have expressed desire for negative RFCs. (Personally, I’m on the fence).

  3. I certainly missed a bunch, so please just ping me, and I will add them

7 Likes

My immediate reaction is: That’s not so much complete as it is comprehensive. I don’t think we should expect or aim for the latter in summaries. In fact, one of the most important parts of a truly useful summary is to completely leave out details that are not significant (which is also why they can be contentious…), rather than forcing every reader to decide what is and isn’t significant.

(sorry if that sounds dismissive of all the hard work you put into that post!)

4 Likes

Yeah, it occurred to me that the post was rather long, but it turns out to be rather hard to choose things to omit… Any tips?

2 Likes

Maybe focus more on (a) things that seem uncontroversial, (b) things that are controversial, © a little exploration of the different sides of the controversial parts, as in who are taking what sides and why might they have differing perspectives?

3 Likes