Compiler "Steering Meeting"

#41

We didn’t really get very far into what that agenda should be. I have definitely been thinking a lot about the RFC / stabilization process, but mostly I guess from the perspective of the lang team. I am very interested in trying to do more “holistic” planning, so that we are not working on language design questions without some idea of who will implement them, and without a rough stabilization plan (I’d like us to work more serially, rather than the current scheme where we just have a series of pipeline stages with unbounded queues that people keep tossing things into willy nilly).

So, definitely a crucial bit for me is the interaction between the lang/compiler team. On the one hand, I think that the “what it ought to do” and “how to do it” are definitely distinct concerns, but of course there is also feedback both ways between them. I think what probably makes sense is that when we are in the implementation stage we kind of have two “tightly coupled” working groups with overlapping members working in concert.

All that said, I wonder if this particular part discussion ought to be had with both lang + compiler team in concert. Interesting question.

5 Likes

#42

I am curious what came out of the all-hands meeting last week. Are there meeting notes or something? Any notable decisions?

4 Likes

#43

We have a steering meeting tomorrow. Here are is an array of possible topics that have come up lately:

  • How to integrate “future compatibility warnings” into triage to ensure they “move along” at a more regular pace
  • The roles and expectations of “journeypeople” and “full members”

There are also a few things that came up that might fit better as “announcements”, although maybe there is enough to discuss:

  • Projects looking for support, are people available?
  • We’re trying to build an “expert map” showing the various parts of the compiler and who knows about each one in the Meta WG, and would like people to take a look (no link yet, will be coming soon)

Other suggestions, announcements, etc?

1 Like

#44

Another idea: how to manage the steering meeting agenda more efficiently =)

1 Like

#45

Something I’d like to have soon is some kind of written-down policy on how handle out-of-tree crates hosted in github/rust-lang. That’s probably a topic for a steering meeting (but doesn’t have to be this one)?

0 Likes

#46

Good suggestion. In our “meta WG” meeting yesterday, @davidtwco volunteered to go over our notes on that topic from the All Hands and create a summary, so maybe we should wait until that is done and we have a good starting point.

Correction: @davidtwco is very fast. They’ve opened a PR already.

1 Like

#47

I created this HackMd document to accumulate steering meeting discussion topics in between meetings. I’ve added the things from this thread so far.

1 Like

#48

2019.03.01

As always, these minutes are also available on the compiler-team repository.

Zulip topic

Before the meeting started, @nikomatsakis proposed an agenda and linked to a document to collect agenda ideas for future meetings.

Also, @mw asked about discussing the policy for out-of-tree compiler crates (expanding on discussions from the All Hands’ organizational meeting), a draft for which was already in-progress by @davidtwco.

The meeting commenced with a call for involvement in the RLS 2.0 working group led by @matklad as sharing expertise between the RLS 2.0 effort and rustc would be helpful. There is documentation (videos and guides), issues with E-mentor and other resources available for those interested. @davidtwco, @pnkfelix and @Xanewok mentioned that they would endeavour to find some time to dig into RLS 2.0.

Next, @nikomatsakis announced an ongoing effort by @spastorino to collate an expert map about who has expertise in which areas of the compiler, with the intent of making it easier to find the right people to ask questions and assign issues/PRs to, and invited everyone to help flesh it out and add themselves. @nikomatsakis also suggested that this would be good in documenting the overall structure of the compiler in some ways.

Following this, the main topic for the steering meeting, the out-of-tree crates policy was discussed (for things like polonius, chalk and the upcoming diagnostics refactoring). The current draft of the policy was linked and @davidtwco emphasised that it was very much a draft and there are lots of unresolved questions.

@nikomatsakis highlighted a desire for having a uniform experience between out-of-tree crates and normal in-tree development - particularly surrounding who can land PRs, highfive reviewer assignment, and ensuring anyone in the team can publish new crate versions.

@davidtwco noted that discussions surrounding tooling that could make out-of-tree crates easy to work with was left out of the policy draft as that isn’t part of the policy.

Both @nikomatsakis and @mw wanted there to be a simple guide to follow for creating out-of-tree crates - this should include links to examples and things to copy-and-paste.

@pnkfelix raised concerns about current policy wording suggesting that only general-purpose functionality should be factored out and suggested clarifications about the minimum bar for expected re-use might be beneficial. @nikomatsakis mused that making libraries too general purpose could carry a higher maintainance/breaking-change burden. @mw suggested that this would need to be decided on a case-by-case basis. @nikomatsakis clarified that we should absolutely be following semver in these crates.

@nikomatsakis then raised the question of review policy in out-of-tree crates - should major PRs require multiple reviewers? can crates have reviewers that are not reviewers of rustc? Going to on say that the idea of these crates having their own communities is appealing and that one option might be that if a contributor was involved in crate sufficiently to review it, then perhaps they should get rustc r+ too or that out-of-tree crates might offer a way to do more fine-grained notions of r+. @pnkfelix suggested that long-term we might want in-tree tests of these crates to guarantee that they maintain compatibility.

@mw then proposed walking through an example - the self-profile working group will likely create a mildly sophisticated on-disk string table that will be shared between rustc and external tooling created by the working group and that this would be an ideal candidate for an out-of-tree crate.

This raised the many questions. Do compiler team members have the rights to create repositories in the rust-lang organization? Is it best to start in the rust-lang organization or prototype in a user-hosted repository? What do we name these crates? Is there guidance on adding crates.io dependencies to rustc?

@davidtwco suggested that we would need to collate a list of things that would need to be added to a repository (licenses, code of conduct, etc.) and things that would need set up (bors, highfive, etc).

@wesleywiser mentioned that while anyone on the compiler team should be able to r+, only those actively involved should be on the highfive rotation. @nikomatsakis noted that it would be great for active working group members to have the opportunity to gain r+.

@mw asked if we would set up community infrastructure (such as Zulip instances or streams) for the out-of-tree crates, but @nikomatsakis thought it best that this be tied to the community infrastructure already in-place for the working group.

Then, @nikomatsakis re-raised the question of whether these out-of-tree crates are best kept in the rust-lang organization. @mw suggested that this is best as it simplifies bors and highfive integration and would use existing team permissions. @pnkfelix asked how challenging migration from a personal repository to the rust-lang organization is. On this point, @nikomatsakis said that transferring is easy but messy - transferring can leave a redirect behind which will conflict with the default name of a fork, which requires an email to GitHub to resolve.

@davidtwco interjected that it should be made clear what team and working group is responsible for any repository we create in the rust-lang organization. @nikomatsakis suggested a template for the README of out-of-tree crates would be valuable and should point to the responsible working group.

Changing gears, @mw asked if there were any example workflows for handling changes in external crates to which @nikomatsakis suggested that a rustc-guide chapter on that and how to make local copies for debugging would be necessary. @davidtwco asked what information should be kept in the policy document and what information should be kept in the rustc-guide. There was some consensus that setting up a new out-of-tree crate is policy and working with one is for the guide.

@mw then raised the question of where consent of compiler team members should be gathered before creating new out-of-tree crates. @pnkfelix suggested it could be a triage meeting item and @nikomatsakis suggested using @rfcbot and that the general presumption would be that if a compiler team member wanted to create a out-of-tree crate then it would probably be a good idea. @mw mused that something asynchronous would be preferred and that agreement within the working group itself should come first. @nikomatsakis suggested that the goals of approval should be discussed, such as helping to pick a suitable name.

@mw suggested that a list of these crates would be valuable and there was agreement that it should not be in rust-lang/rust to avoid the bors queue. There was some discussion about searching for issues across repositories in an organization.

@davidtwco then agreed to write these minutes and his hands are starting to get tired at this point.

5 Likes

#49

2019.03.22:

Zulip topic.

Announcements

  • oli-obk mentions that wg-diagnostics is planning to use the annotate-snippets crate within rustc.
  • nikomatsakis is planning, as part of the @WG-meta working group, an RFC that will propose more intermediate levels for the compiler team to recognize committed, regular contributors.
  • Zoxc almost has a series of PRs pushing queries all the way back to parsing.
  • davidtwco has, after numerous attempts, managed to produce a have a working branch that unblocks RFC 2008.
  • pnkfelix applauds oli-obk’s fixmeh project, which lists all the FIXME comments in the compiler.

Main topic

Our main topic of the day was enumerating the needs of the compiler team:

Can we try to enumerate what are some of things the compiler team needs to be tracking? What communication is (or should) be happening? I’d like us to be able to come up with a good list of all the things we think should be happening, so that we can compare against our existing processes, and try to ensure they are covered (or adjust the processes as needed).

We produced a paper document that contains a pretty comprehensive listing, sorted into categories.

After other topics that came up along the way:

3 Likes

#50

2019.04.12

Zulip topic.

Announcements

Main topic

Our main topic of the day was a proposal to add a regular design meeting:

So this was our plan for the main topic today:

The goal of the proposal was to have a central place where the team can review designs and have design discussions. These discussions could have a few forms:

  • In some cases, people might bring a thorny problem, looking for help in finding a design. (Example)
  • In other cases, people might be bringing an existing idea for evaluation and broader discussion, looking to reach a conclusion. (Example)

In each case, there will be a proposal describing the question at hand and the goals of the meeting. The expectations for this proposal would vary depending on the kind of discussion. For “early stage” discussions, the proposal might be fairly sparse. (Perhaps like this example.) For meetings that aim to reach firm decisions, the proposals would be expected to contain more details; it would be useful to list things the author expects are hard to do and also points where there are design decisions to be made.

The expectation is that these meetings will help us propagate knowledge of what is going on through the project; they will help people get feedback on designs (right now, it can be hard to get people to make time to really talk out a design). Finally, they can help newcomers to get more familiar with things, since people can lurk in the meetings and soak up details (not to mention the paper trail).

While the basic idea of the proposal was popular, we wound up changing some of the details for how it would run. We settled ultimately on reusing the slot for the existing steering meeting (Friday mornings, Boston time) but making it weekly.

Every Nth meeting (likely 3 or 6) would be a designated steering meeting. The role of these steering meetings it so select the next N-1 topics that will be discussed (from a list of proposals). In between meetings, people can add meeting proposals to the list for the next round.

When we make these decisions, we would be trying to take a “global view”. In particular, we don’t want to be doing designs unless there are people who will put them into practice. This probably means that we will prioritize design questions arising from existing working groups, but we might also take up design questions that might become the basis for a new working group, as long as there are people who would want to take part.

We also spent some time discussing the idea of a mentoring or ‘internship’ program for helping “frequent contributors” make the jump to “compiler team member”. The designs that we discuss in this meeting would hopefully wind up detailed enough that frequent contributors can pick them up and implement them, even if the people making the design are busy with other things. Similarly, design documents can become the basis for rustc-guide chapters.

We plan to flesh out some more details and put this idea into action starting May 3rd (the next steering meeting).

6 Likes