Compiler "Steering Meeting"

I’ve been thinking about this.

First off, what I don’t think we should shoot for is deep technical discussion. I don’t think a synchronous chat meeting is the best forum for that. I suspect that would work best either on internals or (better) at the Rust All Hands that is (likely?) taking place this February.

Here are two other ideas.

Idea #1. Do a “State of the Team” retrospective, where we survey and discuss what works and doesn’t work with our current setup. I wrote up a gist with a possible plan (credit to @ag_dubs for the basic structure). Note that time allotted exceeds 1 hour – seems like the first meeting might want 90 minutes.

Idea #2. Do a survey that mixes “retrospective” with “where to go from here” – basically asking something like:

  • Github Username
  • Role (member, contributor, general public)
  • Things you think work well with compiler team
  • Things you think compiler team should fix
  • Things you think compiler team should prioritize over the next year (pick all that apply)
    • Raw compilation speed (time to run cargo build non-incrementally)
    • Incremental updates and RLS integration
    • Feature work (chalk, const generics, polonius)
    • Other

We could then discuss the results of these and try to lay plans.

Thoughts?

1 Like

Oh, heck, let’s go with the survey. Here is the link:

Please answer by Thursday evening, so I can collate the results Friday morning (Boston time)!

2018-10-26

We had our first T-compiler steering meeting today! What follows is a summary of the major points raised during the discussion. If you’d like to read the detailed minutes, please see the Zulip chat log. I’ll insert links here and there into that log where appropriate, if you want to see the original comment.

To start, before the meeting we did a survey asking folks what they thought worked well and what needed improvement in and around with the compiler team. I summarized those results in a Google doc, which shows the things people wrote and how often (in the case of duplicates). We then voted on what to talk about (that’s what the columns with the x are).

In the end, we opted to discuss the most popular topic, which sort of subsumed a number of entries:

  • “how to plan what to do and expose our plans”
    • hard to answer “what to do next” or “how can I help” (8)
    • public roadmap (4)
    • hard to do high-level design + planning and not just triage (3)
    • no central place to get “big picture” of what’s going on (3)

(Originally, we thought we’d cover many more things, but that did not turn out to be true. =)

To start, there were a number of comments on the survey (and in the meeting) to support the idea that Working Groups are an effective way to organize, particularly for newcomers, but it works best if we focus the WG on a concrete topic (e.g., NLL). WG leaders ought to be actively triaging and looking for places to help people get involved, and so they can be good ones to answer the question of “what to do next”.

We can also use WG as an organizational tool to expose the “big picture” of what’s going on. In addition to WGs, though, it would be good to just list out the specialities of compiler team contributors, as well. WGs can also be an organizational axis for labels with the WG focusing creating a steady supply of E-mentor issues.

However, this places a big burden on WG leaders. One thing we noted is that having more than one leader is good and that this itself can be a mentoring opportunity.

In terms of getting people involved, the findwork site was raised, as well as the difficulty of keeping it up to date (perhaps WGs can help?).

We pointed out that when we do our planning at the next Rust All Hands, we can try to figure out WGs then.

We discussed some how the diagnostics efforts could be increased.

We discussed the rustc-guide as well as the NLL contributor YouTube videos and ways to improve them. One solid thought was that we might want to create a rustc-guide WG (which has its own zulip topic here). We talked in particular about how both experienced folks and newcomers can contribute there.

11 Likes

Today in the compiler team triage meeting, we decided to make the “Steering Meeting” every three weeks, at least to start. So a “twice per release” checkin.

That puts the next meeting on Friday, November 16 at 10am Boston time. If you would like to be added to the calendar invite, please let me know.

The next question: what should we talk about?

I was thinking it would be good to discuss compiler priorities for the next year a bit. I think it’s pretty clear that compilation time + fast, snappy RLS is going to be a priority, but I’d like to drill down a bit into some of the promising leads we have for that and what we can do to evaluate them – I’m particularly interested in how we can estimate the impact and choose relative priorities.

(I don’t expect us to reach final decisions here; but I see a need for us to start formulating thoughts in anticipation of the upcoming Roadmap discussion, and I would also like to start nailing down our agenda for the upcoming Rust All Hands in February – that is a great opportunity for us to finalize various architectural decisions in these areas, and I’d like to ensure we have all the data we need to make an informed decision.)

Thoughts?

One concern is whether we should maybe focus a bit harder – e.g., maybe it’s a good idea to discuss RLS specifically, or “overall compilation time” specifically?

3 Likes

This reminds me, there is a brainstorming document available here.

1 Like

Reminder, next meeting is this Friday:

I still hope we can talk about our priorities and planning for the next year. I'd like to do a few things:

  • Gather up all the major initiatives that are being contemplated, and ensure that they are represented in some way on my compiler planning diagram
  • For each one, identify:
    • Major steps or refactorings that ought to be represented themselves
    • Experiments we can do to get an estimate for how effective something will be (especially relevant to performance tasks)
    • Figure out places where it would be helpful to discuss in detail at the All Hands

It might also make sense to go through the All Hands Planning Document and try to prioritize and break things up. It'd be good to know which people ought to be a part of which discussion. As I recall from last time, it is pretty hard to solve the dependencies, so having some kind of input there is important.

Thoughts?

1 Like

Let me clarify what I am talking about. What I mean is, it'd be good to have meetings broken into "must attend", "may attend" to some extent, so that we can figure out which things can overlap etc.

But I'd also like to be able to answer the question "if we are only going to have time to cover 2 or 3 topics, which ones?"

In addition: what can we do now so that when we are trying to make plans, we have the data we need? I for example would really like to know more about the potential impact of various optimizations etc, as well perhaps as data on what scenarios we think are useful to which people.

Thinking more about what these online Zulip meetings are like, maybe we ought to focus for now on brainstorming. I imagine maybe:

  • are there improvements and ideas beyond the ones listed? can we get a complete list?
  • are there specific scenarios we want to optimize for? e.g., for mozilla or other major customers? how can we get something representative of e.g. dropbox, fuschia, and other major users?
  • what experiments can we do to to measure the effectiveness of each idea on each scenario?

Not sure.

I think that’s a good plan. A diagram like that is good for identifying independent task and possible interactions between seemingly independent tasks.

  • I think it would make sense for each major topic to be discussed and plan in some detail before the All Hands, that is, at least each topic that we might want to make implementation decisions for.
  • It would also be good to have a shepherd (team of shepherds) for each topic (or SCC in the diagram).
  • We should also have a table that shows for each area (1) the expected benefits and (2) the projected, transitive implementation cost. That should help with prioritization.

Yes, this.

Let me back up and just give a few thoughts about burnout and stress. I want to get to a place where we have, for each major initiative, one or two leaders. I also want those leaders not to be pushing on more than one big thing at a time, at least not as the primary leader. Speaking as someone who often finds themselves trying to push through a bunch of ideas at once, this involves a bit of stepping back. This is not always easy. But I think it’s crucial to growing the project and to avoiding burnout on the part of those of us who are heavily involved.

Keeping this rule in mind will put some limits on our ambitions. We can’t do everything, at least not at first. But that’s good and realistic, I think.

To use myself as an example, I think I would like to focus on the chalk transition and trait system overhaul (with @scalexm as a co-leader).

But then there is a tension: I’m not sure how to reconcile that with the Polonius and borrow checker work, which I also think is important. I would love it if we somebody else can step up as the “primary driver” for that work, and I can try to stay involved from a distance. (Or maybe it should just wait, which might be ok anyway, because to some extent it’s all blocked on enabling NLL for 2015 and so forth, and we just have to give time for that.)

(A separate question is whether Chalk is a good area to focus on. I continue to think it is, both because of the number of things that it unlocks, and because I think there is good potential for compilation time speedups. I also think that separating out and cleaning up the trait system will pay off in terms of aiding the RLS.)

11 Likes

Something else that is not represented on the diagram, but which I think is important, is figuring out how to manage the RLS and the rust-analyzer project. I continue to see promise in @matklad’s (and others’) attempts to design an interactive IDE experience from the ground up. OTOH, I continue to be concerned about supporting two compilers, and I recognize the value of a more incremental (no pun intended) approach.

I think in my ideal world we would focus on finding places we can cooperate: e.g., on paying down technical debt by refactoring name resolution and type-checking out of the compiler into a cleaner, more abstract codebase that can be shared. It’s not clear how to prioritize or enable that work though.

This ties in with more general questions of how to manage the compiler. I have to make sure that is on the list of All Hands questions. (Update: it is)

4 Likes

Personally I think Chalk is super important because GATs are probably by far the most important lang feature in the pipeline so I would encourage what you said. :stuck_out_tongue:

(This also gets into questions about which compiler is authoritative and that we, very-long-term, might want a formal semantics encoded in Agda/Coq/Lean/Random-Theorem-Prover that is then authoritative)

1 Like

Seconded. I personally don't do my best work when I'm spread out too much and in a leading position, having too much on one's plate can quickly affect one's well-being. This is one reason why I like the idea of having a team of leads, each member of which basically shares the same responsibilities. It can also make a big difference if you have a go-to person for quickly bouncing ideas off of.

6 Likes

I added an initial version of such a table to the brainstorming document:

(unfortunately it's too wide for hackmd...)

UPDATE: It's actually easier to read in edit mode :slight_smile:

1 Like

2018-11-16

We had our second steering meeting today. We drilled into some of our plans for the future, and specifically into some ideas around batch compilation. You can find the meeting minutes over on the new compiler-team repository, which will hopefully become the home for “all things compiler team” over the fullness of time. =)

Some action items that I think I would like to get help with:

  • take another stab at collecting and organizing ideas, incorporating ideas around UX in particular, but also breaking out RLS from batch compilation (nikomatsakis)
  • gather statistics about where we spend our time from perf and perhaps elsewhere (?)
  • draw up a proposal for polymorphization analysis / strategy (maybe nikomatsakis?)
  • collecting some numbers on how parallel queries performs these days (mw)
  • draw the graph of refactor dependencies (eddyb/nikomatsakis, perhaps?)
7 Likes

I just want to say, even though I unfortunately don’t have time to participate in compiler stuff much, I think the way you’re working in the open on internals.r-l.o and sharing what’s going on in the team is really great. I hope more teams and/or working groups will adopt your style over time.

14 Likes

Given that there is the Mozilla All Hands this week, which has @michaelwoerister and I tied up, I propose that we postpone the steering meeting this week by 1 week (until Dec 14). I further propose that we also postpone the next steering meeting by 1 week until Jan 4 (because otherwise it would be on Dec 28, which is a holiday for many folks).

If there are no objections, I’ll adjust the calendar events. =)

3 Likes

2018.12.14

(These minutes are also available on the compiler-team repository.).

The steering meeting was focused on mentoring. Some of the prompting questions:

  • How can we help move people from “working group participant” to “full-fledged team member”?
  • How can we reduce the “bus factor” for cricial modules like the type checker, type resolver, etc?

A summary of the major points raised follows. If you prefer, you can read the full transcript on Zulip.

Major points: Mentoring

Major points: process for refactorings, documentation

Towards the end, we also discussed the procedure we should use for larger compiler refactorings as well as how to ensure the compiler is documented.

5 Likes

Question: Do we want to have a meeting tomorrow? Will people be around? I’ve not really done much prep for it – I think we should probably however start talking about the upcoming Rust All Hands in a bit more detail, so maybe it makes sense to start that conversation tomorrow (i.e., try to figure out what would be the best meetings to have, etc)

4 Likes

We could also defer to next week, since I imagine many people are still on vacation.