Lessons from the Impl Period

community
21

@vorner, sorry because english is not my main language and maybe I expressed myself wrongly. I didn’t meant to propose an agile/scrum process as the ones you do during work days, where you work 40/50hs a week because that’s not doable. I agree with your concerns and think that are completely valid. Anyway, I don’t think that concerns mean that thinking about a more structured process like the ones described is not possible. Note how I’ve said “dailys” in quotes and added a “or with the frequency that works”. I’ve used the word daily because it’s the one used from scrum daily meetings but it’s possible to adapt this kind of stuff to something that works better in this situation. Could be weeklys or something, it all depends on the working team. And also about the timezones, things could be arranged in a way that works and even some stuff could be done in an async mode.

I propose more the spirit and the objectives of those agile/scrum ceremonies rather than the actual known implementation because it clearly has some pitfalls.

22

I've been reading the thread with interest, but have been traveling so haven't had to write up much thoughts yet. Just wanted to add a link to this other thread that I just opened, which is specific to documenting the compiler:

1 Like
23

Ah, then sorry I misunderstood.

I’m still not sure about going too close to scrum. But I guess some inspiration could be borrowed.

If I think about what the scrum dailies do (make sure people know who works on what, who is blocked and if they need help) is to make sure everyone knows what everyone else does and if they are blocked on something and need help. Something like an aggregation of what is happening.

On the other hand, due to scattered team and people not always having time, some kind of asynchronous communication would be preferred.

How about a dashboard that’d show:

  • What the current goal is
  • The task each person is working on, with some little details (the last update time, when they expect to have more time to it, a guess of progress), and a green/yellow/red marker (green ‒ all ok, yellow ‒ I think I have little problems, but might solve them, red ‒ I’m stuck).
  • The suggested tasks to pick from (subset of all the tasks in backlog, so people are not overloaded with info).

The question is, who would update it. Each person for their own status, but there must be a way to decide on the current goal and current tickets. People usually have to agree on those somehow, but without a meeting it is a bit hard.

That’s just an idea, though.

1 Like
24

My big issue with the impl period was the period right before the impl period where a ton of RFCs were rammed through the gate. It made it exceptionally difficult for people to properly voice their opinions on all the RFCs they cared about. RFCs need to be dealt with at a very steady pace, without rushing anything through.

3 Likes
25

That is a shame :-/

On many cases, I would prefer you submit a PR marked as WIP, so that the discussion can happen around the code you’ve written. It’s also less likely to get lost in the shuffle.

1 Like
26

Yes! This is in the direction I think we need. During the impl period, one thing that I was frequently doing was a kind of "check in". I would just go over all the things that were "in flight" and ping each person involved. I had a spreadsheet where I tried to track the last time each person has been contacted. I frequently found people blocked on one thing or another who hadn't had time to mention it yet.

I would like to make this process much easier. Ideally, having a simple way to update your own status. Github doesn't make this any easier -- the fact that, e.g., you can't assign a GitHub issue to people outside of the rust-lang "team" is problematic.

In general, I think the most important point from both what you wrote and what @spastorino wrote is that it'd be nice if there was a bit more structure. For example, it'd be nice if you could "sign up" as working on something, and then have some regular check-in. I'd like to set expectations up front -- e.g., have some issues that are looking for someone with N hours per week to spend on it, and have others that are more suitable for "slowly hack on this over time", since people have different amounts of time available to spend.

Right now the "state of the art" when it comes to this are maintained tracking issues -- see e.g. the NLL tracking issue -- but obviously a lot of pieces of the puzzle are missing there.

It's important when talking about this stuff though not to overdesign. It's so easy to go crazy imagining how great things would be "if only" we had this whole complex setup. Then the bot or whatever you need to make it happen never happens. Probably something lo-tech -- I'm imagining a publicly editable google spreadsheet -- would actually suffice.

OTOH, stuff like @rfcbot has been transformative.

1 Like
27

I’m all for low-tech solutions in there, though some colors/nice UI really help usability ‒ you directly see who is blocked if the box is red.

Also, I think this should not take the time of the leader much ‒ leaders get saturated easily. So manually updating spreadsheet and lists is sub-optimal.

28

I must say that I was looking forward to the impl period but ultimately didn’t participate. One big issue was the sudden switch to infrastructure to Dropbox Paper and Gitter, that I never quite got into my workflow. The other one was that the projects I tried to participate in (website, mostly) weren’t very responsive and hard to reach even over those channels.

Also, completely lacking from this review is an assessment of the impl days, which I was part in organising and am a bit confused that it seems we have been missed during gathering of feedback.

2 Likes
29

Good point! So, I only participated in the impl days as part of Rust Belt Rust, as I was not able to attend RustFest. I'd love to hear from people who were there.

I felt like the impl days at RBR worked out pretty well, though it was small. But @spastorino and I got to discuss quite a lot and that was definitely very helpful.

One thing I thought would've been better is if we could've had possibly better attendance -- both from experienced and new contributors. I remember commenting to @aturon about various ways we might encourage (or even sponsor, though that requires more thought and planning) more attendance.

1 Like
30

I agree, though I mentioned a spreadsheet specifically with the hope that people participating would be able to edit it themselves (world-writable). GH doesn't really offer a good way to do this, though we could write bots or something.

31

I love Zulip, but it's not feasible for open chats, as it has no proper moderation tools, especially not the possibility to kill threads, block DMs, kick users (temporarily) and similar.

That's somewhat feasible for a business chat, especially in smallish organisations, but not for a community chat.

32

Trello for something like this works pretty well :slight_smile: .

33

I meant to chime in earlier with my two cents and experience.

So overall, I did not find contributing to the compiler particularly difficult, compared to other large projects. This is an achievement in and of itself, great job! Some of my comments my appear negative below, but let me assure you, they first of all come from a place of love, and in reality, contributing to rustc is quite nice experience here from a bureaucratic and technical perspective, congrats! In contrast for example, many moons ago I had a (couple) patches take over 3 months to go into LLVM, and even then, I didn’t even get credit for it, they were cherry-picked!

That being said, I would say the most distressing thing about hacking on the compiler is the internal documentation, but not really a lack of it, but rather some of it being either misleading, literally incorrect, or superfluous.

I found this more frustrating than anything; e.g., I would expect that a comment like this to be accurate, but is not (as an aside, i.e., in this particular case, we do not, contrary to the statement, do C++ name mangling, though it is substantially complicated by the linux C++ name mangler now supporting “rust mangling style”…)

At my company there is a strong culture of distrust of inline source code comments (e.g. above), as opposed to just reading the source code directly, for just precisely this reason - the comment goes out of sync/isn’t maintained/is no longer true/was true and now serves actually to hamper the reader by acting as a red herring.

I can provide more examples across other portions of the codebase, and I haven’t really read a lot, mostly just related to debugging and symbols, etc. This is worrisome to me…

Some other things off the top of my head, which some people may or may not have mentioned perhaps:

  1. An “index” for the major components of rustc/rust would be great; google’s libc implementation’s README is a really great example of this: https://github.com/aosp-mirror/platform_bionic#what-are-the-big-pieces-of-bionic . I suppose that the README in the rust/src/librustc/README.md counts, but its not very easy to find, isn’t necessarily the first place you’d look, etc. One problem is that the rust repo also contains libstd, etc. The root README has great instructions for building, but still not an overview of the source code structure; the google example above is great - it’s short and to the point!
  2. Who to ask what - I’m not sure how to solve this, but it would be nice if you had a set of “experts” who have certain fundamental knowledge about particular portions of the codebase, and one could drop a line somehow to ask for clarification, etc. For example, I had a build failure in the debug testing language (which is still a mystery to me), and I still don’t know who knows how it works, or where its even documented how its supposed to work: https://github.com/rust-lang/rust/pull/46457#issuecomment-351539880
  3. Better compile times :stuck_out_tongue: ; it would be really great if cargo check existed in the codebase; when you’re just starting to hack, quick feedback is nice to play with different types, what’s ok, what isn’t, etc.
  4. Rust compiler jargon - I’m really not sure why some of it took off or why it should be exist, and I find it a little strange, but whatever I guess.
  5. There appears to be a deep LLVM bias in the codebase and among devs

I don’t want to offend anyone with the last comment, and its not a bad thing at all, LLVM is great, but I’ve had several occasions with online discussions where it appeared this was actually causing a communication issue, as they only understood the backend in llvm terms (which takes care of some of the more ABI/binary-esque issues for you, and consequently, I suppose, one never therefore needs to think of it), whereas I approach these kind of things more from a “this is what I’m holding in my hand, e.g., the actual emitted binary”. Incidentally, this actual emitted binary is what debuggers and other tools only get to look at, so it definitely helped me notice and track down some related bugs (and know how to start to fix them without too much catch-up work).

On a similar note, a few times I felt like I was being condescended to when spoken to or having questions answered, and while possibly a result of the difficulties of online communication, I did think I should mention it, as I’m not super motivated to contribute again.

I guess that’s kind of a bummer thing to end on so I’d like to reiterate that all of the devs have done great work and to keep it up :slight_smile:

Take care!

2 Likes
34

I have to agree here. There is a dictionary in the top-level librustc README, but it really is not that comprehensive.

35

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.