Lessons from the Impl Period

We use those a lot at my workplace, but from that experience they can become overwhelming when crowded (especially when you spend a bit of time away from your mailbox and must go through the huge backlog on return) and searching their archives isn’t very convenient.

I think they work best for infrequent announcements and complex discussions, but are not so well-suited to rapid-fire everyday chatter.

1 Like

Maybe it’s just me, but that just sounds like a forum would be the thing? You’d make subforums by topics as you want, you have conversations in the threads, and if you need one-on-one conversation that’s publically accessible, you can just do that in a thread where you talk (if the forum software is featurefull, I’d imagine you could have some kind of access management, if you don’t want others to join your conversation, but keep it readable). Everyone can just read what’s interesting, and it’s just a tad slower than real-time chat like IRC.

3 Likes

If you have a lot of dedicated channels on slack, it makes it easier to follow conversations and avoid multiple conversations getting tangled. It does add complexity of policing and redirecting members to different channels. We have done this for a PHP framework which has around 20k members, and a lot of people call it a mess, but a lot of quality discussions take place as well.

It is better to have multiple ways so that people can choose the medium they are comfortable with as long as none of the mediums get diluted.

Being one of the devs that helped during the impl period with Niko as a lead, I’d like to say a couple of things. First that this was amazing and Niko worked great with us.

Then on improve or try side of things I’d like to suggest trying something like what agile proposes in terms of collaboration and team work. In particular from agile/scrum, defining a clear vision of the work needed to be done, defining a sprint with a clear vision, having plannings and “daily” meetings (or with the frequency that works, there’s no need to be dogmatic here) and retrospectives where you can see what things need to be improved and get feedback quickly, would be great from my point of view. We at wyeworks.com use all this with great success since 2008. Also one of the main things that this could help with is on the team building side. Maybe working more as a team and less as individuals and trying to join forces to build a working thing and adapting to what our objectives are more in a collective way rather than taking invidual efforts. Never seen an OSS project doing something like this but … in case we have people dedicated, why not?. It could also be a good idea to have some Agile/Scrum master, somebody that wants to help on this regard, which could also decompress people like Niko from doing this kind of tasks or even from asking people how are they going with tasks. Never heard also of an “OSS” Agile coach/master, so it would be good and somebody may want to do something like this.

I didn’t join in the impl period in the end, mostly because it already required too much time to even be able to participate. I can find the time to post an occasional PR here and there (but with rustc, where making one is a lot of effort due to the large code base and complexity). But with a day job and several other hobbies, I just couldn’t participate in any daily meetings. I guess many more people would get disqualified as a result of that.

Further problem is this doesn’t work that well across time zones (it can be made to work, I’ve actually done that), but that seems workable if it is your main job, not a side hobby.

1 Like

@vorner, sorry because english is not my main language and maybe I expressed myself wrongly. I didn’t meant to propose an agile/scrum process as the ones you do during work days, where you work 40/50hs a week because that’s not doable. I agree with your concerns and think that are completely valid. Anyway, I don’t think that concerns mean that thinking about a more structured process like the ones described is not possible. Note how I’ve said “dailys” in quotes and added a “or with the frequency that works”. I’ve used the word daily because it’s the one used from scrum daily meetings but it’s possible to adapt this kind of stuff to something that works better in this situation. Could be weeklys or something, it all depends on the working team. And also about the timezones, things could be arranged in a way that works and even some stuff could be done in an async mode.

I propose more the spirit and the objectives of those agile/scrum ceremonies rather than the actual known implementation because it clearly has some pitfalls.

I've been reading the thread with interest, but have been traveling so haven't had to write up much thoughts yet. Just wanted to add a link to this other thread that I just opened, which is specific to documenting the compiler:

1 Like

Ah, then sorry I misunderstood.

I’m still not sure about going too close to scrum. But I guess some inspiration could be borrowed.

If I think about what the scrum dailies do (make sure people know who works on what, who is blocked and if they need help) is to make sure everyone knows what everyone else does and if they are blocked on something and need help. Something like an aggregation of what is happening.

On the other hand, due to scattered team and people not always having time, some kind of asynchronous communication would be preferred.

How about a dashboard that’d show:

  • What the current goal is
  • The task each person is working on, with some little details (the last update time, when they expect to have more time to it, a guess of progress), and a green/yellow/red marker (green ‒ all ok, yellow ‒ I think I have little problems, but might solve them, red ‒ I’m stuck).
  • The suggested tasks to pick from (subset of all the tasks in backlog, so people are not overloaded with info).

The question is, who would update it. Each person for their own status, but there must be a way to decide on the current goal and current tickets. People usually have to agree on those somehow, but without a meeting it is a bit hard.

That’s just an idea, though.

1 Like

My big issue with the impl period was the period right before the impl period where a ton of RFCs were rammed through the gate. It made it exceptionally difficult for people to properly voice their opinions on all the RFCs they cared about. RFCs need to be dealt with at a very steady pace, without rushing anything through.

3 Likes

That is a shame :-/

On many cases, I would prefer you submit a PR marked as WIP, so that the discussion can happen around the code you’ve written. It’s also less likely to get lost in the shuffle.

1 Like

Yes! This is in the direction I think we need. During the impl period, one thing that I was frequently doing was a kind of “check in”. I would just go over all the things that were “in flight” and ping each person involved. I had a spreadsheet where I tried to track the last time each person has been contacted. I frequently found people blocked on one thing or another who hadn’t had time to mention it yet.

I would like to make this process much easier. Ideally, having a simple way to update your own status. Github doesn’t make this any easier – the fact that, e.g., you can’t assign a GitHub issue to people outside of the rust-lang “team” is problematic.

In general, I think the most important point from both what you wrote and what @spastorino wrote is that it’d be nice if there was a bit more structure. For example, it’d be nice if you could “sign up” as working on something, and then have some regular check-in. I’d like to set expectations up front – e.g., have some issues that are looking for someone with N hours per week to spend on it, and have others that are more suitable for “slowly hack on this over time”, since people have different amounts of time available to spend.

Right now the “state of the art” when it comes to this are maintained tracking issues – see e.g. the NLL tracking issue – but obviously a lot of pieces of the puzzle are missing there.

It’s important when talking about this stuff though not to overdesign. It’s so easy to go crazy imagining how great things would be “if only” we had this whole complex setup. Then the bot or whatever you need to make it happen never happens. Probably something lo-tech – I’m imagining a publicly editable google spreadsheet – would actually suffice.

OTOH, stuff like @rfcbot has been transformative.

1 Like

I’m all for low-tech solutions in there, though some colors/nice UI really help usability ‒ you directly see who is blocked if the box is red.

Also, I think this should not take the time of the leader much ‒ leaders get saturated easily. So manually updating spreadsheet and lists is sub-optimal.

I must say that I was looking forward to the impl period but ultimately didn’t participate. One big issue was the sudden switch to infrastructure to Dropbox Paper and Gitter, that I never quite got into my workflow. The other one was that the projects I tried to participate in (website, mostly) weren’t very responsive and hard to reach even over those channels.

Also, completely lacking from this review is an assessment of the impl days, which I was part in organising and am a bit confused that it seems we have been missed during gathering of feedback.

2 Likes

Good point! So, I only participated in the impl days as part of Rust Belt Rust, as I was not able to attend RustFest. I’d love to hear from people who were there.

I felt like the impl days at RBR worked out pretty well, though it was small. But @spastorino and I got to discuss quite a lot and that was definitely very helpful.

One thing I thought would’ve been better is if we could’ve had possibly better attendance – both from experienced and new contributors. I remember commenting to @aturon about various ways we might encourage (or even sponsor, though that requires more thought and planning) more attendance.

1 Like

I agree, though I mentioned a spreadsheet specifically with the hope that people participating would be able to edit it themselves (world-writable). GH doesn’t really offer a good way to do this, though we could write bots or something.

I love Zulip, but it’s not feasible for open chats, as it has no proper moderation tools, especially not the possibility to kill threads, block DMs, kick users (temporarily) and similar.

That’s somewhat feasible for a business chat, especially in smallish organisations, but not for a community chat.

Trello for something like this works pretty well :slight_smile: .

I meant to chime in earlier with my two cents and experience.

So overall, I did not find contributing to the compiler particularly difficult, compared to other large projects. This is an achievement in and of itself, great job! Some of my comments my appear negative below, but let me assure you, they first of all come from a place of love, and in reality, contributing to rustc is quite nice experience here from a bureaucratic and technical perspective, congrats! In contrast for example, many moons ago I had a (couple) patches take over 3 months to go into LLVM, and even then, I didn’t even get credit for it, they were cherry-picked!

That being said, I would say the most distressing thing about hacking on the compiler is the internal documentation, but not really a lack of it, but rather some of it being either misleading, literally incorrect, or superfluous.

I found this more frustrating than anything; e.g., I would expect that a comment like this to be accurate, but is not (as an aside, i.e., in this particular case, we do not, contrary to the statement, do C++ name mangling, though it is substantially complicated by the linux C++ name mangler now supporting “rust mangling style”…)

At my company there is a strong culture of distrust of inline source code comments (e.g. above), as opposed to just reading the source code directly, for just precisely this reason - the comment goes out of sync/isn’t maintained/is no longer true/was true and now serves actually to hamper the reader by acting as a red herring.

I can provide more examples across other portions of the codebase, and I haven’t really read a lot, mostly just related to debugging and symbols, etc. This is worrisome to me…

Some other things off the top of my head, which some people may or may not have mentioned perhaps:

  1. An “index” for the major components of rustc/rust would be great; google’s libc implementation’s README is a really great example of this: https://github.com/aosp-mirror/platform_bionic#what-are-the-big-pieces-of-bionic . I suppose that the README in the rust/src/librustc/README.md counts, but its not very easy to find, isn’t necessarily the first place you’d look, etc. One problem is that the rust repo also contains libstd, etc. The root README has great instructions for building, but still not an overview of the source code structure; the google example above is great - it’s short and to the point!
  2. Who to ask what - I’m not sure how to solve this, but it would be nice if you had a set of “experts” who have certain fundamental knowledge about particular portions of the codebase, and one could drop a line somehow to ask for clarification, etc. For example, I had a build failure in the debug testing language (which is still a mystery to me), and I still don’t know who knows how it works, or where its even documented how its supposed to work: https://github.com/rust-lang/rust/pull/46457#issuecomment-351539880
  3. Better compile times :stuck_out_tongue: ; it would be really great if cargo check existed in the codebase; when you’re just starting to hack, quick feedback is nice to play with different types, what’s ok, what isn’t, etc.
  4. Rust compiler jargon - I’m really not sure why some of it took off or why it should be exist, and I find it a little strange, but whatever I guess.
  5. There appears to be a deep LLVM bias in the codebase and among devs

I don’t want to offend anyone with the last comment, and its not a bad thing at all, LLVM is great, but I’ve had several occasions with online discussions where it appeared this was actually causing a communication issue, as they only understood the backend in llvm terms (which takes care of some of the more ABI/binary-esque issues for you, and consequently, I suppose, one never therefore needs to think of it), whereas I approach these kind of things more from a “this is what I’m holding in my hand, e.g., the actual emitted binary”. Incidentally, this actual emitted binary is what debuggers and other tools only get to look at, so it definitely helped me notice and track down some related bugs (and know how to start to fix them without too much catch-up work).

On a similar note, a few times I felt like I was being condescended to when spoken to or having questions answered, and while possibly a result of the difficulties of online communication, I did think I should mention it, as I’m not super motivated to contribute again.

I guess that’s kind of a bummer thing to end on so I’d like to reiterate that all of the devs have done great work and to keep it up :slight_smile:

Take care!

2 Likes

I have to agree here. There is a dictionary in the top-level librustc README, but it really is not that comprehensive.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.