dtolnay and I have been working on a potential new system for tracking
notifications for Rust team members. The primary reasoning behind this is that
we've wanted to surface team member queues to folks doing triage and the
community at large, as a way to help build both empathy for the length of the
queue and make it easier to tell how "soon" a PR might be reviewed.
Currently I believe it's just myself and dtolnay using the tooling, but it is
open for any Rust team member (specifically, listed on rust-lang/team) to try.
We do not currently have plans to expand beyond that, though there is nothing
intrinsically limiting it.
So, how does one look at the notifications? Presuming you're a Rust team member,
you can visit https://triagebot.infra.rust-lang.org/notifications?user=$GITHUB_USER_NAME. For example, see dtolnay's page.
The UI is quite dry, and the HTTP interface is entirely read-only today.
Acknowledging and adding meta information to notifications is done through
Zulip today. See the wiki for details on supported commands.
Currently only direct or indirect team pings (i.e., @Mark-Simulacrum or
@rust-lang/release) will automatically add a notification, across the
rust-lang organization. It is intended that in the future we will support other
modes of notifications, such as "subscribing" to particular threads and so
forth.
It's very early days so far, but dtolnay has told me that this has been very
helpful, and I have personally experienced the same. We're announcing this to a
wider audience primarily to give team members something to try out. We'd love to
hear feedback of any kind! I'm sure that this is not the right tool for
everyone, but even if it improves the lives of a small subset of team members
that would already be great!
Note that all Rust team members currently have a dedicated page, but this does
not mean that they are using the tool. There are plans to surface this in some
way.
Oh, also, forgot to mention. Feel free to leave feedback here, but also https://github.com/rust-lang/triagebot/issues is fine to file issues if you have feature ideas/requests/bugs.
I am very excited about this system, since the number of emails I get from GitHub is out of control to the point that email is no longer a reasonable way for me to track and manage actionable notifications. I expect as Rust's momentum grows, other team members have already experienced the same or will soon.
As I see it there are four major improvements that Mark's triagebot work brings to three audiences:
Team members see where their attention is needed without juggling zillions of emails. This is the most apparent benefit, but not the most important one from my point of view.
Triage team has a way to see what is on each team member's plate and make intelligent load-balancing decisions. Previously, if a PR sat for a while without review, the state of the art was for triage to make a shot in the dark to assign it to a random other member of the reviewing team, which is not necessarily productive.
Community members who send PRs or make issues now have something to track while waiting on feedback.
One of the nightmares for me as a maintainer is what @yaahc describes in this meetup talk: I remember distinctly one time leaving a comment on an issue very shyly being like "hey would it be okay if I take this issue?" and waiting two days for a response and overthinking it and freaking out, and eventually I just deleted the comment and tried to find somewhere else because I was too stressed about it.
Imagine if the comment that @rust-highfive leaves on new PRs included a link to the reviewer's triagebot page. This gives a place for the PR author to track as the reviewer makes progress toward the specific PR, and a clear way to internalize that there is a lot going on across the project – the lack of immediate attention isn't because the PR is stupid, or the author is a minority, or whatever other overthought reason. This makes for a friendlier environment than waiting in an unresponsive void.
Finally, most excitingly, the new system opens a way for community members to participate actively in the triage process. Not called out in Mark's post but documented in the wiki link, we've opened up write permissions on any team member's notification queue to anyone with a zulip id registered with triagebot.
So for example anyone can manipulate my queue in all the same ways I can, by prefixing their triagebot commands with "as dtolnay".
I strongly encourage people to take advantage of this to add or reorder my items! With the volume of things I am involved in, things slip by me and I can use the help keeping track of where my attention is most needed.
The tool does indeed expect everyone to acknowledge everything explicitly, including in that leaving a comment is not an acknowledgement.
I think that there's definitely room for improvement there: for example, it's long been a goal to make triagebot take over from high-five for r? management, and if we did that then it would likely be not too hard to consider such a comment resolved with an approval. Currently the tool lacks any heuristics like that, which certainly simplifies the design, but also does add to the potential human workload. I imagine that some heuristics are probably good ones to add, though I think finding exactly where to stop might be nontrivial.
As to notifications you don't care about, could you elaborate some more? Is that in the context of a team-wide cc perhaps? One thing we don't surface in the UI is whether a comment originates from such a ping or from a direct invocation of a username, but we obviously have this information. Would it be helpful to have that highlighted?
When I've thought about suggesting this tool in the past, I was going to suggest an explicit notation like f? @nikomatsakis as a way to explicitly request feedback from someone (vs just @-mentioning people randomly).
Something else I would really like:
I tend to want to order my notifications by "how frequently did I interact with this issue" as measured in number of comments. In other words, if I posted, the next comment is very likely in response to what I wrote, and I would tend to want to see it -- this is much more true than if there have been 222 comments in the interim.
Not necessarily team-wide, just something I either cannot usefully answer, or something that I need to investigate before answering, but won't investigate because it's not important to me.
Probably not automatable.
I think I agree that it's better to say something meaning "I won't answer" explicitly instead of just keeping silence, and that includes team-wide pings too.
Perhaps a triagebot list can help with that.
From the remaining items, some I've seen before and ignored as "meh, not my business", but some were new. The number of those remaining items wasn't large, so it should be ok to acknowledge them manually.
Would it perhaps be possible to allow (some) non-team members to use ping? For example, I'd like to draw some attention from some Libs team member to https://github.com/rust-lang/rust/issues/64260, but have been unsuccessful doing so.
