The issue with this is that it "punishes" people that review quickly by giving them more to review. So, the incentives this generates are kind of backwards.
We'd have to be really careful not to turn this into a wall of shame calling out the slowest reviewers though. That doesn't sound healthy.
The primary interface to thanks is “give a random N contributors in arbitrary order” IIRC; review-thanks could potentially avoid being used to identify lower throughput reviewers by not offering pagination, only the contribution-weighted random N. If we don't even expose the weight factor, then the higher weights show up more without exposing the numeric value of lower weights when the less frequent do show up.
Disclaimer: I'm not a game theory expert, so I have no idea whether this scheme maintains the positive incentive without the negative incentive, nor if that's even theoretically possible.
The weight would probably want to include size of reviews in addition to quantity somehow; reviewing a huge refactor should probably be weighted more than a small number of trivial comment typo reviews. But as with any social score system, it's dangerously vulnerable to gaming, perverted incentives, and shaming.
Random thought:
This would let people review PRs at their own pace while also not pressuring them to do something. If enough PRs are not being claimed, then it's clear that more reviewers are needed.
Just some random thoughts:
This got me thinking about the difference between the individual contributor role and the manager role in a company. Maybe we do need a role similar to that of a manager (or team leader) for each team?
Sure, it's not that purely management, and open source communities can be very different from organizations like companies.
I know there is a team leader for each group in most Rust teams, but is it true that the team leader now is more in the role of individual contributor?
Ideally, the team leader should be a role focused on helping others contribute more, organizing and coordinating the efforts of team members to build an engaging and vibrant community.
His/her responsibilities include identifying workload issues for team members, helping to reduce the workload, and assigning entry-level tasks to attract more new contributors.
I'm not saying current team leader not doing a great job. Given that most Rust contributors contribute in their spare time, this may be too much to ask of a team leader.
This still lacks any power to actually compel time for reviews from volunteers. I feel like something like a "Developer-in-Residence" that Python has in Łukasz Langa to help wrangle things would be on the order of what is more effectively useful here. Can the Rust Foundation help with such things? If so, what would it take to actually make it happen?
Didn't @Boxy give a talk (rustconf?) along the lines of "Why Your Open Source Project Needs a Project Manager"? I couldn't find it with a quick search, unfortunately.
Even without the power to compel work, someone with a dedicated producer/PM role is beneficial, as it's their stated responsibility to keep things moving and (prioritize and) reassign workload to those available to do it.
The foundation is currently hiring two engineers: Rust Foundation - Rust Foundation
But this doesn't directly help the maintainer burden, as these folks will be on a level playing field, just like any other contributor, given that the foundation does not have technical control over the project. Over time, maybe we will have foundation employees that have established themselves in maintainer roles, but it won't happen immediately.
Sure. I don't believe that Łukasz has powers beyond his community cred he's built up. But it helps a lot to have someone whose tasklist includes "at least look at all incoming PRs (and issues if you have time)". When I was doing it for CMake, it was easy to say "oh, this looks familiar" and help reduce the duplicate or cross-link related issues. The problem is that once you fall off the bandwagon, catching up is near impossible (as happened when I went on a vacation) if it's not a bulletpoint in your tasking.
Łukasz was a CPython core developer long before he became developer in residence (literally a decade).
Sure, but does the DIR title give him any power that he didn't have before?
No, just more hours in the day.
Encouraging people to review PRs can be really simple. When a new contributor opens a PR, rustbot leaves a comment thanking the person (example). Rustbot could do the same when someone reviews a PR for the first time. This tells everyone reading the thread that reviews are appreciated and encouraged by people who aren't auto-assigned. Rustbot could also link to a document that explains how to become an "official" reviewer.
To support people who are auto-assigned to PRs, there should be a way to tell rustbot how many PRs you can review per week. For example, John Doe can tell rustbot to assign him 10 new PRs every week. When John Doe goes on a vacation, he can't do reviews, so he just sets the number to 0 during the vacation. When there are too many new PRs and all the reviewers are at capacity, rustbot should add the label no-reviewer-available instead, and may add a comment such as “All t-libs reviewers are currently busy reviewing 62 PRs. Please be patient until a reviewer is available again.“
Sometimes John Doe overestimates how many reviews he can do, and still has 3 PRs in his queue at the end of the week, so he tells rustbot to only give him 7 PRs the next week. And if he's finished with his reviews early and wants to do more, he can just search for the no-reviewer-available label and claim a few more PRs.
My company uses this strategy to assign customers to our employees in customer service, and it works really well from what I hear. In the Rust context, I expect quite a few advantages compared to the current situation:
no-reviewer-available label communicates more clearly with the PR author why it takes so long to get the PR reviewedSo I took a look at
I think it's a great idea and I've thought of some possible improvements. I've already made some improvements which you can view at my fork here: https://pitaj.github.io/rustc-pr-tracking/
These changes reveal some interesting numbers. As I suspected, PRs are actually left untouched for longer than the "last activity" stat reveals.
At the least, the older waiting-on-author PRs should probably be closed.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.