This is a post about a bad problem we have in Rust, and another plea for help. It’s a braindump, just hoping to get the need out there and see if anybody can help push us in a better direction.
Very often we have questions about the health of the project, about whether various things are getting better or worse, and we mostly just guess. It would be so much better if we had numbers.
We have lots of automated systems and lots of bots floating around the project. A lot of the pieces exist, but they don’t exist in one place.
I read recently that Miguel de Icaza was interested in improving Xamarin’s issue triage using Power BI, and after just looking at the front-page screenshot I immediately wished we had that.
I’m looking for somebody interested in creating such a system for Rust. We can start by creating a SQL database and a batch process for scraping the data, picking some numbers that are easy to get ahold of, and setting up a simple web page to graph them.
The rest of this message is just a catalog if important information we need better visibility into. Feel free to mention more.
- Opened PRs per day
- Closed PRs per day
- Time from PR open to PR close
- Number of “@bors retries” per PR
- CI errors per buildbot builder. Which builders fail the most?
- Time per ‘auto-’ builder run
- Release channel health. What’s the current release? How many days has nightly been broken?
- Opened issues per day
- Closed issues per day
- Time from issue open to issue close
- P-high issue count
- Regression issue count
- Compile-time / runtime % change per day
- Performance improvements/losses per contributor.
Some of this information is generated by perf-rustc, which has an undocumented API that can be used to retrieve data.
- Downloads per crate per day
- Crate publications per day
- Downloads per day
- Downloads per day per artifact
- Twitter mentions
- urlo / reddit posts per day
- Top error codes
Download information is hard to get without backend help.
Approximations may be achievable with rustup telemetry, though not all downloads will ever go through rustup and not everybody will turn on telemetry.
Error codes also can be collected through rustup.