All-hands recap: Tools
A couple of weeks ago the Rust team had its second All-Hands in Berlin (read about the first one), with people present from all the teams.
The tools teams had a bunch of discussions and a lot of progress was made!
Tools team reorganization
Recently, Nick stepped down from leadership of the tools team, so we took this opportunity to reevaluate how the tools team worked, and if the tools team was necessary to have at all (as opposed to just having per-tool teams).
We concluded that the tools team does have purpose: it’s valuable to have a team that has an overview of what’s going on with the tools, and help coordinate between tools. Furthermore, it’s important to look for “tool vacuums” and try to improve the situation when it comes to missing tools.
The nature of this team means that it doesn’t need to be comprised only of subteam leads; instead its composition can be people who are most useful to provide this overall perspective, which may include subteam leads. Furthermore, we wouldn’t need to meet as often.
To some extent figuring out leadership means figuring out accountability, you need to ensure things get done and this is trickier when there’s a team of people “in charge”. Furthermore, it is harder for the rest of the project to interface with a group of people as opposed to just one or two. So we’re going to have two overall “tools team leads” who are responsible for conducting meetings and following up on action items, as well as interfacing with the rest of the project. Two people leads to some redundancy and allows for more flexibility, but isn’t so large that things may get lost in the confusion.
We’re working on selecting our leadership and will have a separate announcement as that is finalized.
You can find the meeting notes here
A few things have been decided for rustdoc:
Generate others outputs than HTML (we’d like to start with JSON) The goal behind this would be to allow better integration into other doc hosts, and let people have alternate ways of inspecting their documentation.
Improve documentation and the testing of rustdoc itself We had a lot of regressions this year due to the lack of testing. A plan has been started to add a crater-like server to test the UI directly.
Experiment with an archive server to reduce number of files This is one of the biggest pain points of rustdoc: it generates a lot of files which is a huge issue for Windows users . This is due to us wanting documentation to work without servers. We thought about how to reduce this number while remaining serverless, and the solution we found is a middle ground between the two: allowing the user to start an archive server (so there is only one file generated).
Intra doc linking This feature is one people have wanted for a while. It removes almost all pain from having to write paths in the documentation: rustdoc resolves the name and then generates the path itself. However, this is still incomplete, and we worked with the compiler team to figure out what remained to be done and how. We expect it to move forward very soon!
To sum this week up: the main word was maturity. Finishing features and improving users’ documentation experience will take over adding new features this year.
More information can be found in misdreavus’ post here.
We’re excited to say that we spent a considerate amount of time talking about the future of the Rust IDE support and the fruit of that is formation of an experimental RLS 2.0 compiler working group:
Initially built on top of rust-analyzer, the primary goal is to discover an ideal infrastructure on top of which we can build an IDE-aware compiler, while also exploring how we can split
rustc into more manageable chunks with better-defined boundaries.
With end-to-end queries (on-demand compiler model) defined as a goal at Rust All-Hands 2018 we discussed remaining work to be done and talked a great deal about possible incrementalization and “lazifying” of the compiler frontend. Currently, parsing, macro expansion, name resolution and partially HIR lowering are not yet translated to the new model and so most of the work and discussion was related to how we can tackle that and design the appropriate interfaces.
Some of the items high on the design todo-list include:
- A way to concretely represent the program - typical IDE tools/transformations prefer a pure AST approach, whereas macro expansion phase prefers a token-based solution
- Find a principled way to decouple name resolution and macro expansion - one can directly influence the other, thus limiting the possible incrementalization and laziness
Last year was a great year for Clippy — we finally became usable with stable rust and distributed with rustup!
Most of clippy’s growth is just adding new lints, and we plan to continue doing that — we have a healthy contribution pipeline and it probably doesn’t need explicit intervention.
We discussed a bunch of implementation issues from the rustc driver perspective. Clippy is currently implemented as a rustc driver; which is basically a drop-in replacement for rustc. This works pretty okay, but often has bugs (usually windows). Furthermore, other custom drivers (e.g. rustfix) don’t interact well on the cargo side. Ideally, a holistic solution for driver integration in cargo would be nice. This would also help improve use cases like cargo-fuzz.
We also discussed alternative lint-writing approaches. Currently lints are written in plain Rust code, matching on the rustc AST and type information. It’s not too complicated, but ideally this could be easier to do. Furthermore, decoupling things from the AST helps stability overall. Something like Coccinelle for lint-writing would be nice, where lints are just written as Rust code with placeholders and conditions. rerast is one foray into such a space. There’s currently a grad student looking into this stuff.
We also specifically discussed short-term solutions for the driver issues causing cargo fix and clippy to not work nicely together.
You can find the meeting notes here
Rustfix and Cargo updates may come later