Building tools on top of cargo/rustc for inspecting crates

[Michael-F-Bryan]

@carols10cents mentioned on the user forums that crates.io doesn’t currently do any analysis of crates. I’m wondering whether it’s possible to build a tool similar to docs.rs which will download any new crates, run them through cargo, then inspect the various types and functions in its public API?

The main idea would be to create a more powerful search index which has access to information about a crate’s internals. For example, whether a crate uses unsafe (cc: @dobkeratops), seeing if a crate is well tested/documented, or giving you the ability to search for things which implement a given trait (e.g. serde::ser::Serializer to see what formats you can serialize to with serde).

Is there any way I can ask the cargo or rustc tools to do something like this, preferably using their command-line interfaces or in a stable way? Or would I need to create my own program which pulls in the compiler and inspects its (unstable) internals?

[carols10cents]

Yup, it’s possible! Note that I’ve let that library bitrot and it might not work at all right now, I’m just linking to it as an example of the possibility.

[Michael-F-Bryan]

Thanks for the example! I’ve been mostly involved in creating applications using Rust and not using the compiler itself so it’s interesting to see how rustc works under the hood.

Is there any reason why you chose to implement this as a lint which gets run by the compiler instead of (for example) something which traverses the Crate AST produced by libsyntax?

Also, does anyone know of any other useful resources (blog posts, prior work like @carols10cents’s cargo-doc-coverage etc.) for working with or building tools on top of the compiler?

[sanxiyn]

https://github.com/nrc/stupid-stats is another useful resource.

[carols10cents]

Is there any reason why you chose to implement this as a lint which gets run by the compiler instead of (for example) something which traverses the Crate AST produced by libsyntax?

I honestly don’t remember.

[nrc]

rustw does the search bit. We are vaguely thinking about using it for a better source view in rustdoc. That is powered by bits of the RLS which has good knowledge of a program and the types. However, this approach is only really good for a predetermined set of things (code exploration, refactoring, etc.). We don’t have much info about unsafe, for example. So for proper flexibility, you probably need to plug-in to the compiler a bit more. Unfortunately, there is no API for that, so it is likely to be hard work and to get frequently broken by changes.

Is there any reason why you chose to implement this as a lint which gets run by the compiler instead of (for example) something which traverses the Crate AST produced by libsyntax?

The AST doesn’t have any type information or data that the compiler gathers, it only has the syntax, which is not powerful enough for the things that you list.

[Michael-F-Bryan]

As a follow up to this, I’ve started a little tutorial which explores building tools on top of rustc. I’ve built on top of @nrc’s excellent stupid-stats, with the end goal to make a cargo metrics subcommand for analysing a crate.

It’s still very much a work in progress, but if you can spare a couple minutes to have a look and give feedback it’d be most appreciated

(rendered/repo)

An Unofficial Guide to Using Rustc

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.