Let proc macros pass information to the compiler

It’s true that Rust is a more complicated language than Java, but it doesn’t have too horrible parts. Stuff like complicated type-inference, proc-macros and const fn are not fundamentally problematic: there’s “just” more stuff to implement, and more requirements for clever on-demandifying and caching. In particular, compile-time code-gen itself is not problematic, as long as it is a pure function with relatively isolated inputs.

I’d list other things in Rust as fundamentally hard for Code Analyzers:

  • macros 1.0, because they require traversing all files in the order of mod foo; declarations,
  • global nature of trait search: when resolving x.foo() you can easily say where the type of x is, and where the trait for foo method is, but finding the impl can be very hard, because it even can be hidden inside some local function.
  • proc macros doing IO: IO makes caching and on-demand harder, it makes overall performance slower, and it’s architecturally difficult to fit into Code Analyzer, because it shouldn’t do any IO by itself.

That said, there are features of the Rust language, which makes it easy to write a Code Analyzer

  • Rust is the best existing language to write a Code Analyzer in
  • rustc already has pretty-cool tech for on-deman analysis
  • and, most importantly, Rust has a project model which is the perfect fit for Code Analyzers.

The last point is worth elaborating (which will also be a response to @petrochenkov comment).

There’s a build system and a project model, and these are different things. Build system is a sequence of arcane shell invocations which somehow produce project artifacts. A project model is a logical model of dependencies between various source files. For each language, there are many different build systems (each sufficiently large project has a unique one), but only a single project model.

For Java, the model is classpath: the set of base directories with sources and compiled jars. C++, with it’s preprocessor, does not really have a project model better than “a sequence of invocations of clag++, with all flags, cwd and env vars”.

And Rust, Rust does not have a single global namespace of symbols, it has crates, and this is awesome, because it is a very precisely defined project model: a directed acyclic graph of anonymous crates, where edges are marked by extern crate names and cfg flags. It is much better than a classpath, because you know precisely when two crates are independent, so you can be much more aggresive with caching and indexing.

Now, the design bug in RLS is that it doesn’t work with project model, it works with a specific build system, and it makes it slow to incorporate changes and brittle. In contrast, IntelliJ Rust works with project model, and, while it’s not exactly fast, it can work with rust-lang/rust code base, it can do completions, it can infer types: https://www.youtube.com/watch?v=8xoTR0d6R_0&feature=youtu.be

As another point for “you can have a horrendous build system and precise Code Analyzer”, consider IntelliJ itself. It is a huge project, and I think it is build with every tool you can use to build Java. I definitely saw Ant, Maven, Gant, Gradle and JPS in various parts of it. However, IntelliJ is developed in IntelliJ and, naturally, all code insight features just work: IDEA knows little about it’s build system, it only has a project model which is a set of XML files describing classpaths of various modules. Now, syncing those XML files and build-system is a pain. However in Rust a similar task would be much simpler: it has a stronger project model and it has a mostly one build system (Cargo). For the “rustc in IntelliJ” video, I haven’t done anything special, I’ve just opened rust/src folder in IDEA, which then asked cargo metadata about project info.

EDIT: @Michael-F-Bryan sorry for derailing a thread, guess I’ll head to 2019 Strategy for Rustc and the RLS :slight_smile:

9 Likes