For the last few months, @nikomatsakis, @wycats, and myself have been working on an experimental compiler to see if we can build one that can handle both straight-line/traditional compilation as well as be able to be used in an interactive environment like an IDE. Still very early days, but it’s showing some promise and we wanted to chat about the project with the wider compiler team.
The project is built on Salsa, with each stage (parsing, typechecking, etc) being salsa-ified. This lets us create a lot of queries that can answer specific questions about entities in the system, like typechecking a single function or grabbing the header of a function without its body.
To prove things out, we also built Language Server Protocol (LSP) support into the compiler, so you can run the compiler in IDE mode from inside VS Code and get interactive features.
If you’re interested, there’s a link to learn more:
Not yet, though that’s definitely in the cards as it matures. We’d like to get a sense for how much we are paying as we do more complex things, to make sure we’re not doing analysis we don’t need to do.
Right now we’re doing more of a breadth-first approach to make sure we have at least some coverage of features broadly before we begin improving the speed of things like LSP calls.
As an aside, while I’m aware LSP has won mindshare over the last 2 years, I don’t think it’s the end of the road there.
But in the meantime it’s better than what came before it, that’s for sure.
The LSP implementation is separate and on-top of the logic core of rust-analyzer. If some new protocol were to come along, the protocol layer could be implemented on top of the same core and work alongside the LSP protocol interface as well.
(The LSP interface is of course as stable as LSP, but the core logic Rust interface is very unstable.)
The crucial difference is that Lark is for the Lark language, while rust-analyzer is for Rust. Both are experiments, but you already can use rust-analyzer as a daily IDE.
Another super-important difference from IDE POV is that in Lark everything is a macro, if I understand it correctly. For example, struct is a macro. This "entities decide about their syntax themselves" seems to be a pretty hard nut to crack using traditional IDE syntax trees (like the ones in IntelliJ, Roslyn or rust-analyzer).
EDIT: to make it clear, using something like rowan would be possible, but less useful, because it would be hard to write IDE features like formatting, auto-indentation, typing assists, code assists and refactorings "from the outside", each specific entity would have to bundle it's own specific refactorings, etc.
