"So you want to hack on the Rust compiler?", a plan for a book

[Tom-Phinney]

@mark-i-m

I like that idea, but not every new feature touches every part of the compiler. It might end up being a pretty complicated tutorial, right? I guess I could imagine something of the following form:

Add a simple syntactic feature and make it desugar to something that already exists Make the feature more complex so that you actually have to change the AST or HIR Make the feature more complex so that it interacts with MIR or one of the compiler passes (e.g. type checking) Make the feature more complex so that it interacts with LLVM

I don’t know what kind of feature that would be, though. Maybe some more experienced compiler hackers have examples from recent PRs on the real compiler?

I have a quite extensive example of a language enhancement that could be used, along with a brief discussion (to be included) of why extending the language in that way is unneeded and unwise.

My initial use of Rust was for writing a crypto package that would otherwise be written in C. The cipher algorithm at the heart of the package is an extremely-efficient military-grade autokeyed stream cipher which uses wrapped add/sub/mul as well as rotate. I thought that there should be lexemes for those operators and their assign variants, so I went through the nightly compiler source finding all the files that would need to be changed to add those operators. I worked out the required changes, preceding each such new or changed line with an appropriate meta-information statement (.e.g.,

#[unstable(feature = "operator_tokens_for_wrapped_add_sub_mul", issue = "0")]`)

to control inclusion/exclusion. There were a lot of files, because the changes affected the lexeme parser and AST and HIR and MIR.

I concluded that two new lints were needed, but did not develop the expertise to work them out because they required enhancing the compiler to maintain a distinction between unsigned and (probably-)signed expressions.

Even without the lints, this would be an example that goes from source (new lexemes) all the way to LLVM (for LLVM's rotate intrinsics).

I started to document these proposed enhancements as a pre-RFC, then realized that I should determine how useful the enhancements would be to the general Rust population. I grepped the compiler, as well as the various crypto crates. My conclusion was that only 5% of the uses of add/sub needed wrapping arithmetic, which was too infrequent to justify complicating the language. (There was almost no use of wrapped mul.)

I saved this work; it would be fairly easy to migrate it to the latest stable release and use it as an extensive serial example, first of lexeme parsing, then of AST structure, then of the HIR, followed by the MIR. ending with calling LLVMs intrinsics.

Edit: Changed inline presentation of above #[unstable(feature = … example.