Syntax sugar makes macros inconvenient. Possible to de-sugar all the way back to everything looking like functions?

Disclaimers:

  1. I know Rust syntax does not currently support this
  2. This is a train of thought, and isn't remotely rigorous

My line of thought:

  1. A lot of operators are essentially syntax sugar for actual functions. Like a+b is exactly equivalent to a.add(b)
  2. In the compiler, everything is represented as a tree, and there multiple intermediate representations that are also trees
  3. Trees can be represented as functions, eg Lisp
  4. It would be very convenient for macros if I could auto-convert any given Rust code to a lisp-like functional form, and apply the macro on the simplified syntax. Being able to convert back to regular Rust format afterwards would also allow for interesting code editor tooling. (Does rust-analyzer already do something like this internally?)
  5. Might allow interesting language features to be experimentally implemented before the syntax bikeshedding phase
  6. Miri might be adjacent/tangential to this?
  7. Key point: in this conversion to a functional syntax, there would ideally be no loss of information, so the code could be converted back to the original state. I assume that the Rust compiler does optimizations at several steps as code is converted through the various intermediate representations, which would prevent that.

Not exactly equivalent. E.g. they have different type inference properties, different behavior for built-in types (e.g. const support), if taken literally, the syntax a.add(b) doesn’t even have to end up using the Add trait because you only gave a method name.

And in many cases, the same applies to macros already. E.g. many users of the syn crate use it because it represents Rust syntax as a tree.

There is a point to be had about syn-like capabilities not being available to macro_rules macros, and as far as I understand that’s actually a possible area of focus in the near future:

I also have an RFC in progress ("macro fragment fields") to allow macro_rules! macros to better leverage the Rust parser for complex constructs. Over the next 6 months, I'll shepherd and refine that RFC, and design extensions of it to help parse additional constructs. (I expect this RFC to potentially require an additional design discussion before acceptance.) The goal will be to have enough capabilities to simplify many common cases of attribute macros and derive macros.

(^^^^^ quoted from the section linked above; note that this text is from a proposed project goal; so every statement should be understood with the context in mind that any of this can still change, or even end up not being selected as an official goal at all)

I’m starting to become increasingly less convinced I understand what point you’re making when using the term “functions” here :thinking:

Now we’re talking about two-way convertibility? So not even actually any “de-sugar”ing? Or … if it isn’t a two-way conversion that can turn back into the original, unmodified code, then how else should this be useful for code editor tooling?

I’m not sure I understand this point either. Implementing language features before finalizing the syntax is already something that’s regularly happening. The ? operator, .await, &raw const … / &raw mut … operators all had preliminary versions of the syntax. How does this relate to macros?


Your title starts with “Syntax sugar makes macros inconvenient”. I believe it would help if you gave some examples for the “syntax sugar” you are talking about, and for the inconvenience it causes. I love abstract discussions, too, but it if’s both highly abstract & general, and (as you acknowledge yourself) not remotely rigorous, it’s very hard to even understand what’s being talked about. Some very concrete examples would help a lot :wink:

1 Like

I'm a bit surprised by that. I'll have to take a closer look at how it works.

There, I'm talking about how Lisp can be described as the text representation of an abstract syntax tree. With each function being a node, and its arguments being that node's children. Probably poorly worded.

This is why I added the extra clarification about not losing information on the last point. Is de-sugaring an irreversible process? Going back to the example of a+b vs a.add(b) (even if it isn't as strictly applicable as I had thought), I could easily see an automated conversion back and forth between the two versions. I am assuming that the "sugared" version is strictly preferable, so keeping track of intentionally non-sugared code isn't a factor.

"allow in more cases" is probably a better way to phrase it (I have no concrete examples at this time)

100% agree, but I don't have any, because I'm trying to piece together a jumble of half-remembered vibes and hunches into a semblance of coherency. Closest I've got is annoyance at having to account for generics when writing a declarative macro (just the vibe, no actual code).

Rather than "syntax sugar", it might be more accurate for me to say "any syntax at all", because de-sugaring everything to the extreme is essentially just parsing down to an abstract syntax tree, which could then be put to text in a Lisp-like syntax.

One of the design axioms in that RFC you linked sounds quite close to what I'm getting at:

Macros should not have to recreate the Rust parser (or depend on crates that do so). Macros should be able to reuse the compiler's parser. Macros shouldn't have to parse an entire construct in order to extract one component of it.