Plans for quasi-quoting libsyntax

Is there any plan on having quasi-quoting for libsyntax? It'd be useful both for codegen within the compiler (example) and compiler plugins. Right now, one either constructs the AST item manually or creates it from a string using parse_* (but really just dumping it into a file because the spans are all wrong). Ideally, this would be done using existing quasi-quoting machinery.

"Why not just use the normal quote?", you ask. Unfortunately (but with good reason) the conversion from proc_macro::TokenTree to syntax::tokenstream::TokenStream is private, so there'd need to be a special quote for libsyntax to avoid re-parsing the stream.

My hunch is that it'll be a long time before this happens (if ever), but I was wondering if it's on the roadmap. In any case, I'd like to express support to any existing efforts!

My understanding is that there's a strong desire to do so, but that doing so would require great care to avoid accidentally making some key bit of how Rust parses syntax into forever-stable API.

Would it not be just as unstable as all of the "public" methods in syntax::parse*? Probably the easiest thing to do would be to expose ToInternal else have to reimplement the whole thing in terms of libsyntax TokenTree. That'd make a really weird dependency loop, though.

* RIP pprust :wink:

Quasi-quoting machinery generating AST existed, but was removed in https://github.com/rust-lang/rust/pull/51285.
I don't know why it wasn't used in the compiler's built-in macros (perhaps it was at some point?).

The general long term plan is to migrate built-in macros to the same token-based model that proc macros use, which will obsolete AST builder interfaces entirely.
Some form of quasi-quoting would be useful to do that migration.
Not sure about details, perhaps it will be based on syntax::tokenstream::TokenStream rather than on proc_macro::TokenStream because we may want to avoid the proc macro server/client overhead in this case.

1 Like