I’m not an expert by any means. But I do have some experience with procedural macros and how they’ve been done in the Scheme/Racket community/literature.
Why I’m scared
Procedural macros + modules seems to have broken the Scheme community – most notably, Racket seems to have splintered off due to the schism over simplicity vs. power in the Scheme language. I don’t have a source, but it seems to me that Scheme has reverted to its R5RS roots in its most recent standard of R7RS-small, and procedural macros were carved out of the language.
Breaking hygiene
Is breaking hygiene within scope for procedural macros? It appears that Rust is already equipped on nightly with reengineering attempt to adjust the parser to produce syntax object-like things with “sets of scopes” to unique define what scope a binding falls in (similar to Flatt’s 2016 “Bindings as Sets of Scopes” for Racket’s new macro expansion scope-tracking system). Furthermore, how much affordance are given to macros that are used incorrectly if they break hygiene? How nice will the error messages look?
Security concerns
Non-procedural macro crates can invoke procedural macro crates. Since procedural macro crates can be arbitrary Rust code, they are also free to produce any side-effects and introduce bugs into the compiler runtime, as a whole. Imagine: when a procedural macro crate fails to terminate or creates a fatal error, what will stop users from thinking, “Oh, wow, Rust is failing,” when it’s actually a procedural macro crate that’s causing the problem? From what I understand,
Isolation of procedural macros
Could it be possible to isolate the execution of procedural macros? And is there a way to limit the types of side-effects that are able to be performed by the compiler proper? I’d like to think a permissions system could be done where the crate can be given access to a camera or a database through a permissions-based system so that only certain compile-time crates would be able to access “juicy features” so that if anything goes wrong, you’d be able to pinpoint the problematic crate in question.
From time to time people open up issues on the main Rust repo about compilation failures coming out of procedural macros breaking. We point them to the right place and close the issue. It's not a particularly significant problem in practice.
This is not really a concern unique to Rust code, or even to procedural macros in Rust code. Build scripts do the same but happen to run outside of the compiler rather than as plugins inside of it. If you're building a C project, you're probably running a configure script and then make, both of which can do arbitrary things. You can use standard system features like chroots and cgroups to limit the privileges that the build processes have, but you fundamentally need some level of trust in the things you're building.
What worries me more is how procedural macros are going to interact with RLS… Like, you can do arbitrary expensive computation in such a macro, and you’ll have to run it after every keystroke…
With an optimal implementation of incremental compilation shouldn’t you only need to re-run proc macros after either the proc-macro code changes or the tokens it’s applied to change? I guess it depends on if they’re defined as being const, or if there’s a way to mark them as being const.
I have no idea how incr comp is handling them now, or what the plans are, or what’s required for soundness. But common decency suggests that libraries should strive to make their proc macros const. E.g. I wouldn’t be surprised if some of the impetus for diesel going from recommending infer_schema!() in code to $ diesel print-schema > schema.rs externally is that it’s much better for the constness of code.
Yeah, if the procedural macro is a pure function of the input token stream (i.e., it doesn't talk to database, and doesn't try to read other source code, or do name resolution or type inference), than it should be amendable to pretty robust caching.
Isn't const orthogonal with side effects, though? Suppose a procedural macro relies on data read in from an external database -- if Rust is not aware of the change, how can one be sure to re-invoke the macro? I thought Rust doesn't have purity as a first-class feature of procedures anymore?
I’m an outsider to the Rust community, but a long-time core developer of Racket. The reasons that the Scheme community is not unified, and that Racket doesn’t use the name Scheme anymore, are not about the existence of procedural macros, and disagreements about how they should work have not been the most important factor splitting the communities.