Summary
Allow proc-macros have some side-effect. Side effect includes logging files and modifying (static) variables which may keep across the full compile procedure.
It is worth mention that, allowing side-effect does not means move proc-macros out of sandbox. With proper design, we could allow sandboxing the proc-macro, with its side-effect remains.
Motivation
Register function
There is a need that, some function defined in different locations, must be registered together. For example, R needs a R_init_packageName function to initialize the callable functions. There are currently 3 ways to achieve that, but only the most ugly one is acceptable.
DRY (Don't Repeat Yourself)
According to the doc, Don't Repeat Yourself is one of the reason to write macros. If we neglect this rule, the register function could be simplified to, just repeat the function again:
use extendr_api::prelude::*;
/// Return string `"Hello world!"` to R.
/// @export
#[extendr]
fn hello_world() -> &'static str {
"Hello world!"
}
// Macro to generate exports.
// This ensures exported functions are registered with R.
// See corresponding C code in `entrypoint.c`.
extendr_module! {
mod rext;
fn hello_world;
}
As you can see, the fn hello_world is repeated twice. and thanks to this feature, rextendr won't dealing the register function.
Considering that, the reason writting a macro is that, we do not want to repeat, and we may easily forget register the new function manually, thus extendr's way might not a good enough choice.
Safety
Another R plugin crate, savvy, using a different rule, instead writting macro manually, savvy choosing a new tool, savvy-cli, to write wrappers directly.
savvy-cli update
It is much simpler than rextendr, but it also violate a rule, that is safety. Since there are some plans that sandboxing Rust's proc-macros and build scripts, the proc-macros and build scripts could be regarded more safer than any other executables (although currently not).
After the sandbox is ready, the cli method should be regarded as unsafe method (again, although currently not).
Some dirty attempt
Actually, make a register function with proc-macro is possible, but the proc-macro's code could be very dirty:
Let us started with the documented example:
use proc_macro::TokenStream;
#[proc_macro_attribute]
pub fn show_streams(attr: TokenStream, item: TokenStream) -> TokenStream {
println!("attr: \"{}\"", attr.to_string());
println!("item: \"{}\"", item.to_string());
item
}
With this macro, such program complies:
#![feature(custom_inner_attributes, prelude_import)]
#![tests_pm::show_streams]
fn main() {
println!("Hello, world!");
}
and the proc-macro yields:
attr: ""
item: "#![feature(custom_inner_attributes, prelude_import)] #[prelude_import] use
std::prelude::rust_2021::*; #[macro_use] extern crate std; fn main()
{ println!("Hello, world!"); }"
Using such proc macro as an inner-attribute, with a small parser, Rust has the ability to write a register function, but the cost is, the proc macro must read all the codes, and the parser must have the ability to expand macros in order to prevent the missingness of macro generated functions.
Since it is possible to write macros that grab all the calls, and there exists the need that grab all calls together, there is no reason disallowing visit or modify a static (global) variable.
IO
The most important thing is IO, some people think IO should be banned since proc-macro should do nothing but permute symbols. But such opinion ignores some special conditions.
In case the output non-binary file is really needed (for example, when writting a plugin for another language, we should write doc for the plugin user, rather than Rust user, thus we cannot just send rust doc directly and say look, that's the doc.).
Still, we could write an inner-attribute macro, makes a pseudo doctest target that just prints all the documents for FFI, then tell user running cargo doc to generate the FFI documents, but that's very direy, and we need a cleaner way to write such documents.
Explanation
Since currently, the proc-macro will only started and exit once while compiling a crate, this RFC is mainly for documenting its behavior.
Currently, macros are expanded in order even with a std::thread::sleep_ms(1000):
use tests_pm::*;
#[show_streams] // first
fn foo(){}
#[show_streams] // second
mod foo{
#[tests_pm::show_streams] // third, even it is in another file.
fn foo(){}
}
#[show_streams] // forth
fn main(){}
Since the program only opened up once, the static variable is reliable to store things, and since the expanding order is fixed, there is not so much ambiguous.
Drawbacks
- proc-macro in parallel is disallowed since we need the macros expand in order.
- it makes proc-macro harder to debug, since each time a proc-macro is called, it might modify the global status and make the function harder to trace (although using a global inner attribute instead has even more problems.)
Rationale and alternatives
As we discussed above, writting a proc-macro and use it as an inner attribute is the solution which could be infered through documented features.
Although proc-macro cannot output things by self, it could easily create an output binary executable, which output all the things proc-macro want to yield.
Both things are (not so well) documented, but will increase the complexity of code.
As for current and pending alternatives
- Idea: stateful procedural macros Although this macro seems more flexible, such macro needs lots of implementations.
- Global Registration (a kind of pre-rfc) This suggests define a opaque type which could be read at runtime. With side effect of proc-macros, we could easily construct such a slice.
Prior art
As this RFC mentioned above, for writting R plugins, there are 2 crates, rextendr uses export_modules! macro, forcing users export the function again, savvy do not need this, but instead, it needs to run a cli command to generate a C wrapper.
Unresolved questions
a) Should we adding additional grammar about such macro with side effect?
A reasonable choice is adding a ! to attributes:
#[with_side_effect!(...)]item
The item could be {} if unnecessary.
b) Is it possible to make macros expanded into source file directly?
#[attr!] // expand first, write to foo.rs
foo
mod foo; // use the generated source directly
Seems a real chaos, but might helpful debugging the macro.
Future possibilities
- There could be an additional config (maybe in Cargo.toml) to decided whether allowing the proc-macro have side effects. Such support might be done after disable a feature is allowed.