I've realized that derive macros often do a lot of unnecessary work. Consider serde's Serialize and Deserialize derive macros, both of which go through the following steps:
- The
deriveinput is turned into aTokenStream - Rust calles the function in the
serdeproc-macro dylib, giving it theTokenStream - The derive macro in
serdeparses thisTokenStreamintosyn::DeriveInput - Then the derive macro does a bunch of validation on it, and parses it into some internal AST like
serde::SerdeAst - Each derive macro takes this
serde::SerdeAstand generates macro-specific code, depending if we are inside ofSerializeorDeserialize - The derive macro turns the data into a
TokenStreamwhich then is received by Rust. ThisTokenStreamis then also parsed by Rust to verify that it is correct.
Steps 1-4 don't need to happen more then once. Why do they?
Consider these 2 derives:
#[derive(Serialize, Deserialize)]
struct Something { /* ... */ }
First, Rust does all 6 steps for the Serialize macro, and we end up with this:
#[derive(Deserialize)]
struct Something { /* ... */ }
impl serde::Serialize for Something { /* ... */ }
Then, Rust does all 6 steps again for the Deserialize macro, and now we have this:
struct Something { /* ... */ }
impl serde::Serialize for Something { /* ... */ }
impl serde::Deserialize for Something { /* ... */ }
This seems pretty wasteful. What if there was a better way? What if we only needed to do the first 4 steps once, and subsequent derives could re-use it?
Note, specifically for serde: This is not exactly how serde_derive works, but it should be possible to adapt it into a single AST, where the serialize_with attribute is only allowed if serialize == true in the input, as an example.
Proposal
Let's have a #[proc_macro_derive_group] which allows us to place Serialize and Deserialize under a single function that receives the item, then we don't have to repeat those 4 steps for the 2nd time.
With #[proc_macro_derive]
#[proc_macro_derive(Serialize, attributes(serde))]
pub fn derive_serialize(input: TokenStream) -> TokenStream {
let input = parse_macro_input!(input as syn::DeriveInput);
let serde_ast = SerdeAst::parse(input);
// serialize-specific logic
quote! { #impl_serialize }
}
#[proc_macro_derive(Deserialize), attributes(serde)]
pub fn derive_serialize(input: TokenStream) -> TokenStream {
let input = parse_macro_input!(input as syn::DeriveInput);
let serde_ast = SerdeAst::parse(input);
// deserialize-specific logic
quote! { #impl_deserialize }
}
I showed how Rust will use these 2 functions in an earlier code block.
With #[proc_macro_derive_group]
This new attribute lets you avoid doing extra work.
#[proc_macro_derive_group(Serialize, Deserialize, attributes(serde))]
pub fn derive_serde(input: TokenStream, serialize_called: bool, deserialize_called: bool) -> TokenStream {
let input = parse_macro_input!(input as syn::DeriveInput);
let serde_ast = SerdeAst::parse(input);
let impl_serialize = if serialize_called
// serialize-specific logic
Some(quote! { #impl_serialize })
} else {
None
};
let impl_deserialize = if deserialize_called
// deserialize-specific logic
Some(quote! { #impl_deserialize })
} else {
None
};
quote! {
#impl_serialize
#impl_deserialize
}
}
Let's dissect it.
For this code:
#[derive(Serialize, Debug, Deserialize)]
struct Something { /* ... */ }
Rust knows that Serialize and Deserialize belong to a single group, so it just call serde::derive_serde exactly once:
serde::derive_serde(quote! { struct Something { /* ... */ } }, true, true)
After a single expansion, we now have this:
#[derive(Debug)]
struct Something { /* ... */ }
impl serde::Serialize for Something { /* ... */ }
impl serde::Deserialize for Something { /* ... */ }
Then the other derives can kick in.
Benefit: Compile-times are going to improve. We no longer have to repeat the first 4 steps.
It is of course allowed to still do just 1 of the derives: #[derive(Serialize)] or #[derive(Deserialize)], in which case the respective argument will be set to false.
Where this idea came from
I just made an RFC for the #[ignore] attribute: https://github.com/rust-lang/rfcs/pull/3869
In order to test this RFC out, I want to create a crate that re-implements the standard library's derives: PartialEq, Hash, PartialOrd, Ord and Debug
But each derive macro will have a helper attribute ignore_derives so you can write stuff like this:
use derives_with_ignore_derives_attribute::{Debug, PartialEq, Hash};
#[derive(Clone, Debug, PartialEq, Hash)]
pub struct Var<T> {
pub ns: Symbol,
pub sym: Symbol,
#[ignore_derives(PartialEq, Hash)]
meta: RefCell<protocols::IPersistentMap>,
#[ignore_derives(PartialEq, Hash)]
pub root: RefCell<Rc<Value>>,
#[ignore_derives(PartialEq, Hash)]
#[ignore_derives(fmt::Debug)]
_phantom: PhantomData<T>
}
Essentially all logic for the 5 derives will be the same, they'll all just process the DeriveInput a little differently and emit different code.
There should be zero reason why I'd have to parse the TokenStream into DeriveInput 5 times instead of 1.
This feature has the potential to improve compilation performance of derives macros
