It should be possible to have
#[a_macro]
impl Foo {
completely arbitrary syntax that matches rusts core syntax rules with
regards to strings, labels, chars, balanced ()[]{} and so on. after all
this is inside an {}. basically anything youd be able to put on a
functionlike macro.
}
Would make some things look nicer.
Can you give any examples?
Working on an proc-macro implementation of impl trait inside impl but it currently needs to be a functionlike.
Currently only valid ASTs, e.g impl blocks, can be tagged with a proc macro attribute. However, impl Foo { valid token stream } are not always valid AST. It's valid token stream which is expected to be wrapped inside function-like proc macros, e.g. a_macro!{}.
Are you suggesting to relax the rule so valid token stream could also be tagged with proc macro attributes? This breaks the assumption that "proc macro attributes are just attributes, which should not change the syntax".
The benefit is that attributes can carry extra data as arguments. But the limitation is that attributes should not change syntax, as mentioned above.
I don't think proc macro attributes are always nicer than function-like proc macros. The following functionality was first implemented with proc macro attributes, but finally I decided to use function-like proc macros, and found it is more clear: A proc macro that may generates many impl blocks.
Can you please provide a concrete example of a situation where this would improve the status quo, and why function-like macros are such a negative in that situation?
It would lower a layer of nesting (and thus wouldn't introduce a TokenTree::Group), which is always nice.
Additionally if something like what @oooutlk brought up used attributes, it would be automatically compatible (composable) with what we're doing, rather than requiring special support for it.
In other words, this would work:
#[inherent_trait_impls]
impl<T0, T1> Enum2<T0,T1> {
#[def_impls]
impl trait AsRef<[u8]> where _Variants!(): AsRef<[u8]> { // or maybe something different here
// etc
}
}
whereas right now what we're implementing can't see beyond the def_impls! because it doesn't have explicit support for it (nor is explicit support planned -- sorry!)
I personally really like that Rust's attribute macros can only be applied to valid syntax. It makes it easier to read code. I've never really like macros that create a new DSL. I find them harder to read, harder to maintain, and harder to debug.
As for that most recent example, I still don't see the motivation for this proposal. Why would it be better if that worked?
This is not possible when #[derive] is involved, since #[derive] performs cfg-expansion on its input. That is,
#[derive(Foo)]
struct Foo {
#[cfg(FALSE)] removed: u8,
not_removed: bool
}
will invoke the custom derive macro Foo with struct Foo { not_removed: bool }. This is only possible if the attribute target is valid Rust. If we were to allow any valid token stream inside the braces, we would need to add a special case to disallow this when the attribute is #[derive].
Additionally, what you're proposing isn't really 'completely' arbitrary syntax - to make sense, it requires that the impl header be valid Rust. That is, the parser would need to allow this:
#[a_macro]
impl Foo<[u8; { bool; { 1 + 1 }]> {
some arbitrary tokens here
}
but not:
#[a_macro]
impl Foo<[u8; { bool; { 1 + 1 }]> another_token {
some arbitrary tokens here
}
since the parser needs to be able to determine the the { .. } block is 'part of the impl'. This seems very inconsistent, and will probably confuse people encountering it for the first time.
But derive is already special - you can't modify the tokenstream in a derive macro. So it's fine.
That's a property of the derive macro implementation - it doesn't affect the fact that derive must applied to a syntactically valid item.
Also, there's been work done to make derive 'less special', and act more like other attribute macros (https://github.com/rust-lang/rust/pull/79078). Your proposal would make derive more 'special' to achieve a vague goal.
In a bang-proc-macro, the outermost delimited group is not passed to the macro. So, my_macro!(Foo) and my_macro { Foo } both invoke my_macro with Foo.
Why does removing a single layer of nesting justify complicating the rules for attributes and parsing? Would your rule apply to code like this (once expression attributes are stabilized):
#[my_attr]
for val in &[100] {
some arbitary_tokens { here }
}
If that code does work, then I think it will make expressions significantly harder to read, especially for beginners. If that code doesn't work, then your proposal would make both items and #[derive] special w.r.t attribute targets.
Another benefit of attribute macros requiring well-formed input is that, well, macros can assume we'll formed input, and so can the compiler's error recovery mechanism(s), and so can IDE syntax highlighting and intellisence.
With bang-style proc macros that take arbitrary token trees as input, it is provably impossible to implement any assistance (in the general case), whether that be as complicated as type aware autocomplete, or as simple as regex-based syntax highlighting. (proof)
With attribute macros, that (mostly) doesn't apply, specifically because the input has to be syntactically valid. Normal syntax highlighting works, and the standard compiler parser's error recovery can help catch and point out syntax errors. (Intellisence is still broken in the general case, but the common case where it works is much larger, due to the social pressure that if it looks like valid code it should translate to very similar valid code with minimal semantic difference.)
I consider it close to the biggest problem that current macros assume syntactically correct input. This allows them to use a parser that doesn't necessarily give good error messages on syntactically incorrect input, because the compiler already did that step. Changing this will cause nearly the entire ecosystem of proc macro attributes to go from a syntax error being the nice compiler error to proc macro panicked: called Result::unwrap on an Err value: ParseErr with no span information.
All of this just so that attribute macros can be used instead of bang macros.
(And at one point closer to initial non-derive proc macros than now I was favorable towards this proposal. I no longer am due to the points iterated above and in the other posts.)
Proc macros aren't really meant to be full syntax replacements for Rust. For that, you're better off writing a separate file that you compile to Rust, potentially within a proc macro wrapper. (And I'm highly in favor of adding a proc-macro API for "give me a proper TokenStream for this path." This would make compile-to-Rust proc macro embedded languages much more ergonomic, and allow them to lean on Rust's error reporting to point into their own files.)
The idea is to provide syntax extensions, not replacements. It should still be in line with the rest of Rust syntax and all that.
Anyway, finished it, feel free to have a play with it to see the use-cases. https://crates.io/crates/impl_trait
It would be really nice if this "just worked" with things that extend impl trait syntax, like the crate @oooutlk brought up. But currently if we want to compat with that crate we need to explicitly and deliberately parse their macro which is just silly - it seems to encourage us to try to push these syntax changes upstream, even tho we feel this is against Rust's own philosophy which generally attempts to push things into crates instead (like rand, syn, quote, etc - those are all crates instead of being part of std or proc_macro).
When we talked about derive - derive itself is special in that it's a proc macro.
derive gets to define its own rules. It's okay for derive to not allow modifying the tokens, and to pre-apply cfg attributes. Because it's just another proc macro.
(Arguably the way Derive macros are treated - being specially defined in proc_macro - is what's special/needs to be extended to arbitrary proc macros. That is to say - why can't one make a custom derive-like attribute today?)
Nevertheless, we could start a whole separate thread about this. :)
This would still be true, attribute macros are applied outside-in so your macro would still be passed in the unexpanded form of their macro (or theirs yours, depending on the order).
This isn't true for macro_rules! macros though, right? Those too have to be syntactically valid I think. And an IDE should be able to look at just the inputs to get some help for highlighting. (I think)
Yes but we wouldn't see it as a macro but just as some attributes, which we pass through.
Specifically, it would not be an opaque blob of [Ident, Punct('!'), Group] to us, but just another attribute on an impl trait instead. The attribute would simply get passed through and their macro would see the expanded form of the impl trait block, whereas the ident/punct/group stuff would need to be specially handled by us.