# Macro meta functions

**URL:** <https://internals.rust-lang.org/t/macro-meta-functions/16743>\
**Category:** language design\
**Created:** [June 3, 2022, 7:48pm UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743 "2022-06-03T19:48:24Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [June 3, 2022, 7:48pm UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/1 "2022-06-03T19:48:24Z")

</div>

[RFC #3086](https://github.com/rust-lang/rfcs/pull/3086) defines a metafunction syntax for declarative macros, e.g. `${ignore(...)}`. As part of determining exactly what the "arguments" should look like, I've been tasked with collecting a list of currently proposed metafunctionality (included below). I'd also like to ask everyone else: what other kinds of metafunctionality would be useful for writing declarative macros?

| metafunction | argument kinds | purpose | source |
| --- | --- | --- | --- |
| `assign` | (new) binder, token stream | assign a binder for later use in expansion | [Zulip: macro\_rules back to repetition root (comment)](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/macro_rules.20back.20to.20repetition.20root/near/284159362) |
| `call_site`, etc. | token tree / token stream | specify a named hygiene for token expansion | @CAD97 spitballing |
| `concat` | ? | concatenate identifiers into a new identifier | [rust-lang/rfcs#3086](https://github.com/rust-lang/rfcs/pull/3086) |
| `copy_span` | binder, token tree | apply the span of a binder to a token tree | [An idea for breaking full macro hygiene](https://internals.rust-lang.org/t/an-idea-for-breaking-full-macro-hygiene/16620) |
| `copy_span` | token tree, token tree | apply the multispan of one token tree to another | [An idea for breaking full macro hygiene](https://internals.rust-lang.org/t/an-idea-for-breaking-full-macro-hygiene/16620) |
| `count` | binder | get the number of times a macro binder repeats | [rust-lang/rfcs#3086](https://github.com/rust-lang/rfcs/pull/3086) |
| `count` | binder, integer | get the number of times a macro binder repeats at a given repetition depth | [rust-lang/rfcs#3086](https://github.com/rust-lang/rfcs/pull/3086) |
| `eager` | token stream | do eager expansion of macros | @CAD97 spitballing |
| `if` | ? | do control flow things in macro expansion like a template engine | [Zulip: macro\_rules back to repetition root (comment)](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/macro_rules.20back.20to.20repetition.20root/near/284159895) |
| `index` | n/a | get the current repetition index | [rust-lang/rfcs#3086](https://github.com/rust-lang/rfcs/pull/3086) |
| `index` | integer | get the repetition index at a given repetition depth | [rust-lang/rfcs#3086](https://github.com/rust-lang/rfcs/pull/3086) |
| `ignore` | binder | treat the binder as used for the purpose of repetition without expanding it | [rust-lang/rfcs#3086](https://github.com/rust-lang/rfcs/pull/3086) |
| `ignore` | token stream | do expansion of the token stream for side effects (e.g. controlling repetition) but discard the results | [rust-lang/rfcs#3086](https://github.com/rust-lang/rfcs/pull/3086) |
| `match` | ? | do control flow things in macro expansion like a template engine | [Zulip: macro\_rules back to repetition root (comment)](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/macro_rules.20back.20to.20repetition.20root/near/284159895) |
| ? | ? | Cartesian product of repetitions | [rust-lang/rust#74524](https://github.com/rust-lang/rust/issues/74524) |

---

<div class="post-metadata">

**Author:** ![Lej77](https://avatars.discourse-cdn.com/v4/letter/l/e5b9ba/32.png) [@Lej77](https://internals.rust-lang.org/u/Lej77)\
**Post date:** [June 3, 2022, 11:35pm UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/2 "2022-06-03T23:35:17Z")

</div>

Maybe this (metafunctions) can be used to improve the generics situation for declarative macros. There could be a `generics` fragment and then some metafunction that does something similar to [syn](https://docs.rs/syn/1.0.96/syn/index.html)::[Generics](https://docs.rs/syn/1.0.96/syn/struct.Generics.html#)::[split\_for\_impl](https://docs.rs/syn/1.0.96/syn/struct.Generics.html#method.split_for_impl).

For example, currently its quite easy to accept simple Rust types in declarative macros:

```rust
pub trait Trait {}

macro_rules! simple_derive {
    (
        $(#[$attr:meta])*
        $vis:vis struct $name:ident {$(
            $(#[$field_attr:meta])*
            $field_vis:vis $field_name:ident : $field_ty:ty
        ),* $(,)?}
    ) => {
        $(#[$attr])*
        $vis struct $name {$(
            $(#[$field_attr])*
            $field_vis $field_name : $field_ty,
        )* }
        impl Trait for $name {}
    }
}

simple_derive!(
    /// Attribute
    pub struct Example {
        /// Attribute
        pub value: usize,
    }
);

```

But it becomes harder when the types are generic, so maybe something like the bellow could be allowed to make it easier:

```rust
macro_rules! simple_derive {
    (
        $(#[$attr:meta])*
        $vis:vis struct $name:ident
        $generics:impl_generics $(where $bounds:bounds)? 
        {$(
            $(#[$field_attr:meta])*
            $field_vis:vis $field_name:ident : $field_ty:ty
        ),* $(,)?}
    ) => {
        $(#[$attr])*
        $vis struct $name $generics $(where $bounds)? {$(
            $(#[$field_attr])*
            $field_vis $field_name : $field_ty,
        )* }

        impl $generics Trait 
        for $name ${ty_generics($generics)}
        where
            ${bounds_generics($generics)}
            $($bounds)?
            $($field_ty: Trait,)*
        {}
    }
}

simple_derive!(
    /// Attribute
    pub struct Example<'a, T, const N: usize> {
        /// Attribute
        value: &'a [T; N],
    }
);

```

---

<div class="post-metadata">

**Author:** ![afetisov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/afetisov/32/8508_2.png) [@afetisov](https://internals.rust-lang.org/u/afetisov)\
**Post date:** [June 4, 2022, 1:10am UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/3 "2022-06-04T01:10:33Z")

</div>

Am I the only one who finds that ad-hoc definitely incomplete list of syntax extensions for an already complicated system an abomination?

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [June 4, 2022, 2:55am UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/4 "2022-06-04T02:55:06Z")

</div>

Note that _all_ of these are just proposed ideas, and every one I could find at that. The purpose of collecting this list is specifically to define what we actually need/want out of declarative macro metafunctions (and if we want them at all or should unaccept that part of RFC#3086).

And if you're saying the list itself is just poorly formatted: sorry, I tried my best ☹

The idea of providing some of this functionality is that it will allow more macros which are currently required to use the much more complicated proc macro system to become decl macros instead. So in that way, the idea/hope is that by giving decl macros more power we decrease the required complexity of macros which can then move from being implemented as proc macros to being decl macros instead.

---

<div class="post-metadata">

**Author:** ![y86-dev](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/y86-dev/32/9867_2.png) [@y86-dev](https://internals.rust-lang.org/u/y86-dev)\
**Post date:** [June 4, 2022, 7:28am UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/5 "2022-06-04T07:28:06Z")

</div>

Hmm I do not really like the assign operation, instead I would like to see more classic control flow operators on meta variables. When I first learned macros, I asked myself why they were not implemented using those and I have not found a deal breaking problem with them.

So why do we not just allow something like (example from the assign zulip link):

```rust
macro_rules! layered_repetitions {
    // Input
    {
        $($path:ident).* {
            $($field:ident),*
        }
    } => {
        // Output
        $@for $field in $field {
            // Repeat complete $path for every $field
            $($path).*
            $field
        }
    }
}

```

Syntax of course open for bike shedding later, it might even be possible to need no additional prefix.

If you want to iterate over two variables at the same time, then

```rust
macro_rules! print_fields {
    {
        struct $name:ident {
            $($fields:ident: $types:ty),*
        }
    } => {
        fn print_struct(s: &$name) {
            $@for ($field, $type) in ($fields, $types) {
                println!("{}: {:?} ({})", stringify!($field), s.$field, stringify!($type));
            }
        }
    }
}

```

There might even be some more syntax to also add indexing and other goodies. I could also imagine the use of other control flow items such as `match`, `if` etc.

---

<div class="post-metadata">

**Author:** ![anon32976453](https://avatars.discourse-cdn.com/v4/letter/a/49beb7/32.png) [@anon32976453](https://internals.rust-lang.org/u/anon32976453)\
**Post date:** [June 4, 2022, 7:31am UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/6 "2022-06-04T07:31:07Z")

</div>

To my mind, this work is a result of Rust still firmly stuck in the sunken cost fallacy. We are still trying to add on more workarounds on top of an unfit design. The current macro\_rules system as it currently stands has foreign syntax and semantics and simply isn't fit for purpose, given we all hope Rust would be the language of the next 50 years.

We could do much better by taking inspiration from Nemerle and more recently [https://www.circle-lang.org/](https://www.circle-lang.org/), projects that have much better thought out macro systems. E.g instead of trying to force a scheme like recursive macro definition and trying to add to it metafunctions to count numbers of repetitions, wouldn't it be way WAY easier to just use plain Rust for loops and iterators?

---

<div class="post-metadata">

**Author:** ![y86-dev](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/y86-dev/32/9867_2.png) [@y86-dev](https://internals.rust-lang.org/u/y86-dev)\
**Post date:** [June 4, 2022, 12:58pm UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/8 "2022-06-04T12:58:59Z")

</div>

I feel like there is some middle ground between "We implement C++ templates" and the current macro system.

The macro system has some poor limitations and one is thus required to write a proc-macro. I do not like that, because many people will never want to touch proc-macros and they always are such a hastle to integrate into your projects (add a helper crate etc.). I definitely agree with you that we do not want to create a half baked system, but if we can improve the current marco system in such a way that it is easier to understand and a little bit more powerful then I think we should do that.

For example, why do I need a proc-macro to concatenate `ident`s?

I like the argument destructuring approach where you specify how the input looks like, but expanding the macro itself should be more ergonomic.

iirc in the future `macro` will be used to declare macros, we could use that opportunity to also define a new macro syntax that would deal with these problems.

---

<div class="post-metadata">

**Author:** ![chrefr](https://avatars.discourse-cdn.com/v4/letter/c/e480ec/32.png) [@chrefr](https://internals.rust-lang.org/u/chrefr)\
**Post date:** [June 5, 2022, 10:28pm UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/9 "2022-06-05T22:28:19Z")

</div>

Some metafunction to break an AST component back into tokens?

Anyway, I agree with @afetisov that this is probably a bad idea. Resorting to proc macros isn't so terrible; they can be way more readable.

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [June 5, 2022, 11:30pm UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/10 "2022-06-05T23:30:32Z")

</div>

Even ignoring the cost of bringing in syn, I personally find

```rust
macro_rules! rpc_impl {
    {
        fn $f($($arg:ident: $Arg:ty),* $(,)?) $(-> $Ret:ty)? {
            $($body:tt)*
        }
    } => {
        fn $f($($arg: u32),*) $(-> ${ignore(Ret) u32) {
            let _ret = {
                $(let $arg = <$Arg>::rpc_decode($arg);)*
                $($body)*
            };
            $(<$Ret>::rpc_encode($arg))?
        }
    }
}

```

much easier to understand than all of the ceremony required to set up an attribute macro.

Declarative macros are also much simpler for IDEs to support (even with more complicated meta functionality) because they're still purely declarative, so the IDE can know that e.g. you're editing in `$($body:tt)*` above and that editing there won't change the macro expansion. Proc macros are [formally impossible](https://github.com/matklad/proc-caesar) to track in such a fashion _because_ they're defined procedurally.

Then there's also the extra bit that proc macros have to be compiled separately from the main crate. This can be improved in the future such that it can be part of the same package, but it will always need to be a separate build-time part rather than just in the normal crate.

---

<div class="post-metadata">

**Author:** ![y86-dev](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/y86-dev/32/9867_2.png) [@y86-dev](https://internals.rust-lang.org/u/y86-dev)\
**Post date:** [June 6, 2022, 8:17am UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/11 "2022-06-06T08:17:53Z")

</div>

I agree with @CAD97 proc-macros are very useful, but I think we can make decl macros work for many cases where we currently need a proc-macro.

What also came to my mind was named matches (I do not know a better name). So for example:

```rust
macro_rules! my_macro {
    ($p(print)? $a:ident + $b:ident) => {
        if ${exists($p)} // how to specify print here?? {
            println!("{} + {}", stringify!($a), stringify!($b));
        }
        $a + $b
    };
}

```

So I would like to assign a name to some fixed construct in the input. A workaround is at the moment to use `ident`/`tt` and then later in a helper macro require the concrete construct, but then the error is less readable and the macro always drags around an `ident` that actually is only ever something constant.

I do not know if this fits here, but I think that we could use better support for inner helper macros. They should not be callable from outside the outer macro and should be easier to declare (maybe another parameter syntax? but this might be a bit too extreme).

---

<div class="post-metadata">

**Author:** ![dhm](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/dhm/32/4879_2.png) [@dhm](https://internals.rust-lang.org/u/dhm)\
**Post date:** [June 9, 2022, 2:50pm UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/12 "2022-06-09T14:50:36Z")

</div>

> [@CAD97](#):
>
> I'd also like to ask everyone else: what other kinds of metafunctionality would be useful for writing declarative macros?

- The `assign` one is cute; we'd have to see how it fares: I don't see it as paramount, but I imagine it being quite convenient 🙂
- `concat` is so direly needed that it could indeed warrant being special-cased as a special metavariable function.

But the main thing we need, _really_, is this so-called `eager`.

## An _eager_ expansion operator such as `${macro!(…)}` is a Game Changer

Nothing\[1\] more is needed. That is, rather than trying to come up with dozens and dozens of metavar combinations, we can just define custom ones with macros and `eager`.

For instance, to concatenate identifiers:

```rust
#[test]
fn ${concat_idents!($fname, _test)} ()

```

to replace `${concat(…)}`, as well as `${respan!(…)}`, and so on and so forth.

* * *

This means that a very important question to sort out is whether this `eager` mechanism will be featured:

- If so, then let's not overcomplicated metavar expressions, and just let them defer to user-defined helper macros (now, the stdlib could consider adding more of those, but then these would just be T-libs question rather than T-lang ones!);

- If not, then we'd need T-lang extensions for every new usability need, or just a lack of support as we currently do.

* * *

Aside: unrelated, but the "named" capture groups idea, which has already been brough before, in this very thread, and in this forum as a standalone thread, is indeed quite important to get rid of the `$(@$($name:tt)?)?` hacks that are so pervasive nowadays (in other words, the current _status quo_ in this regard is horrendous). But since named capture groups is yet another whole level of macro modification, I think a way simpler and more elegant solution, which was also suggested in that thread (but not in this one!), is to be able to have `:empty` capture groups.

So, to take @y86-dev's example, this would give:

```rust
macro_rules! my_macro {(
    $(print $if_print:empty)?
    $a:literal + $b:ident
) => ({
    $($if_print
        println!(…);
    )?
    $a + $b
})}

```

- Currently, we have to write:

* * *

1. EDIT: this term was a bit too adamant, my bad. Obviously there are still meta(var)-based functionality such as index, count, and whatnot which would require _dedicated_ metavar functions, @CAD97, you're completely right 💯. But I suspect there won't be that many of those: the vast majority of use cases ought to be manageable using macro helpers

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [June 9, 2022, 5:59pm UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/13 "2022-06-09T17:59:59Z")

</div>

> [@dhm](#):
>
> ## An _eager_ expansion operator such as `${macro!(…)}` is a Game Changer
> 
> Nothing more is needed.

That's not quite true: `index` and `length` (the RFC-accepted ones) aren't doable with just macros, as they need access to compiler context. `assign` is just a cute assistant, but also isn't possible without compiler support. But I think you're correct on the rest.

> **said macros**
>
> ```rust
> macro call_site($($tt:tt)*) { /* proc macro */ }
> macro concat_idents($($i:ident)+) { /* proc macro */ }
> macro with_span($donor:tt=> $($tt:tt)*) { /* proc macro */ }
> 
> macro count($($tt:tt)*) {
> 0 $( + 1 ${ignore!($tt)} )*
> }
> 
> macro if_tt {
> ( if ($($tt:tt)+) { $($then:tt)* } else { $($else:tt)* } } => { 
> $($then)* };
> ( if ( ) { $($then:tt)* } else { $($else:tt)* } } => { $($else)* };
> }
> 
> macro ignore($($tt:tt)*) {}
> 
> ```

---

<div class="post-metadata">

**Author:** ![programmerjake](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/programmerjake/32/5893_2.png) [@programmerjake](https://internals.rust-lang.org/u/programmerjake)\
**Post date:** [June 10, 2022, 9:33am UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/14 "2022-06-10T09:33:39Z")

</div>

`count!` can't be done with a macro like that if you need a literal rather than an expression, e.g. if you want to use `count!` with `concat_idents!`.

---

<div class="post-metadata">

**Author:** ![chrefr](https://avatars.discourse-cdn.com/v4/letter/c/e480ec/32.png) [@chrefr](https://internals.rust-lang.org/u/chrefr)\
**Post date:** [June 11, 2022, 9:35pm UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/15 "2022-06-11T21:35:08Z")

</div>

Then you can use eager expansion twice.

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [June 12, 2022, 3:46am UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/16 "2022-06-12T03:46:53Z")

</div>

This doesn't work; eager expansion only applies to macro expansion, and the `count` I provided expands to an expression like `0 + 1 + 1 + 1 + 1`.

---

<div class="post-metadata">

**Author:** ![chrefr](https://avatars.discourse-cdn.com/v4/letter/c/e480ec/32.png) [@chrefr](https://internals.rust-lang.org/u/chrefr)\
**Post date:** [June 12, 2022, 5:00am UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/17 "2022-06-12T05:00:29Z")

</div>

Ah, correct.

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/system/32/14092_2.png) [@system](https://internals.rust-lang.org/u/system)\
**Post date:** [September 10, 2022, 5:01am UTC](https://internals.rust-lang.org/t/macro-meta-functions/16743/18 "2022-09-10T05:01:07Z")

</div>

Closed on September 10, 2022, 5:01am UTC

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.
