Idea: postfix let as basis for postfix macros

These are two separate language features, but they should be discussed together because each one is not that useful on its own but together they can be extremely useful.

The first is postfix let which works like this:

foo.let bar {
    baz()
}

Is syntactic sugar for:

{
    let bar = foo;
    baz()
}

Pretty simple, but not very useful (though it will bring some joy to those who request pipelining functions / operators). It does, however, enable the next feature I'm going to suggest:

Postfix macros. A feature that was suggested and discussed many times before, but always had the problem of "capturing" the expression it operates on. In foo().bar().baz!(), how would baz! receive foo().bar() as parameter? And won't it be surprising and confusing that the macro can modify syntax written outside its body? But if it acted like a normal macro and only emit tokens that'll take its syntactical spot in the token stream it won't be that useful since all it'll be able to do is slap methods calls and fill their arguments.

But if we have postfix let, postfix macros could behave like regular macros and only replace theirselves, but still be useful because a postfix let is not a closure so it allows flow control!

For example, if we look at old try! (ignoring that it is now a keyword), we could write it as a postfix macro like so:

macro_rules! .try {
    () => (.let value {
        match value {
            $crate::result::Result::Ok(val) => val,
            $crate::result::Result::Err(err) => {
                return $crate::result::Result::Err($crate::convert::From::from(err))
            }
        }
    });
}

Or await!, when the idea was to have it as a macro and base async/await on generators:

macro_rules! await {
    () => (.let mut future {
        let future = &mut future;
        // The above borrow is necessary to force a borrow across a
        // yield point, proving that we're currently in an immovable
        // generator, making the below `Pin::new_unchecked` call
        // safe.
        loop {
            let poll = ::futures::__rt::in_ctx(|ctx| {
                let pin = unsafe {
                    ::futures::__rt::std::mem::Pin::new_unchecked(future)
                };
                ::futures::__rt::StableFuture::poll(pin, ctx)
            });
            // Allow for #[feature(never_type)] and Future<Error = !>
            #[allow(unreachable_code, unreachable_patterns)]
            match poll {
                ::futures::__rt::std::result::Result::Ok(::futures::__rt::Async::Ready(e)) => {
                    break ::futures::__rt::std::result::Result::Ok(e)
                }
                ::futures::__rt::std::result::Result::Ok(::futures::__rt::Async::Pending) => {}
                ::futures::__rt::std::result::Result::Err(e) => {
                    break ::futures::__rt::std::result::Result::Err(e)
                }
            }
            yield ::futures::__rt::Async::Pending
        }
    })
}

Which will allow foo().await!().try!() without designated syntax (which is still a good idea in such common cases, but it's also good to be able to create macros for the less common but still common cases)

3 Likes

Another, simpler option is to desugar foo.mac!() to mac!(foo,) or mac!(foo), this way all existing macros can more or less work as is (or with a minor modification). We don't have to introduce 2 new syntactic forms at once if we only care about postfix macros.

11 Likes

It's an interesting idea that doesn't seem to have been explored previously. Perhaps it generalizes to offer other capabilities as well. I'd like to see that explored by the community on this forum before deciding that there's a less-confusing approach to postfix macros (which latter I very much want).

6 Likes

It doesn't seem good to introduce this only so that postfix macros aren't the new feature be added. Postfix macros are a more natural extension, following the parallel between UFCS and method call syntax.

Meanwhile, since binding scopes in Rust already follow a top-to-bottom, left-to-right order (i.e. source order), this doesn't seem to have any significant benefits other than supporting postfix macros, at which point we could just do postfix macros instead.

5 Likes

Also note that the postfix let syntax can be done via a post-fix macro

1 Like

I personally prefer the obvious postfix-macro form. It's easier to understand, less confusing, with fewer side-effects than the proposed "postfix let" syntax. I was just wondering whether the proposed "postfix let" syntax offered any other not-so-obvious capabilities.

6 Likes

IMO the thing that’s really proposed here is not really a postfix let feature (OP themselves admit that the postfix let on its own is not too useful) but an idea how to do postfix macros without confusing syntax. And it offers a common denominator for what existing postfix macro-like features already share. A more naive postfix-macro feature can lead to something that looks like a normally evaluated expression suddently not being one. The idea that’s presented here is essentially that some().complex().expression().macro_call!(xyz) wouldn’t just pass some().complex().expression() syntactically to macro_call (which by-the-way would have to be given to the macro pre-parsed as an expr anyways, meaning that without any additional features that turn it back into token trees you couldn’t really inspect it syntactically with macro_rules anymore anyways). Instead the expression being passed syntactically, it would be evaluated to create a temporary and then the macro would be allowed to refer to this temporary.

From this point of view, the idea of creating (something like) a temporary from the expression means we translate the macro by creating a block, assigning the expression and putting the actual macro expansion below that inside the same block. To still allow macros that expand to statements one might actually additionally want expand this idea to add capabilities that allow adding extra statements outside of such a block, too, (for example for creating new variables in the enclosing scope).

3 Likes

It could still be passed to another macro that'll parse it.

Wouldn't this have implications on lifetimes? Consider:

foo().bar!();
baz();

One would usually expect the value of foo() will be either destroyed after foo().bar!() or moved to somewhere else as part of .bar!(), but if it needs to be put in a temporary in a scope that allows defining variables that can be used outside, it will survive and will only be destroyed after baz();. This is quite surprising a behavior.

In addition, why should the arguments to postfix macros be passed differently at all? The whole point of a macro is to operate on syntax. Note that evaluating and passing a temporary is strictly less general, because it's possible for a macro author to perform this conversion if needed, but it's not possible to go back from an evaluated temporary to its original syntactic form where the expression came from.

Furthermore, the different behavior between postfix and non-posftix macros would really cause lots of confusion.

4 Likes

I was deliberately unspecific in my comment because this wasn’t an important point for me, and also an optional idea, but just to be clear what I had in mind was not for the “temporary” to live longer than the expression by default but just to maybe allow some explicit way of passing data out into the enclosing scope (admittedly then .bar!() could indeed still implement this kind of surprising behavior). On a second thought it doesn’t make too much sense, you could just as well use normal prefix macros for translations into statements like that. It probably is much nicer to have postfix macros be for expressions only.

An additional thought is that maybe an even nicer translation and one that’s more consistent with ordinary methods would be one where the “temporary” behaves like an actual temporary and stays alive for the whole expression, perhaps the macro gets the ability to create even more temporaries on its own as well, that are still around for a function or method call surrounding the macro invocation.

That’s not true. Currently, once something is parsed once in macro_rules it cannot be split up into single tokens or anything again. Take this playground for an example.

Gotta say postfix-let is very unreadable.

However, there has been a comparatively better alternative proposal: postfix-match (as mentioned in https://boats.gitlab.io/blog/post/await-decision/#space-await-the-most-viable-alternative), which supports every feature OP needed.

foo.match {
    bar => baz(),
}
macro_rules! .try {
    () => {.match {
        $crate::result::Result::Ok(val) => val,
        $crate::result::Result::Err(err) => return ...,
    }}
}

macro_rules! .await {
    () => {.match {
        ref mut future => loop {
            ...
        },
    }}
}
9 Likes

The noisy macro syntax (bar!() instead of just bar()) is supposed to serve as a warning that this is not a normal function, and normal expectations do not apply.

I guess there's a bit of a chicken-egg problem that the parser has to parse tokens before macro invocation as an expression to find the postfix macro first, so you can't make things too weird.

You can easily have bar!{ foo => baz }, but I wouldn't expect {foo => baz}.bar!() to work.

But there still may be a value in letting the macro control order of evaluation and having ability to wrap the expression or not evaluate it. A silly example could be return.unless!(cond). If it was a let-bound value, you'd end up with let tmp = return; unless!(tmp); rather than macro-generated if !cond {return}.

2 Likes

Some expectations still apply though. When a macro is invoked I expect the macro token together with its arguments to be replaced inside the syntax tree. This means, for example:

  1. The macro can affect control flow and prevent things after it from executing - but not things before it.
  2. The macro can define new things in the same scope as itself, but not in external scopes.
1 Like

So to summarize and highlight, the "method-position macro part" of the proposal is that instead of a postfix macro expanding to some expression, it should expand to some "method call position" valid tokens.

I honestly think that postfix macros should not be able to rewrite anything outside of their brackets. (Or in other words, the head of the expression is evaluated before entering the macro-generated code.)

If you take that as a given, then using a "postfix match" to write "method-position macros" makes intuitive sense. You could also have macro_rules! macros work both in expression/statement position and "method-call position" by making those separate match arms:

macro_rules! r#try {
    ($expr:expr $(,)?) => { $expr? };
    .() => { .match x { x => $crate::r#try(x) } };
}

However, you can also achieve the same result just by having a temporary introduced as part of the macro call syntax:

macro_rules! r#try {
    ($expr:expr $(,)?) => { $expr? };
    $self.() => { $crate::r#try($self) };
}
3 Likes

That's a fair point. However, solving this only requires a slightly stronger constraint, namely that the first argument be syntactically valid Rust (ie. parseable into an AST node, not only into a token stream). It does not require full evaluation, so I think that would be taking it too far.

I can see why you are saying this — rewriting stuff outside the macro brackets can be surprising, because they are visually separate from the macro invocation, right? On the other hand, not being able to rewrite it is also surprising, because it is still an argument to the macro, after all.

It seems to me that postfix macros would create a lot of confusion, no matter what.

1 Like

it's really difficult to read for me. My brain can't understand what ".let" could mean.

I would disagree with this. I also think that taking a conservative approach here and (perhaps only initially) not allowing too much can be a great way to get postfix macros actually happening. If you want to rewrite syntax or define new variables in scope, you could always still use a prefix macro. Postfix shines in longer expressions, returning values, something like

something().modified(using(foo)).into_number().formatted!("My number: {}").send_off_to(somewhere())

I don’t see much potential for confusion in such a setting.

As far as I understand, you’re comparing a confused user of a macro (confused as in their code behaves differently than expected) with a “confused” macro implementer (“confused” as in: wondering why the compiler won’t allow them to implement something they thought could be possible). I would prefer the latter over the former.

5 Likes

My suggestion was for a postfix let because that's the simplest possible construct that does the job - it binds a value and does nothing more (maybe also it destructures, but that happens everywhere else Rust does binding). I do agree, though, that postfix match has some advantages:

  1. Pattern-matching in the middle of an expression is useful on its own.
  2. The syntax tree (and therefore semantics) it generates is identical to that of a regular match.
2 Likes