Rust macros facility proposal: add macros `expansion_order` attribute

Suppose we need to build an event handling system. Scattered about of the user source code there are event handler - functions with a specific signature, marked with attribute macros #[event_handler] and event dispatcher functions which are marked with #[event_dispatcher] attribute macros.

# [event_dispatcher]
fn  event1_dispatcher (src: & mut EventSource, event1: & Event1) {}

# [event_dispatcher]
fn  event2_dispatcher (src: & mut EventSource, event: & Event2) {}

# [event_dispatcher]
fn  event2_dispatcher2 (src: & mut EventSource, event: & Event2) {}

# [event_handler]
fn  handler2 (src: & mut EventSource, event: & Event1) {}

# [event_handler]
fn  handler3 (src: & mut EventSource, event: & Event2) {}

# [event_handler]
fn  handler4 (src: & mut EventSource, event: & Event2) {}

# [event_handler]
fn  handler5 (src: & mut EventSource, event: & Event2) {}

After all macros expansion, as result, the following code expected.

fn  event1_dispatcher (src: & mut EventSource, event1: & Event1) { handler2 (src, event1); }

fn  event2_dispatcher (src: & mut Event Event, event2: & Event2) {

 handler3 (src, event2);
 handler4 (src, event2);
 handler5 (src, event2);
}

fn  event2_dispatcher2 (src: & mut EventSource, event2: & Event2) {

 handler3 (src, event2);
 handler4 (src, event2);
 handler5 (src, event2);
}

fn  handler2 (src: & mut EventSource, event1: & Event1) {}

fn  handler3 (src: & mut EventSource, event2: & Event2) {}

fn  handler4 (src: & mut EventSource, event2: & Event2) {}

fn  handler5 (src: & mut EventSource, event2: & Event2) {}

The moment of the macros expansion in generally almost unpredictable, since it determined by macros textual location and it contained module insertion point.

To accomplish the project is needed to hold information about all available event handler functions before any event_dispatcher macros expansion. So it can be used, to compose their event dispatcher function body. The information is collected in #[event_handler] macros expansion process. We need somehow prevent any #[event_dispatcher] macros expansion before all #[event_handler] macros are proceeded.

This requirement make building the project in Rust macros quite difficult.

My proposal is to add to the macros an expansion_order attribute. So programmer, if necessary, can enforce some order of macros expansion.

Macros without the expansion_order attribute are expanding, as it’s now - immediately, in place.

But expansion of macros with new order attribute, are delayed, until all macros without order attribute are completely expanded.

At the very end, the lasted macros, are sorting according the expansion_order attribute and expand sequentially.

Inside group of macros with same expansion_order the order of expansion may vary. Only different attribute value can guaranty expansion order.

Correct me someone if I’m wrong, but this feels very much against the idea how the macros should work.

In particular, I think everyone assumes that macros (even proc macros) are independent, that one run does not influence the output of another call. It would be very surprising to the programmer if this didn’t hold. Furthermore, am I wrong in assuming that during incremental compilation rustc might decide not to re-run all of the macros, just the ones whose input has changed?

2 Likes

In my point of view, macros is a just code generator facility. In can be damn stupid like in C or advanced like in Rust… It does not matter where and how it get information to generate the code. It can get some information from source code itself, from a database, http request. etc.

If it convenient and produce correct code - it’s fine.

you want advanced CTFE with global state.

not really. I am talking about some improvement in macros realm, not user code side,

advanced CTFE lets you do things like calculate event handlers at compile time and generate a compile-time object that calls them. the optimizer can then inline it and eliminate the indirections.

For me CTFE is likely upcoming Rust static function. Not macros.

you don’t need macros when you can dynamically build objects at CTFE with interior mutability and stuff.

// macro const being hypothetical syntax for a compile-time but not (necessarily) runtime global
macro const EHANDLER_IDX_MAP: AnyMap;
struct EHandlerStuff<T> {
    callbacks: &'static CTFECell<Handler<T>>,
}
struct Handler<T> {
    cb: fn(T),
    next: Option<&'static Handler<T>>,
}
impl Handler<T> {
#[inline]
fn callback(&self, T) {
    (self.cb)(T);
    self.next.callback(T);
}
}

the compiler needs to be smart enough to realize the whole thing is const after compilation and can be inlined.

if macro const can be accessed at runtime you even get free reflection!

I wanted to explain that:

  • If it does very surprising things users don't expect (like giving different output for the same input), then it probably is not convenient, because it'll lead to bugs.
  • It'll likely not produce correct code for the incremental compilation reasons.
2 Likes

It depends on how deep granulated incremental compilation is. If a crate is the smallest unit – it is fine. Otherwise the project like Diesel, in Rust became impossible.

You force Diesel to refresh updated database scheme, and it does, but “incremental compilation” just ignore it result or do not read and apply it fully? How come?

Your code seems to involve matching event sources and sinks only by function signature (though you didn’t include definitions for Event1 and Event2). So if someone had a mousedown event and a mouseup event that happened to use the same structure they’d get confused, under this system. That’s the sort of surprise you get when you match by shape.

If you added a definition for the event itself, the code would naturally be associated with that and would benefit from CTFE. Adding a new, complicated feature to macros wouldn’t seem warranted, in this case. Do you have any other motivating examples?

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.