Interaction of `#[macro_export]` and macro definition scope

I have this fun snippet here:

fn main() {
    let x = 1;
    dbg!(fun!());
    #[macro_export]
    macro_rules! fun {
        () => { x };
    }
}

It doesn’t currently compile because macro invocation can’t find x. It feels like it should compile though: x is in-scope for both fun!() and the macro definition. It certainly should compile by the rules described in the reference, but I did not think about this example when I wrote down these rules, so the reference might be wrong here? In any case, this seems like a thing that needs clarifying.

2 Likes

Interestingly, this compiles:

fn main() {
    let x = 1;
    #[macro_export]
    macro_rules! fun {
        () => { x };
    }
    dbg!(fun!());
}

So the working rules seems to be “captures don’t work when macro callsite is above macro declsite”, which seems somewhat weird.

1 Like

Macros seem to be usable either in a sort-of “direct” manner, in which case, ordering matters, e.g.

fn main() {
    let x = 1;

    macro_rules! fun {
        () => { x };
    }
    
    dbg!(fun!());
}
fn main() {
    let x = 1;

    dbg!(fun!()); // error: cannot find macro `fun` in this scope

    macro_rules! fun {
        () => { x };
    }
}

and you can do things like shadowing:

fn main() {
    let x = 1;
    
    macro_rules! fun {
        () => { what? };
    }

    macro_rules! fun {
        () => { x };
    }

    dbg!(fun!());
}

or through the mechanisms of item visibility, like other items such as fn foo or struct Bar, which you can get via #[macro_export] as you did, or with a use statement, e.g.

fn main() {
    let x = 1;

    dbg!(fun!()); // error (like above)

    macro_rules! fun {
        () => { 42 };
    }
}

vs

fn main() {
    let x = 1;

    dbg!(fun!()); // works!

    macro_rules! fun {
        () => { 42 }; // n.b. we still aren't using `x` here at the moment
    }

    use fun;
}

where duplicate use would simply be an error

fn main() {
    let x = || 1337;

    dbg!(fun!());

    macro_rules! fun {
        () => { x() };
    }

    use fun;

    dbg!(fun!());

    macro_rules! fun {
        () => { x() + 1 };
    }

    use fun; // error[E0252]: the name `fun` is defined multiple times

    dbg!(fun!());
}

I’d assume, the behavior is explained by implementation details of the mechanism that allows macros to be used like other items in the first place; items, even if contained within block expressions, are generally independent of the block they are defined in, acting as-if they were implemented as static/global stuff on the module level.

It doesn’t surprise me that a macro that’s made to “behave like other items” like this can end up switching “too” strongly into such a “behave like an item” mode.

Do note, that the behavior here appears to be “resolve x like an item” i.e. if there's for instance some fn x, that’ll do, and there’s no compilation error after all.

fn x() -> i32 { 42 }

fn main() {
    let x = || 1337;

    dbg!(fun!()); // 42

    macro_rules! fun {
        () => { x() };
    }

    dbg!(fun!()); // 1337

    use fun;
}

It also seems like another macro definition can shadow an external - or used - item without problems:

fn x() -> i32 { 42 }

fn main() {
    let x = || 1337;

    dbg!(fun!()); // 42

    macro_rules! fun {
        () => { x() };
    }

    use fun;

    dbg!(fun!()); // 1337

    macro_rules! fun {
        () => { x() + 1 };
    }

    dbg!(fun!()); // 1338
}

I think this completes the picture above both macro_rules declarations do also shadow the macro provided via the use; and it’s also this shadowing why the ordering mattered in your original code:

fn main() {
    let x = 1;
    dbg!(fun!()); // <- resolves to `fun` available via `#[macro_export]`
    #[macro_export]
    macro_rules! fun {
        () => { x };
    }
}
fn main() {
    let x = 1;
    #[macro_export]
    macro_rules! fun { // this macro_rules shadows
        () => { x };   // the globally available macro `fun`
    }                  // just like `let x` would shadow some `fn x`
    dbg!(fun!());
}

fun’s definition – for purposes of through local variable-like scoping rules, shadows “itself”.


Is this whole exact behavior desirable? IDK; probably not really. Certainly it could be technically-breaking to change it (but it’s also somewhat realistic that nobody might actually depend on this).

That a top-level

macro_rules! fun {
    () => { x };
}

fn main() {
    let x = 1;
    
    dbg!(fun!()); // error
}

can never refer to a local variable is also clear; that’s “just hygiene”.

But arguably, the fact that the x can fall back to referring to a top-level thing seems pretty surprising. Maybe considering this to be turned into a error first can open up space for future changes like making your original code example compile, too.

Without such a restriction, I’d think that alternatives such as making

fn x() -> i32 { 42 }

fn main() {
    dbg!(fun!()); // 42

    let x = || 1337;

    dbg!(fun!()); // 42

    macro_rules! fun {
        () => { x() };
    }

    dbg!(fun!()); // 1337

    use fun;
}

instead behave like

fn x() -> i32 { 42 }

fn main() {
    dbg!(fun!()); // 42

    let x = || 1337;

    dbg!(fun!()); // 1337 // <- this changed

    macro_rules! fun {
        () => { x() };
    }

    dbg!(fun!()); // 1337

    use fun;
}

feel very much against proper hygiene, now the resolution of a name in a macro-expansion depends on whether or not a local variable was in scope which (that local variable) wasn’t mentioned in the macro-call. (But instead at the macro definition, but there, it’s always in-scope.)

The current status where the behavior changes with the question of which fun / or which way the same fun / was imported, might be less bad?

Except… now I’m noticing some more cursed cases – you can rename the macro with use, and the behavior stays the same

fn x() -> i32 { 42 }

fn main() {
    use fun2 as fun3; // a second step of `use` for good measure
    // but skippin this second step and using `fun2` below has the same behavior, too

    let x = || 1337;

    dbg!(fun3!()); // 42

    macro_rules! fun { // there's no way this now
        () => { x() }; // also *actually shadows* fun2 and fun3, is there!?
    }

    dbg!(fun3!()); // 1337 // <- still prints 1337 here, vs 42 above..

    use fun as fun2;
}

And I keep coming up with interesting test cases. E.g. if this is working with x being a macro, which is[1] something where hygiene doesn’t apply like for local variables then the behavior is “simply” depending on the new shadowing x being in scope or not:

fn main() {
    dbg!(fun!()); // 42

    macro_rules! x {
        () => { 1337 }
    }

    dbg!(fun!()); // 1337

    macro_rules! fun {
        () => { x!() };
    }
    use fun;

    dbg!(fun!()); // 1337
}

macro_rules! x {
    () => { 42 }
}
use x;

  1. i.e. name-resolution of macros ↩︎

5 Likes

Another fun case:

fn main() {
    let x = 1;
    #[macro_export]
    macro_rules! fun {
        () => { x };
    }
    dbg!(crate::fun!());
}

This works, despite explicitly opting out of textual scoping and using “the exported version”.

2 Likes

Note that “globals” take priority over captured locals:

fn main() {
    let x = 1;
    
    macro_rules! fun {
        () => { x };
    }
    
    {
        // We have to do it in a nested scope, otherwise `let` stops working.
        const x: u32 = 2;
        dbg!(fun!()); // 2
    }
}

Oh my…

Edit: well actually, I think was aware that badly named globals can cause issues; I think I had tested let x = …; in a macro-expansion before, where the intention was that x is a clean identifier used only in the macro, but some struct x or const x can cause errors such as let bindings cannot shadow tuple structs.

1 Like

Actually, no, I’m wrong, it’s just shadowing. This prints 1:

fn x() -> i32 { 2 }

fn main() {
    let x = || 1;
    
    macro_rules! fun {
        () => { x };
    }
    
    dbg!(fun!()());
}

Notably,

const tmp: u32 = 1;
dbg!(tmp);

fails, but I think this is a separate issue.

And

const tmp: u32 = 1;
dbg!(42);

fails the same way.

… yikes, and macro items (feature(decl_macro)) behave the same way.

I posit the possible behaviors are:

  • item, item, item — just wrong
  • item, item, local — current; macro expanded identifiers refer to locals textually after the macro is defined and "captures" the hygiene, items otherwise
  • item, local, local — macro expanded identifiers refer to locals when that local is in scope, items otherwise; consistent with writing that name in that location
  • local, local, local — compile error; macro expanded identifiers always refer to the local it names at the definition site, and can't use it before initialization

With the usage of #[macro_export], there's a fourth case of expanding the macro after the "captured" name is out of scope. This also currently refers to the item name.

The "item, local, local" case I could see as just a result of name lookup rules being to check for a local then for an item if there's no local in scope to match. The actual "item, item, local" behavior… it's almost certainly an implementation artifact of how hygiene isolation works and retrofitting the lexically scoped macros 1.0 to allow namespaced lookup. My guess is that the lookup information is let-scope defined alongside with the macro, essentially capturing the identifier and defining a place alias name which the macro expands to, thus it not resolving to the local before the macro in the code block.

3 Likes