Up-to-date documentation on macro resolution order?

I'm wondering if there's an up-to-date write-up on how name/macro resolution actually works in practice right now? I've read through RFC 1620, RFC 2126, the 2018 relative paths proposal, and various other discussions, but there's a lot of material, and i'm not sure i'm getting everything. Plus, a lot of what's accepted in the RFCs seems to have not actually been implemented?

For context, i'm attempting to faithfully reimplement macro expansion and name resolution (for items only) for a side project. Right now my best bet seems to be just running experiments manually against rustc... there's some weird edge cases in there, e.g.:

// SUCCEEDS
crate::g!();
mod a { #[macro_export] macro_rules! g { () => (); } }

// SUCCEEDS
fn z() { info!("x"); }
#[macro_use] extern crate log;

// FAILS
q!();
macro_rules! q { () => (); }

// FAILS
g!();
#[macro_use] mod a { macro_rules! g { () => (); } }

I'm concerned that my code may not faithfully reimplement rustc's logic. Having a normative or guide-level document to explain what the correct behavior is would be really helpful.

2 Likes

Iā€™d love to read that to! My understanding is that it is not really documented consistently in any single place.

3 Likes

Does the following answer your questions sufficiently? https://doc.rust-lang.org/nightly/reference/macros-by-example.html#scoping-exporting-and-importing

  • The first example uses path-based scoping.
  • The second example is also path-based, with the extern crate's macros placed into the "macro_use" prelude (currently not documented as such, but that is what it is called).
  • The third example fails because it is using textual scope, which depends on the definition order.
  • The fourth example fails because because macro_use on a module just means the macro scope does not end when the module is closed. Similar to the third, the order matters.

Unfortunately the details of expansion and name resolution is not documented, and it is quite complex. It is a project I started a long time ago but never finished. I think it will be quite difficult to faithfully reproduce everything rustc does (bugs, warts, and all), though I imagine one could get close with lots of perseverance.

3 Likes

Does the following answer your questions sufficiently? https://doc.rust-lang.org/nightly/reference/macros-by-example.html#scoping-exporting-and-importing

Ah, this is perfect, thank you! I see, that's actually surprisingly simple.

Unfortunately the details of expansion and name resolution is not documented, and it is quite complex. It is a project I started a long time ago but never finished. I think it will be quite difficult to faithfully reproduce everything rustc does (bugs, warts, and all), though I imagine one could get close with lots of perseverance.

My plan is basically best-effort -- allow names to fail to resolve, and just work with whatever does resolve. Plus, put in some elbow grease over time, as you say.

When you say "the details", can I ask what you're referring to? Like, the interactions between different namespaces, visibilities, and temporal ambiguities?

There's wording in RFC 1520 that suggests that resolution should be resilient to small changes in order of operations:

We rely on a monotonicity property in macro expansion - once an item exists in a certain place, it will always exist in that place. It will never disappear and never change. Note that for the purposes of this property, I do not consider code annotated with a macro to exist until it has been fully expanded. ... A consequence of this is that if the compiler resolves a name, then does some expansion and resolves it again, the first resolution will still be valid. ... Note that the rules in the previous section have been carefully formulated to ensure that this check is sufficient to prevent temporal ambiguities. There are many slight variations for which this check would not be enough.

...but I guess I don't know if that design or one of the "many slight variations" was actually implemented.

If you are prepared to wade through some details, there is also https://rust-lang.github.io/rustc-guide/macro-expansion.html#discussion-about-hygiene

2 Likes

Everything, really. Ideally the reference would contain a complete description of expansion and name resolution such that one could build an independent implementation. That is a huge amount of work, though. I dumped a list of high-level topics here that would be good to cover, but that is just a start. There is also some rough documentation here and here.

Although the various things you can find out there like comments, PRs, and RFCs are very useful for context, I don't put too much weight on them since they tend to be out-of-date in various degrees. The current rustc implementation is the ultimate authority, so I personally use that as my primary source. However, it is also constantly changing, which makes it extra challenging to follow.

1 Like

appreciating sharing that. although i can feel i'm not fully prepated. asking for my appetizers project. may i ask you some questions in case i'm gonna have any? thanks a lot.

Sure, but unfortunately, I'm a bit rusty (no pun intended) since that discussion.