Parser error recovery in `syn` for better IDE support with proc-macros

programmerjake · May 30, 2026, 8:16pm

Creating a new thread here as you requested.

programmerjake · May 30, 2026, 8:36pm

Currently when using an attribute proc-macro on some function it is basically impossible to recover from parse errors from syn in a reasonable fashion for IDE support, any syntax error causes parsing to fail entirely meaning the proc-macro's entire output is generally just compile_error! or it panics. It would be very nice to support doing the proc-macro's transformation on the non-erroneous parts of the input so you can still get reasonable code completion and type hints and other IDE niceties even if the input is incomplete or broken.

e.g. if you have a proc-macro like #[async_trait]:

#[async_trait]
pub trait MyTrait {
    async fn my_fn(&self) {
        let = self. // currently typing here
    }
}

syn can't parse that as a trait with a let that's missing a variable and the initializer is missing a method, so it makes IDE completion much worse. (this could be a relatively mild case since completion on the macro inputs ignoring the proc-macro mostly gives the same results (icr if rust-analyzer does that), other macros do more transformation so completion ignoring the proc-macro give bad results or just nothing at all (which is my experience)).

chrefr · May 31, 2026, 2:31am

FWIW, what you should do when syn fails to parse is to return the original code unchanged. This is better for both IDEs and the compiler, although it's not a common practice unfortunately.

kpreid · May 31, 2026, 3:04am

For many macros, that would mean that they produce no errors but fail to have their intended effects. That seems like it would be very confusing.

In most cases, you could output a compile_error!() and the input tokens, though.

chrefr · May 31, 2026, 4:50am

If the code fails to parse, the compiler/IDE will output an error, and it usually will be better.

If the error is macro specific, then yes you should emit both.

kpreid · May 31, 2026, 5:05am

For attribute macros and derive macros, if the macro is getting called at all, you know that rustc thinks that the item is valid syntax and will not be producing an error. if you then get a syn parse error, then you must be in a case where syn doesn’t support the syntax rustc does, and in that case you had better tell the user that that’s the problem, because rustc won’t.
For function-like macros that take expressions (or whatever existing syntax applies in the context of the macro call) as input, you can't know whether rustc will think your input is valid syntax or not, and it might be the case that if you just produce the input, there won’t be any syntax error and the user will be confused.

In both of these cases, you can’t be sure that just repeating the input will produce a syntax error. It may, as I said previously, just result in your macro mysteriously having no effect.

programmerjake · May 31, 2026, 5:08am

Some macros do enough code transformation that, if the inputs failing to parse caused the macro to just output the input verbatim, then that would render the IDE close to useless, e.g. I have a macro that translates:

    #[hdl]
    match output_integer_mode {
        OutputIntegerMode::Full64 => do_mode(64, false),
        OutputIntegerMode::DupLow32 => {
            let shifted = shifted.cast_to_static::<UInt<32>>();
            connect_any(extended, shifted | (shifted << 32));
            connect(x86_sf, shifted[31]);
        }
        OutputIntegerMode::ZeroExt32 => do_mode(32, false),
        OutputIntegerMode::SignExt32 => do_mode(32, true),
        OutputIntegerMode::ZeroExt16 => do_mode(16, false),
        OutputIntegerMode::SignExt16 => do_mode(16, true),
        OutputIntegerMode::ZeroExt8 => do_mode(8, false),
        OutputIntegerMode::SignExt8 => do_mode(8, true),
    }

to:

        {
            type __MatchTy<T> = <T as ::fayalite::ty::Type>::MatchVariant;
            let __match_expr = ::fayalite::expr::ToExpr::to_expr(&(output_integer_mode));
            ::fayalite::expr::check_match_expr(__match_expr, |__match_value, __infallible| {
                #[allow(unused_variables)]
                match __match_value {
                    __MatchTy::<OutputIntegerMode>::Full64 {} => match __infallible {},
                    __MatchTy::<OutputIntegerMode>::DupLow32 {} => match __infallible {},
                    __MatchTy::<OutputIntegerMode>::ZeroExt32 {} => match __infallible {},
                    __MatchTy::<OutputIntegerMode>::SignExt32 {} => match __infallible {},
                    __MatchTy::<OutputIntegerMode>::ZeroExt16 {} => match __infallible {},
                    __MatchTy::<OutputIntegerMode>::SignExt16 {} => match __infallible {},
                    __MatchTy::<OutputIntegerMode>::ZeroExt8 {} => match __infallible {},
                    __MatchTy::<OutputIntegerMode>::SignExt8 {} => match __infallible {},
                }
            });
            for __match_variant in ::fayalite::module::match_(__match_expr) {
                let (__match_variant, __scope) =
                    ::fayalite::ty::MatchVariantAndInactiveScope::match_activate_scope(
                        __match_variant,
                    );
                match __match_variant {
                    __MatchTy::<OutputIntegerMode>::Full64 => do_mode(64, false),
                    __MatchTy::<OutputIntegerMode>::DupLow32 => {
                        let shifted = shifted.cast_to_static::<UInt<32>>();
                        connect_any(extended, shifted | (shifted << 32));
                        connect(x86_sf, shifted[31]);
                    }
                    __MatchTy::<OutputIntegerMode>::ZeroExt32 => do_mode(32, false),
                    __MatchTy::<OutputIntegerMode>::SignExt32 => do_mode(32, true),
                    __MatchTy::<OutputIntegerMode>::ZeroExt16 => do_mode(16, false),
                    __MatchTy::<OutputIntegerMode>::SignExt16 => do_mode(16, true),
                    __MatchTy::<OutputIntegerMode>::ZeroExt8 => do_mode(8, false),
                    __MatchTy::<OutputIntegerMode>::SignExt8 => do_mode(8, true),
                }
            }
        }

chrefr · May 31, 2026, 5:09am

I'm not sure about rustc, but for rust-analyzer this is definitely not true.

Also, syntax not supported by syn is rare enough, and duplicate error, often unclear and in different place, is annoying enough, that I consider this acceptable.

programmerjake · May 31, 2026, 5:10am

This isn't an issue of rustc only calling proc-macros with valid inputs, but of rust-analyzer calling a macro with input that doesn't parse because you haven't finished writing the input code and are trying to get code completions.

chrefr · May 31, 2026, 5:11am

Sure, in that case you have to decide between the complexity of doing as much of the transformation as you can and the cost of having incomplete code not supported by the IDE.

(Also, rust-analyzer does not currently support expression attributes. But that's orthogonal).

programmerjake · May 31, 2026, 5:14am

yeah, currently that errs on the incomplete code doesn't work at all end, since I use syn to parse the input and there aren't really any viable stable alternatives.

it gets used inside a attribute macro on a function, so the attribute macro can walk the function body's AST and do transformations on attributes on expressions when it finds them:

#[hdl_module]
fn my_module() {
    #[hdl]
    match todo!() {
        T::A => todo!(),
    }
}

chrefr · May 31, 2026, 5:18am

FWIW r-a does have a "fixup" system that its goal is to "fix" syntax errors for input passed to attribute/derive proc macros. It's incomplete though.

kpreid · May 31, 2026, 5:29am

That’s fair. And in that case, I can see it being reasonable for the macro to fall back to passing through its input if it has nothing better can do, because rust-analyzer isn’t ever going to be 100% at reporting errors. But it is important that, when run by rustc, macros never just silently stop working.

So, I stand by my original suggestion: output a compile_error!() and the unmodified input. That should get you the completion benefits and also a guaranteed error.

programmerjake · May 31, 2026, 7:55am

unfortunately it's still very annoying since the completions are wrong since the proc-macro's transformation doesn't happen. hence why I think syn (or some similar library that's intended to be used by stable code -- so not rustc_parse and not rust-analyzer's parser) should support parse error recovery so you get the correct parts parsed correctly and the broken parts producing a parse error node or similar.

Noratrieb · May 31, 2026, 10:09am

What I wish for from syn is an easy way to treat bodies opaquely.

#[async_trait]
pub trait MyTrait {
    async fn my_fn(&self) {
        let = self. // currently typing here
    }
}

There should be no reason why the proc macro should struggle with this. The parts it cares about are completely valid. The body, which it doesn't really care about, is delimited in the Group. It should just forward it to the output.

Some function attribute macros like tracing::instrument manually implement something like this.

github.com/tokio-rs/tracing

tracing-attributes/src/lib.rs

d9d4c542d


      
                      vis: &self.vis,
                      sig: &self.sig,
                      brace_token: &self.brace_token,
                      block: &self.block,
                  }
              }
          }
          
          /// This parses a `TokenStream` into a `MaybeItemFn`
          /// (just like `ItemFn`, but skips parsing the body).
          impl Parse for MaybeItemFn {
              fn parse(input: ParseStream<'_>) -> syn::Result<Self> {
                  let outer_attrs = input.call(Attribute::parse_outer)?;
                  let vis: Visibility = input.parse()?;
                  let sig: Signature = input.parse()?;
                  let inner_attrs = input.call(Attribute::parse_inner)?;
                  let block;
                  let brace_token = syn::braced!(block in input);
                  let block: TokenStream = block.call(|buffer| buffer.parse())?;
                  Ok(Self {
                      outer_attrs,

programmerjake · May 31, 2026, 10:55am

sounds good as long as you can also actually parse the body like normal if you need, the proc macro I wrote above depends on that working.

Tamschi · June 2, 2026, 1:39pm

For what it's worth:

I've done quite a bit of practical work on this topic already. Demo crate here.

You'll notice that the demo has "perfect" recovery, i.e. it'll never cause cascading errors, and can report multiple errors in the macro input (and quoted Rust expressions) at once.

I think the fundamental problems (that aren't easily fixable in Syn) are:

a parsing API that doesn't distinguish errors (as diagnostics) and failure,
deep, eager validation of any parsed code and
a few legacy defaults regarding quotes that cause warnings to leak.

loess solves 1. by passing a &mut Errors aggregator into parsing routines. You can define recovering sequence and alternatives parsers declaratively.
(The example isn't quite current. The main crate will have basic combinators so that individual macro crates won't have to create them from scratch.)

For 2. loess-rust by default doesn't recurse into delimited groups. (Specifying another type parameter will take advantage of recovery and fallbacks. I plan to name generic type parameters after the type you'd specify to parse Rust eagerly. The library assumes that errors it emits will be output to stop compilation before the output proper, so parsers try to succeed with plausible stand-ins if they recover.)
This is very WIP and as-needed of course! It's not feasible for me to implement the entire Rust grammar there. There's loess-rust-opaque as stop-gap which wraps Syn parsers to capture delimiterless Rust grammar, but that too I only extend as I need it.

I made my own quote macro to deal with 3.
You can read its documentation on docs.rs. It's a bit fancier because I plan to eventually use it on a fairly complex UI component grammar, but is not a tt-muncher for the same reason. {#error { … }} respects {#located_at(…) { … }}, so it'll squiggle only the relevant parts of the input if you make use of that.
(Some rust-analyzer limitations notwithstanding. Last time I tried, it squiggled more than Rustc in some cases.)

I have a few too many irons in the fire at any given time, so to speak, but I can prioritise this if anyone wants to collaborate on filling in the holes and covering more grammar. I think the core is pretty solid now.

MusicalNinjaDad · June 2, 2026, 1:41pm

What would break if syn were to produce warnings in most cases (using the Diagnostic interface) and output potentially partial parsed code to the macro, leaving it up to the proc_macro to either error if the data it needs is not available, or work on the partial output if everything it needs is there?

Tamschi · June 2, 2026, 2:00pm

I think that's possible without much further adjustments if it produces hard errors instead of warnings, so it would have to be optional. Partial output without hard error seems too sketchy to me.

It would potentially be very noisy since Diagnostic doesn't support error priorities, as far as I can tell.

MusicalNinjaDad · June 2, 2026, 3:08pm

See proc_macro2_diagnostic - Rust for an example of how compiler warnings can be emitted and processing can then continue.

Topic		Replies	Views
How can we improve `proc_macro` without dependencies? libs	14	1744	March 17, 2024
Macros vs incremental parsing compiler	13	2551	May 7, 2019
Crowdsourcing request: What syntax changes that will affect the AST are currently in flight? language design	8	413	May 30, 2026
Let proc macros pass information to the compiler	40	6198	March 25, 2019
The future of syntax extensions and macros	62	13948	March 25, 2019

Parser error recovery in `syn` for better IDE support with proc-macros

Related topics