Parser error recovery in `syn` for better IDE support with proc-macros

libs
21

I meant that in terms of diagnostics ultimately shown to the user, not proc macro developer experience. I solved that with precedence to emulate Rustc's behaviour in my crates.

I think the proc_macro2_diagnostic-API is a bit sketchy for recovering parsers too though, since you'd have to smuggle errors in the "warning" variant to do so.
I'd have to see some example code, but I think having to aggregate manually also makes it a bit more verbose than my approach of just adding , errors to nested parser calls when parsing a sequence of recovering elements. Ymmv on whether you'd like to make forwarding or local handling of side-channel errors easier, of course. I personally find the former easier since, with precedence levels, I don't need local diagnostics handling at all in my macros.

Edit: I missed that the Warning variant emits when Tryd.
Hm, not a huge fan of subtle side effects like that, to be honest, but that's personal preference.

22

"not a huge fan" is nice understatement :wink: Personally, I am violently opposed to that kind of thing.

As I put it in a hacking.md looking at try:

global state is inherently evil, hidden side-effects are inherently evil and usually rely on global state

But I did it anyway as I felt this was one of those rare cases where the ergonomic benefit vs the next best option made it viable. (My todolist has an entry to do the same again with tracing/logging depending on experience with diagnostics)

1 Like
23

This is tangential, but I just found this really interesting parser API that I thought I'd share.

The core idea is that diagnostics, parser-to-input alignment and output are all independently optional when a recovering parser returns:

// example: tuple.rs

impl<T0, T1, T2, Last> PopParsedFrom for (T0, T1, T2, Last)
where
    T0: PopParsedFrom,
    T1: PopParsedFrom,
    T2: PopParsedFrom,
    Last: PopParsedFrom,
{
    // (Enables parser combinators.)
    type Parsed = (T1::Parsed, T2::Parsed, T3::Parsed, Last::Parsed);

    fn pop_parsed_from(
        input: &mut crate::Input,
        errors: &mut crate::Errors, // not used for control flow
    ) -> Parsed<Self::Parsed> {
        // This could also be written with a `Parsed::zip`,
        // so imagine a more complicated parser here.
        let t = T0::pop_parsed_from(input, errors)?
            .zip(T1::pop_parsed_from(input, errors)?)
            .zip(T2::pop_parsed_from(input, errors)?);
        );
        Last::pop_parsed_from(input, errors).map_some_and(|last| {
            t.map(|((t0, t1), t2)| (t0, t1, t2, last))
        })
    }
}

// parsed.rs

#![try_trait_v2]

struct Parsed<T> {
    /// Determines branching.
    pub aligned: bool,
    /// Unwrapped by `?` iff `aligned`.
    pub output: Option<T>,
}

impl<T> FromResidual for Parsed<T> {
    fn from_residual(_: <Self as Try>::Residual) -> Self {
        Self { aligned: false, output: None }
    }
}

impl<T> Try for Parsed<T> {
    type Output = Option<T>;
    type Residual = Option<T>; // Might be useful for something.

    fn from_output(output: Self::Output) -> Self {
        Self { aligned: true, output }
    }

    fn branch(self) -> ControlFlow<Self::Residual, Self::Output> {
        if self.aligned {
            ControlFlow::Continue(self.output);
        } else {
            ControlFlow::Break(self.output);
        }
    }
}

(At least I think I can refer to non-associated Parsed unambiguously in the return type.)

For example, delimited groups, separated repeats and "end of input" can realign the parser.
Optionals, repeats and most punctuation (via Default) can always have Some output.

The nested tuple there is most likely the most concise way to write this without macros.
I think muncher-less macro_rules! implementations will also be possible with some chaff.

Since try_trait_v2 (#84277) stabilisation seems far off, I'm currently emulating this with

with try_trait_v2 stable
Parsed<Self::Parsed> ControlFlow<Option<Self::Parsed>, Option<Self::Parsed>>
? .map_break(|_| None)?
….map_some_and(…)[1] match … { ControlFlow::… => …, ControlFlow::… => … }

which makes it somewhat clunky.

  1. Name pending. ↩︎