Working with fallible iterators

[rkuhn]

When implementing an Iterator<Item = Result<T, E>> or when working with it (e.g. .filter_map) I find myself often using a macro like the following to make my code readable:

macro_rules! handle_err {
    ($e:expr) => {
        match $e {
            Ok(x) => x,
            Err(e) => return Some(Err(e)),
        }
    };
}

This could be avoided if ? knew that Option<Result<_, E>> can adequately be constructed from Result<Infallible, E> by wrapping the Err in a Some. Would it therefore make sense to implement FromResidual<Result<Infallible, E>> for Option<Result<T, E>>?

---

Related question: sometimes I need to update the iterator’s state before returning the error. Is the intention to let that be covered by let-else statements? As far as I can see this wouldn’t work very well because the diverging branch would need to know that the Result is now certainly an Err, which Rust’s type system does not track (to my knowledge).

[CAD97]

@rkuhn

As far as I can see this wouldn’t work very well because the diverging branch would need to know that the Result is now certainly an Err, which Rust’s type system does not track (to my knowledge).

Silly syntax suggestion product of my RustConf flight getting cancelled only after it was originally scheduled to arrive meaning that I'm now up at 5:00:

let Ok(Some(x)) = iter.next()
else match {
    Ok(None) => todo!(),
    Err(e) => todo!(),
    // this is considered exhaustive, as the let
    // branch participates in exhaustiveness
}

[rkuhn]

Ah, thanks for that tidbit — I must’ve overlooked that (and in fairness it is explicitly not part of RFC 3137 ).

[scottmcm]

@rkuhn

Would it therefore make sense to implement FromResidual<Result<Infallible, E>> for Option<Result<T, E>>?

Personally, I'm not yet convinced it that should be implemented, but you're not the only one thinking about this:

Allow ?-converting from Result<T, E> in functions returning Option<Result<T, E>>

[jjpe]

@scottmcm

I'm not yet convinced it that should be implemented

What gives you reason to hesitate? Might be worthwhile to shed some light on that.

[scottmcm]

@jjpe

What gives you reason to hesitate?

See my posts in the PR

Note that the bar is much higher for impls, since they can't be experimented with, can't be deprecated later, and have to go in insta-stable right away. This would easily pass my bar for "let's try that in nightly" if it were something that could be #[unstable], but the certainty requirement is higher for impls.

(Not that I'm on libs-api, who decide such things.)

[jjpe]

I read recently that there might be another way to get iterators to play more nicely with fallibility, by introducing a limited form of effect algebra without growing an actual effect system. That falls under the keyword generics initiative.

From the blog post:

The one most at the top of our mind is a future keyword for fallibility. There is talk about introducing try fn() {} or fn () -> throws syntax. This could make it so methods such as Iterator::filter would be able to use ? to break out of the closure and short-circuit iteration.

[rkuhn]

@scottmcm

Note that the bar is much higher for impls, since they can't be experimented with, …

Yes, I agree that the bar should be pretty high for these changes. So let’s see if we can exceed it

Firstly, none of the try proposals in the PR deliver hassle-free error propagation, in particular for fallible iterators, in the obvious base case discussed here. This means that there is indeed an upside to this proposal.

Secondly, concerns from yaahc’s latest post:

• breaking existing inference
• compiler diagnostic regressions
• locking us out of future try impls we may prefer
• allowing code that previously didn't compile to compile in unexpected ways.

Not sure how to prove this, but I have a hard time seeing a conflicts in this case since right now ? cannot be used on a Result in an -> Option context; the second point is one that may cause grief, but my knowledge of compiler internals is insufficient to judge it. I am also assuming that only stable features count here, my knowledge of nightly or upcoming features is spotty at best.

Thirdly, from my own understanding I’d want to retain the quality that ? — as terse as it is — should only do the obvious thing, no magic. Due to the From error conversion a certain amount of magic is already established (by my standards), though. Looking at possible code patterns:

fn f(v: Vec<u32>) -> Option<Result<(), Error>> {
    let x = try_something()?; // turns Err into Some(Err)
    let y = v.last()?; // propagates None
    let z = u8::try_from(y).ok()?; // turns Err into None
    Some(Ok(()))
}

To my eye these look sufficiently obvious. I didn’t include an example where some errors shall be wrapped in Some while some others turned into None because that will always need some pattern matching anyway.

Without the FromResidual impl the calculation of x above will either require a macro or some multiline decorum, where both obscure the intent in different ways. This is the reason why I’d cast my imaginary vote for merging the PR.

[scottmcm]

@rkuhn

So let’s see if we can exceed it

I would suggest posting in the PR -- I don't know that the libs-api folks deciding on it are reading here.