[Pre-RFC] Allow `?` in patterns

[stebalien]

So, a common idiom when iterating over anything IO related is:

for item in my_io_iterator {
    let item = item?;
    // Something
}

If rust allowed ? in patterns, one could write:

for item? in my_io_iterator {
    // Something
}

And even:

match thing {
    Some(item?) => ...,
    None => ...,
}

Thoughts? Is this too magical ? One concern I have is that this would make the following legal:

let x? = can_fail();

[withoutboats]

I do feel like this is pretty magical, but I also strongly feel the motivation. I’m not sure there’s any other solution to the for item? problem aside from TCP preserving closures (so you could .map(|x| x?) or something), which is a much more complex feature to add.

Basically I’d like an alternative solution, but if there isn’t one I think we should do this.

[xen0n]

The improvement of ergonomics in the suggested use case is real; however I think introducing multiple ways to do the same thing is bad in itself. It is sad that the proposal inevitably allows let x? = f(); along with let x = f()?;, which makes spotting the error processing part much more difficult at a glance.

[glaebhoerl]

This is a cool idea, but normally patterns deconstruct – it’s weird to have a feature which does the same thing in a pattern and an expression, rather than the inverse.

[thomash]

I totally agree, the feature may be a good one but using the ? syntax is extremely confusing.

[zackw]

Question, does this work?

for item in my_io_iterator.into_iter().map(|x| x?) {
    // Something
}

I suspect that it doesn’t, because the closure can’t force the parent function to return, can it? But if it could be made to work, then

for item in my_io_iterator? {
    // Something
}

could be shorthand for it.

? in a closure returning from the parent function would also enable things like

let foo = my_io_iterator.into_iter().map(|x| Something(x?)).collect::<T>();

… but I’m sure someone’s about to tell me that they need ? in a closure to not return from the parent, so I’ll stop now.

[stebalien]

@glaebhoerl

but normally patterns deconstruct

Patterns like ref construct. However, it does have a different name (ref v &). We could use try (as a contextual keyword?):

for try x in stuff {}

---

@zackw

Question, does this work?

for item in my_io_iterator.into_iter().map(|x| x?) {
    // Something
}

Not in rust. Some languages like Kotlin allow things like that; they effectively treat .map(...) as an a form of inline macro (they call them inline functions but that means something different in rust).

for item in my_io_iterator? {
    // Something
}

That’s already a valid rust equivalent to:

let my_io_iterator = my_io_iterator?
for item in my_io_iterator {
    // Something
}

? in a closure returning from the parent function would also enable things like

Well, I don’t know about others but I’d be really unhappy if rust introduced non-local returns like that.

[zackw]

That’s already a valid rust equivalent to:

let my_io_iterator = my_io_iterator?
for item in my_io_iterator {
    // Something
}```

Sure, but the type system can distinguish: yours is for when my_io_iterator is Result<T=IntoIterator<Item=TT>, E>, mine (and the OP’s) is for when it is IntoIterator<Item=Result<T, E>>.

Well, I don’t know about others but I’d be really unhappy if rust introduced non-local returns like that.

It seems to me that both returning and not-returning from the parent would be useful…

[sgrif]

FWIW, TCP preserving closures would solve a much larger and more general problem. (And one that I run to all the time. e.g. https://twitter.com/sgrif/status/830857702691708928)

[stebalien]

Sure, but the type system can distinguish: yours is for when my_io_iterator is Result<T=IntoIterator<Item=TT>, E>, mine (and the OP’s) is for when it is IntoIterator<Item=Result<T, E>>.

It can’t (and I am OP). Result is just a type and there’s no reason the standard library couldn’t implement IntoIterator<Item=Result<T, E>> for Result<T: IntoIterator<Item=T, E>>. Regardless, that would be confusing as hell for programmers.

[stebalien]

FWIW, TCP preserving closures would solve a much larger and more general problem. (And one that I run to all the time. e.g. https://twitter.com/sgrif/status/830857702691708928

They may be useful in some cases but really aren’t worth the trouble in Rust. As a matter of fact, they almost certainly wouldn’t help here because iterators are composable.

Currently, the only way to do a non-local return is to panic!() and catch_unwind. However, catch_unwind is severely restricted: it’s not possible to mutably borrow (or pass a reference to a RefCell, Cell, or anything with interior mutability for that matter) across a catch_unwind barrier. This restriction is necessary to avoid exposing types with violated invariants (e.g. by violating some invariant, calling something that could panic, fixing the invariant).

To make TCP preserving closures useful, you’d have to drop these constraints. Then, to make them reasonably safe, you’d have to label functions that can perform this type of unrestricted non-local return in the type system (so you can avoid calling them while violating invariants). That’s where you’d get into trouble with iterators.

Basically, my_iterator.map(f) doesn’t actually do anything; it just returns an iterator adapter (Map) that applies f to items produced by my_iterator whenever Map::next is called. So, to make this work, you’d either have to:

1. Label Iterator::next as potentially performing a non-local return. As this would affect all iterators so it’s obviously a non-starter.
2. Somehow parametrize the Iterator trait over some kind of non-local return property (something akin to a lifetime). Now you’d effectively have two lifetime systems.

So, you could make this work for the very simple case where you take a closure and call it immediately but, in that case, I’d just use a macro.

---

Note: In your case, you might want to consider proposing RefMut::try_map.

[yigal100]

Why is this a common idiom for iterators, in particular I/O ones? This reads very unnatural to me.

The usage here indicates that if any of the elements of the iterator are erroneous than the entire thing is erroneous. I’d expect in this case to indeed have:

for item in my_io_iterator? {
    // handle item
}

and the iterator should be of type Result<impl Iterator<Item=T>, E>

[withoutboats]

Such a transformation cannot be performed without eagerly evaluating the iterator and collecting the items somehow.

[yigal100]

@withoutboats True, however the exact solution to this depends on the specific details of the problem at hand.

The user may want to do any of:

1. collect the items if the amount should be small and predetermined.
2. use async streams (when we get them ) to yield elements and let the stream wait until next available value instead of returning an error immediately if there aren’t enough elements ready.
3. use blocking i/o but implement re-try functionality of some sort.
4. anything else?

• point 2 refers to the RFC on stackless-coroutines and being able to do: for item in stream

[zackw]

TCP preserving closures

TCP? What does networking have to do with this?

I managed to find an older thread on this forum that reveals you mean “Tennent’s Correspondence Principle,” but that is a really unfortunate acronym collision.

(I’m going to bow out of the discussion as it has gone well into territory that I haven’t had enough experience to speak to.)

[ahmedcharles]

My biggest issue with this so far is that pattern matching means ‘create bindings’, which in the ideal case, never requires executing any code. Obviously, code is required to determine which branch is taken in a match statement, etc.

Having for item? in items {} work, the compiler would have to execute code in addition to creating a binding.

[stebalien]

That’s a very good point. Worse, once we get some form of stable Carrier trait, it will run arbitrary user code.

[withoutboats]

I’m not convinced that’s a big problem. for item in items { } already runs arbitrary user code in the IntoIterator and Iterator implementations.

[stebalien]

Maybe not a big issue but still a little weird. The following would run user-specified code before even entering the function proper:

fn maybe_add<A, B, E>(a?: Result<A, E>, b?: Result<B, E>) ->  Result<A::Output, E>
    where A: Add<B>
{
    Ok(a + b)
}

[withoutboats]

I dislike that totally aside from the fact that its running user code. In the argument position is a good example of why ? is too magical in patterns.

Reflecting more, anywhere I’d use this I can instead just add a let x = x?; statement. The real issue with iterator/result situations is not when youre using a for loop, but when it makes combinators much worse. That suggests TCP-preserving closures are more worth it than something like this.