pre-RFC: Allow return, continue, and break to affect the captured environment

The general idea is that inside a closure, return; would return from the defining function instead of the closure.

fn main() {
  // `return` would be a divergent expression and return from `main`
  let id: i32 = "321".parse().or_else(|| return);
}

This would allow Rust to satisfy the Tennent’s Correspondence Principle.

In short, Tennent’s Correspondence Principle says:

For a given expression expr, || expr should be equivalent.

You could wrap arbitrary parts function body (even the whole function body) in a closure and execute it (|| { ... })() and there should be no compile errors.

Let's look at a common pattern:

// Inside FromRequest::from_request method from Rocket
let connection = match get_connection() {
  Ok(connection) => connection,
  Err(error) => {
    return Outcome::Failure((Status::InternalServerError, error.into());
  }
}

get_connection()? doesn't help me here as I need to decorate the error a touch before it gets returned, and then return it. I could use .map_err:

// Inside FromRequest::from_request method from Rocket
let connection = match get_connection().map_err(|err| Outcome::Failure((Status::InternalServerError, error.into()))) {
  Ok(connection) => connection,
  Err(error) => { return error }
}

However I still need to actually return it. I still can't use ? because this method doesn't return a Result (or something that implements Carrier).

Consider the following:

// Inside FromRequest::from_request method from Rocket
let connection = get_connection().or_else(|err| return Outcome::Failure((Status::InternalServerError, err)));

Once Carrier stabilizes the second option could work nice enough for this use case. The key point here is flexibility though.

Thoughts?

6 Likes

I really like that idea too, and I would like to hear other people opinions on that, especially from compiler developers which will tell us is it possible or not. Because return feels obsolete in general and it would be a really good use in closures to return to outer function, and it will remove a lot of complicated code structures and other things.

I can’t see how this would actually work. Consider rayon, for example, which uses your closure spread across an entire thread pool. How would any of those control-flow statements get back to the original calling frame?

1 Like

I'm new to the TCP principle (for lack of a better short name), but it sounds like you're applying the principle only to closures and not to regular functions, which seems rather arbitrary. Is that deliberate? If we are supposed to include regular functions, doesn't the mere existence of return, break, etc statements in a language throw the TCP principle out the window?

Yes. I’m excluding function statements because the key word is expression. In languages that implement this normally there are only function expressions (no statements) and all functions are closures.

This does complicate storing/returning closures quite a bit. But lifetimes could help out a lot. You can tie the lifetime of a closure that uses return, break, or continue to refer to the function that defines the closure. You likely wouldn’t be able to use a closure with such a lifetime with rayon (I haven’t actually looked at rayon).

Likewise you couldn’t have a move closure ( move || {} ) that does this as well.

I guess if they’re simply !Sync, that will exclude rayon use, and they should probably probably be !Send as well for general threading. But even locally, the call stack prior to actually using the closure could be arbitrarily deep. A return of this sort would feel a lot like exceptions, having to unwind many layers before getting to the place that should actually return.

I don’t know if that’s feasible at all. I’m only speaking from intuition, not deep rustc knowledge, but it seems rather complicated to me.

1 Like

Well, the key thing to discuss first is it a behavior we want?

You need to consider both what and how, to some degree. There’s no use describing the perfect unicorn with no way to breed it. :slight_smile:

As currently proposed, this also seems like a clearly breaking change, so to some extent it’s moot what we think (did you have a non-breaking version of this in mind?)

While the TCP principle seems like a good thing for many “basic” non-control flow expressions, I don’t feel any desire to have it hold for all function expressions which may contain breaks, returns, panics, ?s and other very useful “early exit” mechanisms. So even if breaking changes were on the table, I don’t see how this would be a net benefit to the language.

It would be a breaking change, indeed. I assumed that if there was enough support it could eventually go in a Rust 2.x.

You could have some way of decorating a return,break,etc. to indicate the outer scope. That would give the functionality but still violate TCP. I'd be okay with this though as the functionality is what I'm after.

Something like (to mimic break labels):

return 'super;

Something like this came up in passing in this thread from last summer. To avoid breakage, it was suggested to annotate the closure to indicate that it might have control-flow effects on the outer scope:

let connection = get_connection().or_else(do |err|
    return Outcome::Failure((Status::InternalServerError, err)));

I don’t have a strong opinion on the idea in general: yeah, if we had that I would probably find uses for it, but so far (which isn’t very far) I don’t feel I need it.

2 Likes

Some kind of annotation that changes “the scope of a return statement” seems like a much better idea. The only objection I would have then is that there doesn’t seem to be much motivation for it.

We might want TCP preserving closure but if we do we definitely want them as well as non-TCP preserving closures. Not only are current closures a very useful pattern in many cases, this is the kind of breakage that seems unfathomable - it would undoubtedly split the ecosystem.

3 Likes

Implementation-wise, there are two options:

  1. Use the stack unwinding framework from zero-cost exceptions, the same that C++ uses for exceptions, and Rust uses for panics. This isn’t as zero-cost as the name implies, but once you have enabled exceptions/unwinding panics, you’ve already paid that cost. The performance tradeoff in zero-cost exceptions means that unwinding is quite slow. That is usually ok because it is indented for exceptional control flow, but a return in a closure does not look like exceptional control flow, so it would be surprising if it were very slow to use.
  2. Let the closure return something akin to a Result type which indicates how far up the stack we should unwind. This is similar to how Swift does error handling. This has a more even tradeoff between the two different ways of returning, but it imposes a larger cost on the normal return path since the exceptional case must be checked all the way up the call stack.

Either way, from the compiler implementer’s perspective, this feature is very similar to adding exceptions to the language.

Similarly from a software engineering perspective. It is notoriously difficult to write exception-safe code in C++ because for every function call (implicit or explicit) you have to consider what happens if an exception is thrown. RAII / Drop helps, but it is not a universal cure. I imagine there’s a lot of Rust code out there with data structures that would end up in an inconsistent state if it calls a closure that panics.

For example, slice::sort_by() doesn’t specify what the slice looks like after the comparator closure panics.

1 Like

Also to be clear while we may want this for various practical use cases, the fact that someone wrote in a book that languages should do this and called it a “principle” is not in itself a strong motivation. It would not be correct to say that the behavior of Rust’s closures today is “wrong.”

13 Likes

I completely agree. I'm after the functionality it enables rather than strong adherence to the "principle" which is why I'd be okay with return 'outer; or similar.

1 Like

I’d like to see something like break 'outer;. This could be very handy in some cases. This shouldn’t conflict with the possibility of adding break with values later on though.

3 Likes

Often when I have found myself craving for a feature like this, it has been because handling an exceptional result or an error inside a closure is a pain. But this feature itself doesn’t fit into Rust as it currently stands because it may need to unwind deep from the stack to be able to return, and that is an exceptional and surprising control flow to anyone that has been handed over a closure – the same as panicking which we should discourage.

Instead, I wish that the standard library had some helpers to ease the common cases when handling errors inside closures. I’ve implemented a try_map and flip methods in crate try_map for mapping cases like Option<Result<T>> to Result<Option<T>> to be able to handle errors from inside the closure as the outer layer. Stuff like that would be nice to have in std too.

I think that break 'outer; would be nice (I just some time ago felt the need for something like this, and used returning from an inline closure a stop-gap measure, and got scolded by clippy.) but it would need to be essentially a local control flow.