A slightly more general, easier to implement alternative to the Try trait

Hello everyone!

So I was taking a look at the std::ops::Try that allows to use ? with custom types by implementing that trait, and after noticing it was not stabilized yet, I read some of the comments from the tracking issue that were raising a few drawbacks of the current trait.

Mainly, the separate from_ok and from_error methods, the naming that is very skewed towards the error handling use case, or how it is difficult to implement the trait for a type that doesn't map directly to result.

So, I'm suggesting another trait that's a bit different from Try, in the hope that it would be easier to implement for users, and would cover a slightly larger use-case.

The initial inspiration for the trait was looking for a naming that was less tangled with the idea of "trying" something. So I tried to find a naming more descriptive of what the operator is doing, and I thought that instead of relying on Result, the trait should use a different, specialized enum:

#[must_use]
pub enum ExtractOrReturn<Early, Extract> {
    /// Return early with value of type Early
    ReturnEarly(Early),
    /// Extract value of type Extract
    Extract(Extract),
}

Like std::cmp::Ordering is a specialized enum to express the result of a comparison operator, ExtractOrReturn is a specialized enum to express the result of the ? operator, where the ReturnEarly variant is used when it should return early, and the extract variant when it should produce an expression.

This enum made me realize how asymetric the ? operator really is. When "extractring" a value from expression e in a let x = e?; statement, there should be only a single type that e can be extracted to. However, since we want easy casting to the return type of the function, a expression e of type T typically needs to be convertible to various return type when returning early in a e? expression.

This consideration leads me to the following trait:

/// Trait to implement to be able to use the `q!` macro on an instance of the type
/// 
/// A type implementing this trait defines how an instance of this type
/// can be either extracted to the Extract type, or returned early as an instance
/// of the Early type.
pub trait Question<Early> {
    /// The type that should be extracted
    type Extract;

    /// A function to determine if the expression x? should either extract
    /// a value from x of type Extract, or return early a value of type Early.
    /// To do so, this function returns either ReturnEarly(early), or
    /// Extract(extract).
    fn extract_or_return(self) -> ExtractOrReturn<Early, Self::Extract>;
}

In the trait above, the type parameter Early allows to implement the trait for a variety of function return types, while the associated type Extract ensures that a unique type is extracted from the expression in the e? (for a function with a given return type, at least).

While the naming is not ideal (the std::ops::Sub trait is not named std::ops::MinusSign, so I shouldn't have a trait named std::ops::Question(Mark), I feel like implementing this trait for types is generally easier than for Try.

  • The implementation for Option<T> does not need the NoneError type:
/// Extract T from Option<T>, or return None early.
impl<T, U> Question<Option<U>> for Option<T> {
    type Extract = T;

    fn extract_or_return(self) -> ExtractOrReturn<Option<U>, T> {
        match self {
            Some(u) => ExtractOrReturn::Extract(u),
            None => ExtractOrReturn::ReturnEarly(None),
        }
    }
}
  • The implementation for Result<T, E> retains the ability to convert E to a compatible error type F:
/// Extract T from Result<T, E>, or convert its error to return
/// a Result<U, F> early.
impl<T, U, E, F> Question<Result<U, F>> for Result<T, E>
where
    F: std::convert::From<E>,
{
    type Extract = T;

    fn extract_or_return(self) -> ExtractOrReturn<Result<U, F>, T> {
        match self {
            Ok(ok) => ExtractOrReturn::Extract(ok),
            Err(err) => ExtractOrReturn::ReturnEarly(Err(err.into())),
        }
    }
}

Here is the implementation for the enum of @nikomatsakis:

enum StrandFail<T> {
    Success(T),
    NoSolution,
    QuantumExceeded,
    Cycle(Strand, Minimums),
}

impl<T> Question<StrandFail<T>> for StrandFail<T> {
    type Extract = T;
    fn extract_or_return(self) -> ExtractOrReturn<Self, T> {
        match self {
            StrandFail::Success(t) => ExtractOrReturn::Extract(t),
            other => ExtractOrReturn::ReturnEarly(other),
        }
    }
} 

No need for a helper type anymore!

  • Here is @skade's SearchResult type:
enum SearchResult<T> {
    Found(T),
    NotFound,
}

impl<T> Question<SearchResult<T>> for SearchResult<T> {
    type Extract = SearchResult<T>;
    fn extract_or_return(self) -> ExtractOrReturn<Self, Self> {
        match self {
            SearchResult::Found(_) => ExtractOrReturn::ReturnEarly(self),
            SearchResult::NotFound => ExtractOrReturn::Extract(self), // keep searching
        }
    }
}

fn fun() -> SearchResult<Socks> {
    search_drawer()?; // keeps searching or return early Found(socks)
    search_wardrobe() // Either Found(Socks) or NotFound
}

To test this idea, I have set up a small repository containing the trait, various example implementations, and a q! macro that is a stand-in for the ? operator (since the Question semantics is not implementable in terms of the Try trait).

The implementation of the q! macros is the following, and could be an alternative semantics for the ? operator:

#[macro_export]
macro_rules! q {
    ($e:expr) => {
        match $crate::Question::<_>::extract_or_return($e) {
            $crate::ExtractOrReturn::ReturnEarly(early) => return early,
            $crate::ExtractOrReturn::Extract(extract) => extract,
        }
    };
}

If you followed along this post, thank you for reading all this :heart: ! What do you think of this idea? Is there anything I'm missing, or does this Question trait constitutes an alternative to the Try trait? Do you agree that it looks easier to implement and just a bit more general? It looks like we are really late in the implementation process of the Try trait, is it too late to propose such a "big" change? Should it be worth it, should there be a RFC?

What do you think?

19 Likes

One major difference, this formalization doesn't allow you to change the fundamental type being operated on. I.e. this would no longer be feasible to support the following in general.

fn bar() -> Option<()> {
    ...
}

fn foo() -> Result<(), MyError> {
    ...
    bar()?;
    ...
}

edit: @steffahn provides a better example

To avoid breaking changes, I guess you need to make sure that this still compiles after the change:

use core::task::Poll;

struct Error;

fn foo() -> Result<(),Error> {
    let _x: Poll<u32> = Poll::Ready(Err(Error))?;
    Ok(())
}

fn bar() -> Result<(),Error> {
    let _x: Poll<Option<u32>> = Poll::Ready(Some(Err(Error)))?;
    Ok(())
}

Edit: and most notably also the other way:

fn qux() -> Poll<Result<(),Error>> {
    let _x: i32 = Err(Error)?;
    Poll::Pending
}

fn baz() -> Poll<Option<Result<(),Error>>> {
    let _x: i32 = Err(Error)?;
    Poll::Pending
}

Edit2: One possible approach to resolve this: You could keep the existing Try trait, because that one is easier to implement for Result-like traits anyways, and add a generic instance:

use std::ops::Try;

impl<T, U> Question<U> for T
where
    T: Try,
    U: Try,
    U::Error: From<T::Error>,
{
    type Extract = T::Ok;
    
    fn extract_or_return(self) -> ExtractOrReturn<U, T::Ok> {
        match self.into_result() {
            Ok(ok) => ExtractOrReturn::Extract(ok),
            Err(err) => ExtractOrReturn::ReturnEarly(U::from_error(err.into())),
        }
    }
}

This way, the extra flexibility is gained without losing anything (as far as I can tell).

4 Likes

And @RustyYato: Thank you for taking the time to look at my post!

Just to make sure I understand your comment, what my formulation doesn't capture is the ability to convert between two arbitrary Try types that have the same error type? It indeed looks like something solved by a blanket implementation. Using the Try trait to implement it would work I guess, but I wonder if there could be a trait more minimal? Some kind of MayReturnEarlyError trait (sorry for the naming!) that would just declare the error type?

:thinking: I need some time to think. Thank you again!

1 Like

Regarding being more minimal than Try. Iā€™m not quite sure why the from_ok method is needed in Try. As far as I can tell it is not really used for ? syntax. Maybe someone else can explain the reasoning behind this.

I suspect it was intended to eventually support some form of try block syntax, although this proved controversial enough that it's currently unclear if we ever will adopt such a syntax (full disclosure: I was not a fan myself).

The original question mark RFC does not explicitly state any of this (although the Future Possibilities section seems to be gesturing in this direction), but the Try trait RFC seems to imply that from_ok is required to make types like Option implement Try, which makes sense to me.

1 Like

The Try::from_ok method is required in order to implement Iterator::try_fold efficiently for some adapters (like filter).

1 Like

Another important consideration for the design of this trait is how it impacts type inference for try blocks. Functions always specify their return type, so today ? just leaves the decisions of which Try impl and which error type up to context. But in the block case, this leads to a lot of extra type annotations.

Using a type parameter rather than an associated type, like this Question trait, narrows the scope of that decision slightly, from "which types implement Try?" to "which impls of Question<_> does this type have?", but it still leaves it open. Further, it offers no help in determining how to wrap the trailing value of the block.

Ideally the Try trait (or whatever it's called in the end) would nail things down to the point that try blocks that always use ? on the same type constructor (i.e. Result the vast majority of the time) won't need any type annotations, even to determine how to wrap the trailing value.


On the subject of from_ok, @Ixrec is correct that it is used to Ok-wrap the trailing value of try blocks. However this is specified in the original RFC, in terms of Result::Ok only: https://github.com/rust-lang/rfcs/blob/26197104b7bb9a5a35db243d639aee6e46d35d75/text/0243-trait-based-exception-handling.md#catch-expressions. It is also already implemented on nightly this way.

2 Likes

On the point of inference, even changing ? to use Into instead of From for the return type looked rather ugly:

1 Like

Alright, so I just pushed a from_error branch to my repository, to support returning early from any types that opt-into this behavior and that share the same error type.

The gist of the changes is twofolds:

  1. Add a FromError<E> trait that declares a from_error(error: E) -> Self trait that types like Result or Option can implement to indicate that they can be constructed from Errors. In the process, I reinstated the NoneError type, unfortunately.
  2. Modify the Question impls for Result, Option (and Poll), so that they read:
  • For Result:
impl<T, E, F> Question<F> for Result<T, E>
where
    F: FromError<E>,
{
    type Extract = T;

    fn extract_or_return(self) -> ExtractOrReturn<F, T> {
        match self {
            Ok(ok) => ExtractOrReturn::Extract(ok),
            Err(err) => ExtractOrReturn::ReturnEarly(FromError::from_error(err)),
        }
    }
}
  • For Option:
/// Extract T from Option<T>, or return a compatible error type early.
impl<T, F: FromError<NoneError>> Question<F> for Option<T> {
    type Extract = T;

    fn extract_or_return(self) -> ExtractOrReturn<F, T> {
        match self {
            Some(u) => ExtractOrReturn::Extract(u),
            None => ExtractOrReturn::ReturnEarly(FromError::from_error(NoneError)),
        }
    }
}
  • For Poll:
// Extract Poll::Pending if the future isn't ready, or Poll::Ready(T::Extract)
// if the underlying future is ready and was extracted, or return early if the
// polled type wants to return early.
impl<T, F> Question<F> for Poll<T>
where
    T: Question<F>,
{
    type Extract = Poll<T::Extract>;
    fn extract_or_return(self) -> ExtractOrReturn<F, Self::Extract> {
        match self {
            Poll::Pending => ExtractOrReturn::Extract(Poll::Pending),
            Poll::Ready(t) => t
                .extract_or_return()
                .map_extract(|extract| Poll::Ready(extract)),
        }
    }
}

(BTW, why isn't Try implemented for Poll<T> where T: Try, instead of hardcoding Try for Poll<Option<T>> and Poll<Result<T, E>?)

With these modifications, the provided examples all compile and perform as expected (I added them as tests).

@rpjohnst : regarding inference, I wonder if we could add a method fn as_return_early(self) -> ReturnEarlyResultWrapper<Self> to Result such that ReturnEarlyResultWrapper would just implement Question<Result<T, U>> so that there would be a single possibility for inference in the try block?

@RustyYato: I couldn't find a use of the from_ok method in the Filter adapter. Could you link to some source using it so that I can understand how it is used?

In Filter::try_fold there is a call to filter_try_fold, if you follow that, you get

fn filter_try_fold<'a, T, Acc, R: Try<Ok = Acc>>(
    predicate: &'a mut impl FnMut(&T) -> bool,
    mut fold: impl FnMut(Acc, T) -> R + 'a,
) -> impl FnMut(Acc, T) -> R + 'a {
    move |acc, item| if predicate(&item) { fold(acc, item) } else { R::from_ok(acc) }
}

Which has R::from_ok from the Try trait.

2 Likes

I love the direction here. I've always thought of ? as a general propagation operator and less of an error operator.

Given the proposed types Question and ExtractOrReturn:

#[must_use]
pub enum ExtractOrReturn<Early, Extract> {
    ReturnEarly(Early),
    Extract(Extract),
}

pub trait Question<Early> {
    type Extract;

    fn extract_or_return(self) -> ExtractOrReturn<Early, Self::Extract>;
}

Can I suggest some renames in the spirit of bikeshedding?

// std::ops::MaybeUnwrap
#[must_use]
pub enum MaybeUnwrap<O, R> {
  Output(O),
  Return(R),
}

// std::ops::TryUnwrap
pub trait TryUnwrap<Return> {
  type Output;

  fn try_unwrap(self) -> MaybeUnwrap<Self::Output, Return>;
}

I suggest we think of ? in terms of a binary unwrap or return question that is asked. There are already fairly strong conventions that .unwrap() exists on types that implement Try to unwrap-or-panic instead of allowing a return.

Otherwise I fully agree with the direction here.

11 Likes

I love the direction here. I've always thought of ? as a general propagation operator and less of an error operator.

Thank you for the word of encouragement!

Regarding your suggestions of renaming, I think Unwrap is spot on! This is exactly what we do to get a T from a Result<T, _> or and Option<T>, so I think this is what we should use here too!

I'm less of a fan of TryUnwrap, though. To me, Try means that we attempt to do something, and either succeed (and get to unwrap), or fail (and need to return early). I think this way of thinking does not map very well to, for instance, a search, that may succeed early (and return the result of the search early), or continue.

I think fn unwrap_or_return is both more descriptive of what can happen and more neutral. However, I agree that naming is long, and unwieldy for the trait UnwrapOrReturn...

I also like that you reverse the order between Output and Return, I hesitated, but this looks indeed more natural.

@RustyYato: So, I tried implementing Iterator::try_fold using Question rather than Try, but at the moment, due to the lack of a means for going back to a Question type from ExtractOrReturn, I cannot return a Question type, but just ExtractOrReturn.

pub fn try_fold<I, B, F, R, E>(iterator: &mut I, init: R, mut f: F) -> ExtractOrReturn<E, B>
where
    I: Iterator,
    F: FnMut(B, I::Item) -> R,
    R: Question<E, Extract = B>,
{
    let mut r: R = init;
    loop {
        r = match r.extract_or_return() {
            ExtractOrReturn::Extract(b) => match iterator.next() {
                Some(item) => f(b, item),
                None => {
                    return ExtractOrReturn::Extract(b);
                }
            },
            ExtractOrReturn::ReturnEarly(ret) => {
                return ExtractOrReturn::ReturnEarly(ret);
            }
        }
    }
}

I feel like this might have unacceptable consequences for ergonomics. Does anyone have a better idea of what we could do here?

That doesn't fit the Iterator::try_fold signature, so it doesn't work, you must return the same type as the closure passed to try_fold. Using the default try_fold implementation would be a huge perf regression in cases like iter.chain(other).filter(pred) because the repeatedly calling next on chain is really inefficient.

Edit: talking about the default implementation for try_fold reminded me, it also uses Try::from_ok. This is an even bigger deal than Filter

    #[inline]
    #[stable(feature = "iterator_try_fold", since = "1.27.0")]
    fn try_fold<B, F, R>(&mut self, init: B, mut f: F) -> R
    where
        Self: Sized,
        F: FnMut(B, Self::Item) -> R,
        R: Try<Ok = B>,
    {
        let mut accum = init;
        while let Some(x) = self.next() {
            accum = f(accum, x)?;
        }
        Try::from_ok(accum)
    }
2 Likes

What if there were two traits? Most types would only implement TryUnwrap, but some, like Result and Option, would also implement the super-trait:

// std::ops::Try
pub trait Try<R>: TryUnwrap<R> {
    fn from_unwrapped(u: Self::Output) -> R;
}

? would now mean TryUnwrap, but try_fold, etc., would continue using Try.

Why can't this just be a part of TryUnwrap? Why do we need the extra trait?

For the same reasons mentioned above, because from_ok doesn't always make sense for early returns. This would allow things like

// some value we want to return
struct S(usize);

fn loop_until_found() -> S {
    loop {
         find_item()?;
    }
}

fn find_item() -> Result<S, ()> { unimplemented!() }

impl TryUnwrap<S> for Result<S, ()> {
   type Output = ();
   fn try_unwrap(self) -> MaybeUnwrap<Self::Output, Return> {
       match self {
            Ok(s) => MaybeUnwrap::Return(s),
            Err(()) => MaybeUnwrap::Output(()),
       }
    }
}

from_unwrapped would be the inverse of try_unwrap, and unless you are doing something extremely strange, I don't see how you wouldn't be able to implement from_unwrapped

I don't think that we need to support non-invertable unwraps with an operator. It would make it too easy to create unmaintainable code.

The code you provided fits just fine.

impl TryUnwrap<S> for Result<S, ()> {
   type Output = ();
   fn try_unwrap(self) -> MaybeUnwrap<Self::Output, Return> {
       match self {
            Ok(s) => MaybeUnwrap::Return(s),
            Err(()) => MaybeUnwrap::Output(()),
       }
    }

    fn from_unwrapped((): ()) -> Self {
        Err(())
    }
}
1 Like

I just pushed a new try_unwrap branch implementing the naming suggested by @mehcode. Feel free to take a look and get a feeling of the trait with that new naming!

I also pushed yet another branch containing @RustyYato's suggestion of a from_unwrapped function for TryUnwrap, but while doing so I encountered an obstacle I didn't expect.

The thing is that in my test, I added, for testing purposes, the following struct:

struct Bail<T> {
    val: T,
    cond: bool,
}

With the following TryUnwrap implementation:

// Continue execution if condition is true, otherwise return early the stored
// value
impl<T> TryUnwrap<T> for Bail<T> {
    type Output = ();
    fn try_unwrap(self) -> MaybeUnwrap<(), T> {
        if self.cond {
            MaybeUnwrap::Unwrap(())
        } else {
            MaybeUnwrap::Return(self.val)
        }
    }
}

The idea being to have a helper bail_with function, allowing for the following:

// Continue execution if condition is true, otherwise return val early
fn bail_with<T>(cond: bool, val: T) -> Bail<T> {
    Bail { cond, val };
}

#[test]
fn test_bail_with() {
    fn h() -> usize {
        q!(bail_with(42 == 42, 0));
        q!(bail_with(42 != 42, 1));
        2
    }
    assert_eq!(h(), 1);
}

What should from_unwrapped be for Bail<T>?

    // TryUnwrap<T> for Bail<T>
    fn from_unwrapped((): ()) -> Self {
        // !cannot get back a meaningful T from output here.!
    }

Now, in this particular instance, there is an obvious fix:

struct Bail<T>(Option<T>);

// Continue execution if condition is true, otherwise return val early
fn bail_with<T>(cond: bool, val: T) -> Bail<T> {
   Bail(if cond {
        None // continue execution
    } else {
        Some(val) // return 'val' early
    })
}

// Continue execution if condition is true, otherwise return early the stored
// value
impl<T> TryUnwrap<T> for Bail<T> {
    type Output = ();
    fn try_unwrap(self) -> MaybeUnwrap<(), T> {
        match self {
            Bail(Some(value)) => MaybeUnwrap::Return(value),
            Bail(None) => MaybeUnwrap::Unwrap(()),
        }
    }

    fn from_unwrapped((): ()) -> Self {
        Bail(None)
    }
}

Still, this gave me pause for a bit.

1 Like

That is a problem, fortunately you could change the implementation of bail in this case, but I see the general point, I can see why another trait may be a good idea. But given the workaround, is pretty simple and improves the code (by utilizing niche-filling optimizations), I still think we should keep this as one trait. If you have a different example that doesn't have a simple workaround, I would love to see it.

1 Like