The `?` operator and a `Carrier` trait

[nrc]

RFC 243 proposed the ? operator as a replacement for the try! macro. That RFC was accepted and the ? operator is in nightly Rust. Part of the RFC which was not accepted was the idea of a Carrier (or ResultCarrier) trait which abstracts over Result, Option, and users’ custom result types. The language team felt that such a trait was a good idea, but that we needed to experiment with it before accepting it via the RFC process. Unfortunately we were not explicit about how that experimentation was to be done and the process for acceptance into the language.

I have recently done the implementation work in PR 33389. The intention is that the implementation will evolve through use and feedback, and that the current implementation will not be the final one. The language team has decided that we should merge this PR for the sake of experimentation, and on the understanding that it will go through a rigorous period of examination and FCP before being stabilised. Furthermore, if it proves to be contentious, then we will open an amendment RFC before stabilisation.

Please try out this feature and let us know what you think on the tracking issue. Some specific areas we are interested in:

• overall, does the feature pull its weight?
• what name should the trait should have?
• how it feels using ? with Option
• whether the trait is flexible enough (we’re pretty sure it isn’t yet, see for example this comment)
• should the trait be implemented for bool? (We think probably not, but curious if there are good use cases).
• what else is missing?

[eulerdisk]

I cannot help with testing but I have a question.

Why not a IntoResult trait in the same way IntoIterator works for for loops ? Isn’t that a cleaner solution ? [I mean, to force people use `Result` for errors]

If you already considered that option and for technical reasons is not optimal, I would be glad to know.

[nikomatsakis]

As I said in the meeting, I’m excited to see progress on this front – I would really like to be able to use ? with option values. That said, I think in the meeting we had decided to have some amount of discussion before we land the PR (and of course afterwards as well).

I think the main objection to allowing ? to operate on Option is that it blends together “error handling” (using Result type) and other kinds of “non-result” return values (which typically use Option). I’ve heard @wycats raise this argument, but @tomaka also raises a similar point in this comment:

The wording of the RFC is pretty clear about the fact that the ? operator is about propagating errors/“exceptions”. Using Option to report an error is the wrong tool. Returning None is part of the normal workflow of a successful program, while returning Err always indicates an error.

Are there other concerns as well?

[nikomatsakis]

To respond however to the mental model point. As I said, @tomaka wrote:

Returning None is part of the normal workflow of a successful program, while returning Err always indicates an error.

I am sympathetic to this argument. I think a lot of people have trouble getting a “feeling” for when it is appropriate to use Result and when to use Option. And of course while we’re discussing “error propagation”, we have to bring in panic as well.

That said, if there is one thing I have learned about error handling over the years, it’s that the distinction between an “error” and “normal control flow” is often very context dependent. I think there are plenty of cases where one could reasonably use Option or Result and either would be reasonably suitable. Moreover, if we limit ? to Result, I think what we will see is that a lot of people will use Result just so that they can take advantage of the ? syntax, which may or may not be what we want to encourage.

@nrc gave an interesting example in the lang team meeting. He pointed out rustfmt’s basic architecture is that you can invoke a format() method on a given formatting rule or strategy, which attempts to apply it to a piece of Rust code – it either returns Some, in which case the rule applies, or None. (@nrc, correct me if I am summarizing poorly here of course, I’ve never actually written anything in rustfmt myself.) If the result is None, the typical thing is to go off and try another strategy. Or something like that. Anyway, the key point is that it would be very handy to use ? sugar here for the recursive portions of the computation (and I think that rustfmt in fact uses a variant on try! for this). On the one hand, one could view “failure to apply” as an error, but it’s not “unexpected” in any sense of the word.

I also know that in practice I have frequently wanted to use ? (as well as catch, which we have not yet implemented) with Option values, though I can’t recall precise scenarios. I think just when composing complex predicates where I would otherwise use a lot of early return.

Another interesting question for me, which I think is relevant to this objection, is whether to allow mixing Option and Result (and possibly other Carrier implementers). If you consider them to be “categorically different” than it seems risky to permit an Option<T> to be quietly converted to Result<T,()> (or vice versa). This is a bit of mental dissonance for me personally, since I simultaneously see overlap between the two but also think I would prefer to avoid silent conversions between them – perhaps because I see only partial overlap (i.e., there are plenty of use cases where only one is applicable).

Anyway, I’m still turning this over in my mind.

[nikomatsakis]

I think just when composing complex predicates where I would otherwise use a lot of early return.

I think this is perhaps the heart of the objection – if we permit ? to be used with Option, we will certainly see people using it just as a convenient bit of control-flow, without any semantic overlay. So then the question is whether this is a problem, or simply a useful side-effect?

[birkenfeld]

@tomaka’s other point is also important:

If we ever want to improve some areas of errors handling (for example by adding stack traces), implementing ? on Option means that we will have to exclude ? from the changes.

Is the lang team sympathetic to the notion of traces of Result propagation? If yes, ? is the obvious candidate, but a solution has to be found that does not slow down non-error Option returns.

[nikomatsakis]

You’re right, that is important, and I had overlooked it. I would certainly like some kind of technique that allows for us to give traces, yes – not having a backtrace can be a real pain in the neck. However, I’m not sure how it can even be achieved for Result in any kind of universal fashion (that is, without an error type that “cooperates” to store the data). Perhaps I have just not been following the right threads? Can you bring me up to date (or provide me a link to the right place to read up)?

[birkenfeld]

Perhaps I have just not been following the right threads? Can you bring me up to date (or provide me a link to the right place to read up)?

Unfortunately, I’m not on top of it either, it’s just one of the things on my personal wishlist for Rust, so I’m trying to make sure it’s not forgotten

[tailhook]

Hi, I’ve just posted a plan to add some context to errors created by quick_error.

This is not exactly (or not only) about tracebacks, but it certainly interesting in the discussion. It’s also has been suggested to add that context to the try! macro (I’m not sure how it’s even possible with ? operator).

Maybe we could add it in try construct:

try with context &path {
  File::open(&path)?
}

And add a context support to the carrier trait? In this case, traceback would probably be just another kind of context in for the same Carrier trait.

[tomaka]

@nrc gave an interesting example in the lang team meeting. He pointed out rustfmt’s basic architecture is that you can invoke a format() method on a given formatting rule or strategy, which attempts to apply it to a piece of Rust code – it either returns Some, in which case the rule applies, or None. (@nrc, correct me if I am summarizing poorly here of course, I’ve never actually written anything in rustfmt myself.) If the result is None, the typical thing is to go off and try another strategy. Or something like that. Anyway, the key point is that it would be very handy to use ? sugar here for the recursive portions of the computation (and I think that rustfmt in fact uses a variant on try! for this). On the one hand, one could view “failure to apply” as an error, but it’s not “unexpected” in any sense of the word.

If I understand correctly, if format() returns Some, then the wrapping function should return that value, and if it returns None then it should continue running?

That’s funny, because the behavior I expected from ? is the opposite: if the sub-function returns None then the wrapping function should stop and return None.

I didn’t think of that, but this ambiguity is probably an issue as well. It emphasizes the fact that Some and None are equal and that there’s no reason to favour one of the other.

[nikomatsakis]

If I understand correctly, if format() returns Some, then the wrapping function should return that value, and if it returns None then it should continue running?

No. It’s more like:

fn format(&self) -> Option<String> {
    let part1 = self.part1.format()?;
    let part2 = self.part2.format()?;
    Some(format!("blah blah {} blah blah {}", part1, part2))

or something. =)

[steveklabnik]

I think the main objection to allowing ? to operate on Option is that it blends together “error handling” (using Result type) and other kinds of “non-result” return values (which typically use Option). I’ve heard @wycats raise this argument, but @tomaka also raises a similar point in this comment:

Yes, this is my objection as well.

[WiSaGaN]

I am having trouble seeing the Option early return appears frequently enough to deserve the same syntax as Result.

For a procedure that may encounter several errors along the way, returning on the first error is natural, and sometimes the only right way.

Option on the other hand does not seem to have a clear relationship with procedure (it always makes sense to talk about the result of a procedure). Even if it does, returning None on the first None does not seem to be superior to "returning Some if at least one of the Option is Some".

[matklad]

I think there are two parts here:

1. Introduce “early return capable” type (ResultCarrier)

2. Extend ? to work with ResultCarrier

So for the experiment it may make sense to do 1 and use a macro instead of 2.

[glaebhoerl]

Count me very strongly on the side of: there is no fundamental difference between Option<T> and Result<T, ()>, they can be used interchangeably based on personal taste, ergonomics, connotations, or even coin flips as far as I’m concerned, and my sense is that the reasoning I’ve seen to the contrary is based on fuzzy feelings and lacks substance.

To make me reconsider my opinion, I challenge anyone to:

• Show me a case where using Result<T, ()> instead of Option<T>, or vice versa, would have any practical negative consequence. (E.g., it would introduce a bug.)

• Show me any analogous operation that would be implemented differently for one than the other (besides, like, pretty-printing).

The one point of substance I’ve seen in favor of differentiating them is the idea that Results should carry some form of backtraces, and Options shouldn’t. That would, indeed, be a meaningful (or even fundamental) difference. I must admit I’m starting out skeptical, but that’s something I’ll at least be thinking about.

Because I’m lazy, instead of writing it again I’m going to quote a comment I made about the same subject on reddit a while ago:

My perspective is that:

• “Error handling”, as a concept, is totally subjective. Which things are considered “error handling”, and which are considered something else, is purely a matter of subjective interpretation. Essentially, it’s a matter of what names we give to things.

• It is not the job of the programming language to legislate subjective matters like which things should or shouldn’t be considered “error handling”. This is like the never-ending debate in OOP circles, due to the official position that exception handling should only be used in “truly exceptional circumstances”, over what actually counts as “truly exceptional circumstances”. It’s a meaningless and futile debate.

• What really matters is the logical structure and algebraic properties of the data and code. That is, whatever you name it, it behaves the same way and does the same thing. Essentially, my belief is that names and syntax are extremely important for intuition but aren’t and shouldn’t be significant semantically.

• Option<T> is isomorphic to Result<T, ()>.

• I consider ? and catch to be general control flow constructs which are especially useful for (things which are commonly considered to be) error handling, not dedicated error handling constructs. (Per the above, I don’t think a language should have dedicated “error handling” constructs, or that the concept even makes sense.)

Therefore I see no rightful reason for ? and catch not to work with Option, or any other type which is isomorphic to a Result.

I happily agreed to restricting it to Result at first because there’s no reason not to start small.

Also, with respect to this point:

Returning None is part of the normal workflow of a successful program, while returning Err always indicates an error.

My view is that returning None and Err are both part of the normal workflow of a successful program, and panic!() always indicates an error [in the program].

[withoutboats]

I think the main objection to allowing ? to operate on Option is that it blends together “error handling” (using Result type) and other kinds of “non-result” return values (which typically use Option). I’ve heard @wycats raise this argument, but @tomaka also raises a similar point in this comment2:

Consider this argument:

The * operator is pretty clearly about dereferencing pointers. Using it to perform type conversions between vectors and slices is the wrong tool. Type conversions should be explicit.

This is a compelling argument, but still we made a different choice because it was a big ergonomic advantage.

It will be clear from context what type the ? is operating on, because it will be in the type signature of the function or it will be destructured in the catch. I don’t think disallowing people from using ? on Option is going to make it any clearer to them whether or not they need to use Option or Result; we already have a ?-like operator that only works on Results and we have this confusion. If anything, right now, people lean too heavily toward returning Result and defining error types, I think in part probably because we’ve made Result more ergonomic to deal with than Option.

Code involving Option often has the same flow control structure as code involving Result, I think it should be able to use the same control flow operator. We don’t have concerns about distinguishing between all the methods these types share.

[tomaka]

It will be clear from context what type the ? is operating on, because it will be in the type signature of the function or it will be destructured in the catch. I don’t think disallowing people from using ? on Option is going to make it any clearer to them whether or not they need to use Option or Result; we already have a ?-like operator that only works on Results and we have this confusion. If anything, right now, people lean too heavily toward returning Result and defining error types, I think in part probably because we’ve made Result more ergonomic to deal with than Option.

I think it’s the opposite. People return too often an Option when a Result would be more appropriate. In my opinion it’s because it’s annoying to define your own error type.

[tomaka]

Maybe a better definition for Result would be: “when an Err is returned somewhere in the program, it should be somehow reported to the user or to the developer (for example by sending a message to a socket or by writing to logs or to stderr)”.

[withoutboats]

Maybe a better definition for Result would be: “when an Err is returned somewhere in the program, it should be somehow reported to the user or to the developer (for example by sending a message to a socket or by writing to logs or to stderr)”.

I don’t agree that that is necessarily true at all.

EDIT: More fundamentally, though, I don’t agree that this matters. I don’t see how limiting available control flow operations helps people decide what is an error and what isn’t in any way.

[withoutboats]

My concerns about this trait are rather different:

Does this trait have a relationship with Monad? Should T: |A| Monad<A> require that T<A>: Carrier<Success=A>, or vice versa? Should Monad be implemented for all Carriers, or Carrier implemented for all monads? It seems to me that there’s some relationship between these interfaces, and I think we should have a clear sense of what that relationship is so that if higher-kinded traits are ever a thing, Carrier is forward compatible with that.

Should Carrier be a #[fundamental] trait? Should Carrier not be stabilized until mutual exclusion / auto traits have been fully sorted through, so we can figure out if it needs something like this?

Finally, this seems to me to be an operator overloading trait, and I think it should be mounted under std::ops with the name of the operator like the other ones. As far as I know, currently the name of this operator is QuestionMark. We might want a better name than that (alternatively: that name might already be the best).