Pre-RFC: Catching Functions

[rpjohnst]

I thought nikomatsakis wanted to stop writing Ok(x) because of an initial impulse to write x on its own, based on the context- not because of length or from_ok. In that sense, ok! doesn’t solve the problem at all- it makes it worse.

For what it’s worth, I disagree that wrapping the trailing expression breaks type-based reasoning, for precisely that reason. The try block uses bare values on the inside and Results on the outside, distinctly unlike a coercion. This is probably the biggest disagreement here?

[josh]

A block that always converts from T to Result<T, E>, assuming it has some structure that implies such a type conversion (e.g. not using return) seems fine. A block that sometimes converts from T to Result<T, E> but also has other ways to exit that accept an E or a Result<T, E> breaks my mental typechecker. For instance, the block you posted has a T at the end, a throw e; in the middle, and an f()?; before that. That seems extremely problematic to me.

(Edited with clarifications.)

[eminence]

As far as ergonomics goes, I often find the need to add Ok to be kind of a “mental impedance mismatch”

I just had a thought/revelation that might explains why I feel so conflicted about this. Maybe it might help others:

I believe there is a tension between your desire (as the code writer) to omit the Ok() and my desire (as the code reader) to see explicitly the Ok().

I came to this realization when I was reading some of the proposed syntax examples and being totally confused if the return value was being Ok-wrapped or not.

[rpjohnst]

Ah, 100% agreement there. I have been assuming the block (and in the full proposal, the function) would always wrap.

edit: Ah, it’s the ? and throw you don’t like.

[aturon]

Thanks so much for your heartfelt comment, @josh!

There’s a feeling in this thread that feels like inevitability, like “of course this is going to happen, we just have to sort out a few details”. I know more than one person who feels like attempting to post in this thread would be like stepping out in front of an oncoming train, so they just don’t.

I haven’t been following the thread closely, but I wanted to respond to this point. I actually recently had a twitter conversation on this exact topic.

One of the pathologies that arises from time to time is the following killer combo:

• You care deeply about a particular thing.
• You feel powerless to influence said thing.

This can lead to demoralization, or to very strongly-stated arguments (in a desperate attempt to exert influence). I think this is basically the feeling you’re expressing right now.

A confounding factor: once someone is on a subteam, seeing even a blog post or pre-RFC can generate that sense of inevitability, because they are part of the group that will ultimately make the decision.

But there’s a flip side to all of this. As I think you know well, the Rust teams put a lot of work into listening and trying to address constraints and concerns that are brought up. In practice, it’s very rare that a blog post or pre-RFC turns into a feature without significant changes.

I think the modules work last year was a great example of both phenomenons. People felt they had to fight hard every step of the way, despite the extensive discussions and revisions. Even though in the end we landed on a proposal with wide consensus, it was exhausting on all sides to get there. To put it crassly: when you’re trying really hard to listen, it’s very painful when people are shouting! (Or even just talking a lot – getting 230 comments on your pre-RFC is a lot to cope with)

Ultimately, this comes down to trust, which is a currency that must be continuously earned by team leaders. The only way I know to earn that trust is to just keep listening and trying our hardest to address concerns. That, of course, needs to work in both directions: we should all try to acknowledge the values and concerns arising from each side. By doing so, we can hope to move past initial strawman designs (and the visceral reactions they generate) to improved designs that strike a better balance.

On this particular thread, speaking for myself and @nikomatsakis, at least, the immediate takeaways were:

• There are definitely some serious downsides here around clarity and learnability.
• It’s going to take a lot of work to achieve sufficient consensus to land an RFC on this topic.

That is, where you sense inevitability, we sense exactly the opposite.

Finally, on a more personal note, one of the reasons it’s so valuable to have you formally participate on the Lang team is that you do a great job of representing a certain class of concerns along the lines of what you’re expressing here. As a general rule, “passes the @josh test” is a significant milestone for these kinds of proposals

[withoutboats]

On the face of it, that’s not much. However, I’m aware that a search of this kind simply can’t detect things like exploratory integration and refactoring, where the ergonomic burden of switching between fallible and infallible versions exists as well. So, take with a grain of salt.

I would expect crates.io to be a very poor showing for this kind of operation:

• It is far more likely to occur within a commit than between commits, much less between releases (where it is a breaking change if its in the public API).
• It is far more likely to occur in end applications (which tend to have a lot of code “in the middle of the result monad” than libraries do). Libraries dominate crates.io.

Occurring between releases in 7-10% of libraries is, in my view, a very high showing.

But it will also happen that, for some users, they will first encounter errors. For example, I often teach people – as a starting step – to use some framework or something, and enums are just not needed yet. But they still may need to call some_str.parse::<u32>() (for example), in which case they encounter the possibility of error. And maybe then it’s nice to be able to just ‘use errors as errors’, without first diverting to enums.

I would push harder on this. In the long run, as Rust gains production adoption, most users will learn Rust “on the job.” Since we’ve seen that applications tend to have huge amounts of code that lie somewhere between the origins of errors and where they are handled, I would find it very likely that a new users’ first commit to the code base they are learning within will be in the context of error handling.

I have been assuming the block (and in the full proposal, the function) would always wrap.

That is what the proposal would have; I haven’t heard anyone voice support for accepting both T and Result<T, E>.

---

There’s a feeling in this thread that feels like inevitability, like “of course this is going to happen, we just have to sort out a few details”.

I want to add to Aaron’s excellent post that this thread has nullified my productivity since I started it. I wrote this sketch in 30 minutes, its nowhere near the end of the line, and there’s absolutely nothing inevitable about it. The reaction like this is both unwarranted and uncollegial.

I expected to receive a smattering of comments, constraints, iterations, etc, that I could review between other tasks during the week. I was faced with this deluge of feedback that had to become my primary focus. I also found it emotionally exhausting, because I have felt that every time I try to drill deeper into the nuance of the question I in turn receive very reactive, dramatic broad-strokes responses that don’t seem willing to drill down with me into the finer grain details of the trade offs involved.

What’s really unfortunate is that what this thread demonstrates to me that when an idea is at the “30 minute sketch” stage, if I suspect it will be at all controversial, I can’t post it on this forum or dealing with that thread will be what I am doing all week. I’m likely to turn to this forum less for early stage feedback like this. Instead I will develop ideas in a more private way and post them only when I can roll out elaborate and convincing narratives to go with them. This does not increase the influence of this community over the direction of my work.

[KasMA1990]

As far as ergonomics goes, I often find the need to add Ok to be kind of a “mental impedance mismatch” – I forget it, then get an obscure compiler error, then add it.

For this case, would it make sense to rely on IDEs as a backup strategy? If you can get this error as you write your code, would that change the equation for you in any way?

[HadrienG]

I want to add to Aaron’s excellent post that this thread has nullified my productivity since I started it. I wrote this sketch in 30 minutes, its nowhere near the end of the line, and there’s absolutely nothing inevitable about it. The reaction like this is both unwarranted and uncollegial.

I expected to receive a smattering of comments, constraints, iterations, etc, that I could review between other tasks during the week. I was faced with this deluge of feedback that had to become my primary focus. I also found it emotionally exhausting, because I have felt that every time I try to drill deeper into the nuance of the question I in turn receive very reactive, dramatic broad-strokes responses that don’t seem willing to drill down with me into the finer grain details of the trade offs involved.

What’s really unfortunate is that what this thread demonstrates to me that when an idea is at the “30 minute sketch” stage, if I suspect it will be at all controversial, I can’t post it on this forum or dealing with that thread will be what I am doing all week. I’m likely to turn to this forum less for early stage feedback like this. Instead I will develop ideas in a more private way and post them only when I can roll out elaborate and convincing narratives to go with them. This does not increase the influence of this community over the direction of my work.

I think this goes back to a point which was made multiple times earlier: the RFC process could use some fine-tuning in order to avoid this “deluge of heated discussion” effect, or the equally harmful “experts talk a lot, beginners feel afraid to come in” effect.

I am way too bad at dealing with people to provide a good concrete proposal of how a good RFC process change in this direction could look like. Just a simple search through i.r.l.o came up with a number of existing discussions on the topic. But I suspect that one important part of it would be doing away with the notion of a “pre-RFC”.

The term looks too much like “RFC”, and people will at least unconsciously understand it as such (i.e. posting an RFC on i.r.l.o instead of Github). The RFC-like formatting of pre-RFC posts further reinforces this impression. And the result is what you describe: idea sketches are treated like final RFCs, making the pre-RFC process essentially useless (it just splits the discussion between Discourse and Github).

Perhaps we need to go in a less formal direction, and instead of emulating the RFC process and formatting so much, go for a less formal “hey ppl, I have this sketch of an idea with three code doodles, I don’t know if this is going anywhere but please let me know what you think about it” approach.

[KasMA1990]

I want to add to Aaron’s excellent post that this thread has nullified my productivity since I started it. I wrote this sketch in 30 minutes, its nowhere near the end of the line, and there’s absolutely nothing inevitable about it. The reaction like this is both unwarranted and uncollegial.

I think @HadrienG has a really good point about the pre-RFC format. Because I think at least a part of this “inevitability” feeling comes from the fact that there here is already a pre-RFC that seems quite fleshed out. This makes it easy to fear that once the feature hits a full RFC then all the discussion has already happened, and there’s little room left to influence it.

[dan_t]

That is what the proposal would have; I haven’t heard anyone voice support for accepting both T and Result<T, E>.

Just to get it right: you can’t directly return a ‘Result<T, E>’ from a try/catch-function?

In a way this makes sense, because having auto-wrapping of ‘Ok’ and allowing ‘Result’ at the same time might result into some quite confusing code.

But having a return type ‘Result’ - with or without the syntactic sugar ’T catches E’ - and not being able to return a ‘Result’ seems also a bit odd, when at the same time ‘?’ is allowed, which might return a ’Result’.

[newpavlov]

One thing which should help with “inevitability” feeling is writing the next iteration (same as with modules proposal) of this pre-RFC with changes according to community feedback and listing of most notable alternative proposals/bikesheddings (important to slow down future discussion by preventing wheel reinvention in the comments thread). @nikomatsakis’s compilation post can be an excellent starting point. This will clearly show that Rust team (@withoutboats in this particular case) is ready to change proposals based on a constructive feedback. Plus it will restart discussion thread, so it will be easier for new people to join and keep track of it.

[josh]

Ultimately, this comes down to trust, which is a currency that must be continuously earned by team leaders. The only way I know to earn that trust is to just keep listening and trying our hardest to address concerns. That, of course, needs to work in both directions: we should all try to acknowledge the values and concerns arising from each side. By doing so, we can hope to move past initial strawman designs (and the visceral reactions they generate) to improved designs that strike a better balance.

I absolutely agree with this. And I want to add further that I do trust that feedback will get taken into account. If I in any way implied that it would not, my apologies.

One of my roles as a shepherd is to help collect and pass along feedback from others. For this particular topic, I found myself receiving feedback via private channels from people I know who didn’t feel comfortable expressing it in the thread themselves.

I think one reason for that is this: it feels fine and reasonable and expected to express feedback that restructures a proposal, or changes the syntax, or steers it in another direction towards the same goal, or says “what if we solve the problem this other way”. On the other hand, it feels awkward, and unwelcome, and difficult, and intimidating, to give feedback that amounts to “we shouldn’t do this, the problem itself is something we shouldn’t solve, this is a feature and not a bug”, and similar. Or, in short, it’s (to some extent by design) easy to say “what about this different approach”, and hard to say just “no, we shouldn’t do anything like this”.

It’s not that people don’t say that. But I think there’s a perception, perhaps not entirely incorrectly, that new people in the community can’t say that, that only a small subset of people are allowed to say that. And in the RFC process in general, I see the same thing: anyone can say “that looks good” or “what about this” or “I think this could be better expressed this way” or even “consider this new way to solve the problem”, but few people get to just say “no, we shouldn’t do this”. I’ve seen how feedback of that form goes over. Even for something that feels incredibly obvious as something we wouldn’t do (I won’t link to any RFCs here because I don’t want to single anyone out, but the lang team has seen several such RFCs), there’s a massive hesitation for anyone other than the official team to ever just flatly say “no”.

That’s something that can contribute to a feeling of inevitability. If you don’t see a member of the team itself saying “no”, then that makes something feel like “this is going to happen, the only question is what it’ll look like when done”. If you have feedback on what it looks like, then that process would feel like it works fine. If you want to prevent it from happening at all, on the other hand…

I think it’s entirely understandable for people to get strongly invested in work they’re doing. Many things simply wouldn’t happen if people didn’t. My concern is that it isn’t seen as acceptable to be just as strongly invested in an issue, but in the opposite direction, namely, wanting to make sure it does not happen. Both such points of view can arise through caring about the language and how it evolves; neither one is less legitimate.

One important thing that I think we don’t handle as well as we could in the RFC process, or the pre-RFC process, is that the question we should seek an answer to isn’t just “how should we do this?”, but “should we do this?”. I don’t think our processes are very good at dealing with the possibility that the answer is “no, we shouldn’t”, or for that matter, dealing with feedback that questions the problem statement itself. That isn’t necessarily seen as a legitimate form of feedback. We need to be very careful about such feedback, and that doesn’t mean that a “no” can stand without a very clear explanation and justification, but given such explanation and justification, it is a legitimate form of feedback.

The RFC-like formatting of pre-RFC posts further reinforces this impression.

A pre-RFC doesn’t need to look like an RFC, and the majority of those I’ve seen don’t. They can just raise an idea and ask what people think, and most of those I’ve seen take that approach. That’s one of the benefits of a pre-RFC: it gets feedback at a very early stage, as much on the problem as on the solution (if it proposes a solution, not all do), and feedback on it can result in a drastically different RFC than would originally have been proposed, or for that matter no RFC at all, or multiple separate RFCs.

responses that don’t seem willing to drill down with me into the finer grain details of the trade offs involved

I’m going to try to respond to this as carefully as I can. For what it’s worth, I feel similarly about this thread, in terms of it being draining.

I want to observe that it is legitimate to not want to drill down into finer-grained details, and to instead give feedback on the problem statement itself instead. I would go as far as to say that if someone has a fundamental objection to the core concept of the proposal, that it doesn’t make sense for them to be attempting to bikeshed the proposal details.

Deferring feedback until a proposal is more complete does not change that.

[josh]

I edited my post about T and Result<T, E> to clarify this point. Quoting my edited post:

A block that sometimes converts from T to Result<T, E> but also has other ways to exit that accept an E or a Result<T, E> breaks my mental typechecker. For instance, the block you posted has a T at the end, a throw e; in the middle, and an f()?; before that. That seems extremely problematic to me.

[rkruppe]

I wrote most of this this post before I saw @josh’s last post, which among other things formulated the “should we do this at all” question. It turns out the post I wrote addresses precisely that question. Several other, even earlier posts also hit this note — the topic isn’t new, but @josh explains well why it’s so important.

I’ve been vaguely pro-Ok-wrapping earlier in this thread and then kinda fell silent precisely because it’s such an exhausting discussion, but to talk about why we should or should not do this, this hits the nail on the head for me:

That is, I certainly understand how enums work, and I still want to not have to write Ok. I almost always forget it, and when the compiler tells me to add it, I do not feel like my code got “clearer” or “better” in any particular way.

In fact, ergonomic improvements for myself and others in my position are my entire motivation for a feature along these lines. Any effect on learnability, positive or negative, is an afterthought for me (partly because I don’t feel particularly qualified to judge them).

Like @nikomatsakis, I understand how enums work and particularly how Result is a first-class value, but that is not the only mental model I have when reading or writing Rust code. The model of an “error path” and a “happy path” is also a deeply ingrained and convenient one, so I use it often. It’s a simplification that only applies to certain kinds of code, sure, but when it works it’s easier to apply (especially when I’m busy juggling more interesting first-class values in my head). This is not unique to error handling, either:

• I can certainly break down an iterator chain into a monstrous type like Map<Filter<...slice::Iter<...>...>>, but 99% of the time that type is irrelevant and I just think of it as a data transformation pipeline.
• If I must, I can keep track of how many layers of references and smart pointers I have, but it’s a lot of bookkeeping that virtually never gives me any insight beyond “is this borrowed or owned”.
• At times, even whether I have a reference or an owned value doesn’t really interest me. I don’t want to pepper sigils into my filter closure just to make it return bool instead of &bool.
• I can bore you to death talking about affine types, and overall I’m glad that cloning is explicit, but almost every time I clone an Rc, I sigh in frustration because I’m made to go through another edit-compile cycle and the only benefit to this specific code is maybe avoiding one refcount bump (from the last use being a move instead of a clone).

Note that all of these examples also apply to reading code. Many of the same abstractions and simplifications I make when writing code, I also make when reading it. Sometimes more so, sometimes less. There might be type inference subtleties that force you to write the code in a specific way, but readers don’t have to worry about that. Or there might be a subtle performance problem that requires reading the code very carefully to identify which iterator types (and thus, specializations) are involved or how often some thing is cloned.

Now, it’s a fact that the trade offs are different for every instance of this. In the case of Ok-wrapping especially, I can easily see a reasonable person coming down on the no side after considering the pros and cons. In fact, some of the points raised in this thread make me doubt whether the problem is even solvable without huge, undesirable changes to the language — maybe any achievable sugar would be too brittle (rarely applicable) to really improve my user experience.

I don’t think one can just dismiss the problem statement out of hand. On the other hand, identifying downsides and concluding they outweigh any ergonomic win that might be had is entirely reasonable.

[pachi]

fn foo() -> Result<T, E> try { let x: T = bar()?; if baz() { fail e; } x }

As a complete newbie that’s still learning Rust, this syntax looks very clear to me.

I can clearly see the return type in the function signature and the try block works like Try (?), which I love: plain return (or and ending expression) is used for the pass case, failure is signalled with fail value and ? unwrap work as expected in the pass or failure case.

I would also expect that this try block could be used in closures, expressions…

[glaebhoerl]

I expected to receive a smattering of comments, constraints, iterations, etc, that I could review between other tasks during the week. I was faced with this deluge of feedback that had to become my primary focus. I also found it emotionally exhausting

I, at the same time, extremely sympathize, and also don’t understand what in the history of the discussions related to any aspect of error handling or the question mark operator could have caused you to form this expectation.

[josh]

The model of an “error path” and a “happy path” is also a deeply ingrained and convenient one, so I use it often.

For what it’s worth, this is exactly the level of discussion I think we need on the problem statement itself. (I’ve seen other mentions along this line in this thread, as well.)

Can we poke at the problem statement a bit more, separate from possible solutions, and develop a clear and well-explained understanding of it?

I feel like we’ve had quite a few different problem statements arise, and I feel like it’d be helpful to discuss them more explicitly so that we can explore if there are entirely different approaches that might solve the same problems, or if there are different formulations of the problem that raise other potential solutions.

Here are a few I’ve seen in the thread, stated initially without any evaluation or attempt to dissect:

1. Wanting to separate the “happy path” from the “error path”
2. Wanting to avoid typing (and having to remember to type) Ok.
3. Wanting to avoid typing Result in a return type.
4. Wanting to simplify code refactoring when adding error handling.
5. Wanting to make Rust error handling look more like throwing and catching exceptions.

Separately from that, I also want to call out a few other requirements stated as desirable in the thread:

6. Wanting to have the compiler flag certain types of code as an error.
7. Wanting to do type-based reasoning and development.
8. Wanting the type after return to match the type after ->
9. Wanting Rust’s error-handling model to provide transparency on how it works and what types are involved.
10. Wanting the mental model developed by early learners of Rust to be compatible with how Rust actually works, and naturally extended into a model that incorporates deeper understanding, with minimal upheaval along the way and no point at which the mental model is wrong or misleading.

This is almost certainly an incomplete subset. Can we capture any additional problem statements or requirements, here?

[nikomatsakis]

Can we poke at the problem statement a bit more, separate from possible solutions, and develop a clear and well-explained understanding of it?

Strong on this from me.

Can we capture any additional problem statements or requirements, here?

So, I raised the point about wanting to have all points where an error could occur marked uniformly (with ?) and @scottmcm (I believe) went further, pointing out that this also ensures that they are all coerced uniformly (via Into).

I think this is sort of an elaboration of the notion of separating the “error path”.

[vorner]

Wanting to separate the “happy path” from the “error path”

Kind of related, I think we should also make sure that the error path stays as an equal, not an afterthought. That makes Rust the robust language it is. When I read Rust code, I’m fully aware there could be errors.

What I mean, if I have a language with exceptions (let’s say python), it is possible to write code that has no notion about errors, but they still can happen and get propagated. And then I don’t know if the author decided propagating exceptions is the way to go, or if he just forget. Or I can just not notice this code might have to worry about errors when reading it. The error path feels less important in these languages, but it could be argued error handling is actually at least as important as the happy path.

And some more:

• Consistency with the current systax
• Not bloating the language too much

[scottmcm]

Wanting to make Rust error handling look more like throwing and catching exceptions.

I think this one can be drilled into deeper:

• Wanting an explicit syntax for raising errors
• Wanting fallible code to look as-similar-as-possible-but-no-closer to infallible code
• Not wanting the existence of fallible code in a method to affect how infallible code in the method is written
• Wanting a method that never fails, but is “fallible” for external reasons (like trait signatures) to have a body that looks the same as if it were infallible.

(Sorry for mostly writing the same thing three times there, but I couldn’t decide which perspective I liked best.)

It could also mean things I don’t think I’ve seen expressed, though please correct me if I missed them

• Wanting finally blocks
• Wanting an integrated construct that handles both try and catch, in the C++/Java sense
• Wanting all methods to be allowed to throw, without it being visible in their signatures
• Wanting all errors to propagate upwards without me needing to do anything

A bunch of my things from 2107:

• Not wanting to type Ok(()) any more than one wants to type ()
• Wanting to rearrange lines in a fallible method with minimal edits
• Wanting to have ? after every fallible method call (that doesn’t need custom handling)
• Never wanting to write something like Ok(foo()?)
• Wanting all returns in a method to take the same type
• Not wanting a T -> Option<Option<...<Option<T>>...>> coercion
• Not wanting this to be Result-specific
• Particularly wanting this to work great with [T]::get, Iterator::min, i32::checked_mul, etc
• Wanting to see the return type of the function written out
• Wanting main in examples to still look good when it returns Result to allow ?
• Wanting existing doctests to continue to work even if the signature of the implicit wrapper is changed to allow using ? in them
• Not wanting a syntax that repeats information from the Try impl for a type
• Wanting something that works with closures as well as fns

---

I think, for now, I’ll go back to using my old amusing-but-clearly-unacceptable syntax for this feature to avoid bikeshedding:

fn add_opt(a: Option<i32>, b: Option<i32>) -> Option<i32> ¿{ a? + b? }