[Pre-RFC]: ExcDrop trait for different drop-glue during unwinding

language design
1
  • Start date: 2015-06-??
  • RFC PR: (leave this empty)
  • Rust Issue: (leave this empty)

Summary

Add a exc_drop language item that will be identified with a ExcDrop trait. When ExcDrop is implemented against a type, the ExcDrop::exc_drop trait method will be invoked when a value of that type goes out of scope due to exception-path unwinding.

Motivation

Drop is useful for resource clean-up, but some forms of resource clean-up may wish to behave differently depending on whether the resource is being released in the “normal” path execution, or in an "exceptional" path execution. For example, consider a hypothetical Fork::scoped routine for IPC in Unix, where, instead of creating a thread and blocking waiting it to exit, forks off a new process, and uses waitpid to wait for a process to exit. Fork::scoped might return a WaitPidGuard object, which will block until the sub-process exits when normally dropped. When dropped due to exception-handling, though, it may be preferable for WaitPidGuard to first issue a signal to the child process (such that the child process would terminate), to avoid blocking during exceptional-path unwinding.

The idea for ExcDrop was originally developed as a Finalize trait in this postponed linear-types RFC (in which implementing Drop around a type with a linear element would allow values of that type to cease being treated as linear, but in which exceptional-path clean-up might still be necessary for values of linear type).

Detailed design

We define an ExcDrop trait, as follows:

#[lang="exc_drop"]
trait ExcDrop {
    fn exc_drop(&mut self);
}

The drop-glue inserted for types that implement ExcDrop will be the normal drop-glue (invoking the Drop::drop function, if defined) for normal, scope-oriented clean-up. On the other hand, exception-oriented cleanup, called from an exception-path landing pad, will instead invoke the exc_drop routine. Considering the parallelism with Drop, only types for which Drop can be implemented will be able to implement ExcDrop. (In particular, ExcDrop cannot be implemented for Copy types.) There are a few corner cases to consider, but I think the basic design covers all the bases:

Raw structure that contains an ExcDrop field.

No complications.

Drop structure that contains an ExcDrop field.

The exceptional-case drop glue for the Drop type will not change. If an exception is thrown during the drop routine, then this landing pad can invoke the exceptional-case code for the ExcDrop field, before terminating the process. (TBD: perhaps there’s a way that ExcDrop could be used to clean up our “panic during panic” story?)

ExcDrop structure that contains an ExcDrop field.

Same logic as Drop structure containing an ExcDrop field.

Alternatives

I considered making exc_drop take self by move, instead of as a mutable borrow. This would allow easy re-use of the scope-based drop glue from within the exceptional-case drop glue, because it immediately moves self into the scope-based path. There are a few drawbacks to that approach:

  1. It would imply that the normal-case drop glue would always be invoked at the end of the exceptional-case drop glue (since self will be consumed by scope-based cleanup at the end of the exc_drop routine), which seems like an unnecessary constraint on the design.

  2. Parallelism with the Drop::drop interface struck me as highly beneficial in the API. For example, if we ever solve the issue with allowing moves out of self during drop, it seems that the same solution should also apply to ExcDrop. This is facilitated by making an explicit parallel to Drop's API.

  3. (Importantly to me, though perhaps not to others): This would complicate the linear-types forward-compatibility story.

Unresolved questions

It feels like ExcDrop may offer a chance for better behavior if we enter a panic-during-unwind scenario. Does it?

2

The concept certainly sounds plausible.

Why not extend the Drop trait with an fn exc_drop(&mut self) { self.drop() } method (implemented by default to call drop)?

What does adding a new trait buy us?

Edit: Another alternative would be a method to find out if we are currently unwinding. Unless the program has many panics, branch prediction should keep the performance penalty small, though it may still be measurable if a program drops many objects that distinguish both cases.

1 Like
3

Already exists: std::thread::panicking().

As for the actual proposal: Since you include only a hypothetical example in the motivation, I infer that there is no existing code that would benefit from this addition, nor anything planned that's blocked on it. That doesn't prohibit thinking about it, but maybe the design would benefit from experience with types that drop differently during unwinding? Using the function above, those can be implemented right now, to see if behaving differently during unwinding is actually useful.

1 Like
4

Thanks, @hanna-kruppe, I hadn’t even looked into the apidocs. And I certainly agree that we should first gather some experience.

I’d also like to add that since the example is about forking threads, a call to std::thread::panicking() is very unlikely to be measurable within the profile, especially when marked unlikely() (as per RFC 1131).

5

Thank you for the comments. Unfortunately, putting the behavior into Drop is unsuitable for my needs: I factored this proposal out from the postponed linear-types RFC, in which implementing Drop for a type that would otherwise be treated as linear would cause values of that type to cease being treated as linear. Where an instance of a linear-type also needs automatic resource clean-up (that is, when unwinding), using Drop would be impossible. (I'll update the text to reflect these ideas in the Alternatives section, when I get a chance.)

That said, I think this does highlight a change I'd like to make to the proposal: In the proposal, the logic is:

  • in scope-based drop, always insert Drop-based drop-glue.
  • in exception-based drop, prefer to insert ExcDrop-based drop-glue; insert Drop-based drop-glue when ExcDrop glue is unavailable.

I think a "better" approach is:

  • in scope-based drop, prefer to insert Drop-based drop-glue; insert ExcDrop-based drop-glue when Drop is unimplemented for a type.
  • in exception-based drop, prefer to insert ExcDrop-based drop-glue; insert Drop-based drop-glue when ExcDrop glue is unimplemented for a type.

...so that only one drop-type would need to be implemented covering both types of drop, while still allowing different drop-glue generation for different return paths when so desired. In other words, the difference between ExcDrop and Drop would then be which one is preferred in the different drop-glue generation cases. The big drawback, then, would be possible user confusion about which trait they should implement for their type (though my advice would be "when in doubt, implement Drop, and I'd probably have a lint warning when ExcDrop is implemented, but Drop is not). I'll update the text to reflect this point, as well.

This is fair, though I will say two things:

  1. In current Rust, as you say, you can already use the std::thread::panicking() API if you want the drop-glue to behave differently in the exceptional case. So there is no critical need for this facility right now, though it could improve performance while reducing cyclomatic complexity in some cases. I'll dig some more on this point, but if anyone else has any applications in mind already, I'm all ears.
  2. My linear-types proposal does block on this facility. This proposal was originally part of the linear types RFC, and working towards linear types is my primary motivation for this facility. I present it independently because I felt that it was easier to understand and discuss in isolation, and because it is a relatively light lift for inclusion in the language. I'm also happy to think that the design has already benefited from the discussion so far...
1 Like
6

Why didn’t you say so earlier? That clearly motivates having a separate trait for drop-on-unwind. Though I would advise on implementing Drop for T where T : ExcDrop, not vice versa.

7

Blanket impls of Drop for T where T: ExcDrop or of ExcDrop for T where T: Drop would make it impossible to implement the blanket impl’d type outside of the standard library unless some form of specialization lands.

If you just want to optionally define different behavior for the type when dropping during unwinding, which by default performs the same behavior as the Drop method, adding a method fn drop_except() to the Drop trait, with a provided method that just calls the method drop(), is probably the best solution, rather than introducing another trait.

8

Is it decided that linear types need a trait like this? Is implementing Drop really the best way to mark something as not-linear? I’m sorry if this is a stupid question, but personally I am not familiar with the linear types proposal and don’t care a lot for it. However, I do care about how many lang items there are and how orthogonal they are, so I’m uneasy about the prospect of adding such a trait without knowing whether it will pull its weight. If linear types end being done differently (or not at all), would this still be a useful trait? Useful enough, compared to just branching on thread::panicking?

1 Like
9

Oh, and another piece of information that may be helpful here:

https://github.com/Manishearth/humpty_dumpty implements one of the main use case for linear types, namely a lint (not yet finished) that aims to ensure (among other things) that objects of a given type be explicitly dropped.

10

ExcDrop is different from std::thread::panicking - the latter would also take the exceptional paths for e.g. objects created by a panic-handling destructor.

11

Sorry, I'm trying to do my best with limited free time, and did not communicate as well as I could have. I tried to make more-or-less the same point in the Motivation section (in the second paragraph), but obviously didn't make it clearly enough.

I considered this, but I think there are negative trade-offs for future compatibility with linear types. The linear-types proposal I submitted included a facility for wrapping a linear type with an affine type by implementing Drop on the wrapping type. @hanna-kruppe asked about this design point specifically, and I'll try to answer his question below, but if this is believed to be a good mechanism for putting a linear type inside an affine type (and I think it is), then simply adding a function to the Drop trait won't work.

"Is it decided that linear types need a trait like this?" No. But one of the most common objections I had seen to previous discussion about linear types in Rust is that linear types do not play well with unwinding. This facility was intended to address that concern.

"Is implementing Drop really the best way to mark something as not-linear?" In my opinion, for the linear-types mechanism described in my RFC, yes. That mechanism (originally proposed by @eddyb, though I may not have captured the spirit of his proposal perfectly) has the compiler treat a variable as linear when it contains (or may contain, in the case of enums) a linear component. In that case, the variable goes from linear to affine as the linear component is partially moved out. For an affine type, the Drop hook (or, actually, a better drop hook in which partial moves would be allowed, discussion) provides a natural point in which the linear components could be moved out, allowing the variable to be treated as affine again.

"If linear types end being done differently (or not at all), would this still be a useful trait?" I believe so. This is another reason I wanted to discuss this independently of the linear types proposal: if this facility is not considered generally useful, then it has failed one of my design objectives.

"Useful enough, compared to just branching on thread::panicking?" Yes, I think so. As @arielb1 points out, branching on panicking cannot capture all patterns that this facility allows. Returning to the motivating example, let's say that, on panic, we want to execute some external program to log details about the panic condition:

struct MyStruct {
  ...
}
impl ExcDrop for MyStruct {
  fn exc_drop(&mut self) {
    let _ = fork::scoped("/usr/bin/logger goodbye, world");
  }
}

If the hypothetical fork::scoped routine branched on panicking, it would immediately issue a signal to the child process as the ForkGuard goes out of scope, even if logger didn't finish its operation. I haven't thought this through far enough to make this claim with confidence, but this is why I raised the point that ExcDrop might be an ingredient to improve our "panic while panicking" story.

12

I meant to include this in my blanket reply earlier:

Yes, I've seen that, too. I know I'm largely to blame for this, but I was hoping to keep this thread focused on discussion about ExcDrop. Forward compatibility with linear types is a design constraint for me, but as I said in reply to @hanna-kruppe, if ExcDrop isn't considered independently "worth it", then it's failed one of my design objectives, and I'd try to find another approach.

13

@aidancully: please note that supporting this trait would increase the size of Rust’s runtime, which has been described as being of “None, for some large values of None”, thus making Rust less suitable for use on embedded systems, where usually every byte counts.

So we really want to make sure this feature pulls its weight and there is no alternative that implements that functionality in a pay-for-use way.

14

Why do you say so? I don't think that's true. If this feature were adopted, the designed outcome would be that a different drop-glue function could be called after an LLVM exception-catching landing-pad than would be called in the normal execution path, as opposed to the present case where there is only one drop-glue function that will be called in both execution paths. That's it. I don't think there are any new run-time implications.

For what it's worth, my interest in linear types is directly related to trying to make Rust more suitable for use in embedded environments. I work in embedded systems (I defined the MAC software architecture for this product, which, as far as the MAC software is concerned, is a small-memory system), and am highly interested in the opportunity to use a better language than C++ in these environments. At present, the Rust language encourages too much use of dynamic allocation to be easily used in this sort of environment. (In fact, the MISRA and JPL C coding standards prohibit the use of general-purpose dynamic allocation once the program has entered steady-state operation.) One component of my effort to reduce the necessity of dynamic allocation in Rust is to support linear types (so that we can enforce some management on resource reclamation).

15

I say so because once you introduce the trait you will need machinery to dispatch it – at least you will need the second drop-glue. Now it may well be the case that this machinery can somehow be compiled away to nothing when not used (and if this becomes an RFC, it would be a useful avenue of research), but it's certainly not without cost.

Therefore I argue that we should look around if we can have the benefits of this feature (and before all linear types and the reduction of allocation necessity that you claim) without paying the cost. Nothing more, nothing less. Once we find out that the cost is negligible, or that we cannot get around it, we have a good motivation to implement the additional drop glue.

16

There is existing machinery in place - we already call drop routines in the exceptional path. The only difference is in which drop glue would be called, which would be statically determined at compile-time. The difference is between:

    landingpad
    call drop_glue_normal

and

    landingpad
    call drop_glue_exc

in the unoptimized LLVM-IR. If the user were to use the same function body for exc_drop as she does for drop , then the bodies of the drop_glue_* routines would also be identical. This could be considered a cost, but it's one the programmer explicitly opted into by implementing ExcDrop for the type - she asked for there to be a different drop-glue routine in this execution path. I still don't see a cost.

17

I haven't read through linear type proposals enough to understand this aspect fully. But if you want types to call the Drop function unless they implement ExcDrop, with these as separate traits, it'll need to be done at the language level and not through the normal trait system. I don't know if this is worth introducing more special rules about how Drop functions into the language.

18

I’ve thought about this more, and have a new idea to achieve the same ends, while possibly addressing some of the raised objections:

Define two drop traits (tentatively called DropMain and DropExc). DropMain defines the function to be called in normal-path drop, DropExc defines the function to be called in exceptional-path drop. Define blanket impls against DropMain and DropExc for Drop-types. Pre-linear-types, have a compiler lint that will complain if only one of DropMain or DropExc are defined for a type; post-linear-types, the lint will allow DropExc to be defined without DropMain if the type is linear.

Rationale is that, if we agree that exceptional-case drop can have different requirements than scope-based drop, then the existing Drop API represents two concerns, instead of one. The underlying concerns should be separated, and in the (common) case that we don’t actually care about why the resources need to be cleaned up, then we use the blanket-impl mechanism to deliberately obscure which cleanup-path is being used.

The APIs for DropMain and DropExc probably want to take @eddyb’s Interior<Self>, instead of &mut self, or otherwise use a better self argument (allowing partial moves out), in which case the revised proposal would also block on solving the move-from-self-during-drop problem… Which is a larger-scale problem than I was hoping to address immediately, but sometimes things go like that.

19

If this is semi-naively implemented, every struct that (recursively) contains a type implementing exc_drop and drop differently will need to generate two versions of its own drop glue, an exc_drop that recurs on its members with exc_drop and ditto drop… right? This would include all trait objects, since the underlying type might implement them differently. That wastes code space and compile time.

Actually, I can’t think off the top of my head how this would be fixed without a fair degree of code waste, assuming that just using panicking() isn’t sufficient due to objects constructed from destructors. It’s worth noting that linear types don’t have this problem, since they don’t need regular drop glue - though if one were to hypothetically restrict this feature to linear types, it would be better done by just making the linear marking work some other way than separate drop methods.

Edit: I suppose you could implement a “is_called_from_drop_glue()” method that walks the stack (using the same unwind tables as panicking itself) and determines whether all the stack frames from the drop glue currently being called by the panic process up to the immediate caller’s caller are drop glue. This would probably be rather expensive at runtime, but so is panicking.

1 Like
20

Just pass it as a boolean flag to the drop glue?

i.e. implement the drop glue as

fn drop_glue(self, is_panic: bool) {
    if is_panic {
        <Self as ExcDrop>::exc_drop(&mut self);
    } else {
        <Self as Drop>::drop(&mut self);
    }

    for_each_field!(field, {
        field.drop_glue(is_panic)
    })
}
1 Like