Unwinding through FFI after Rust 1.33

[RalfJung]

To remove the overhead Rust panic ABI would need to match that of the FFI, but then it either matches C++ or C, and is incompatible with the other. We probably can’t match neither C++ or C because that would be a backwards incompatible change requiring tweaking destruction order or even allowing panics to skip destructors.

I am confused here. This sounds like the kind of stuff that would be needed to turn a C SJLJ into a Rust panic, or a Rust panic into a C++ exception, or so. That is not what is being discussed here.

What we are talking about here is Rust code calling C code calling Rust code, and the inner Rust code throwing a panic that the outer Rust code can handle. The C code just needs to let the panic “pass through”. From my reading of what is described above, this is okay if the C code is compiled with the right flag. This is confirmed by the similar case of having C++ (instead of Rust) on both ends of this on StackOverflow:

compiling the C code with -fexceptions will ensure the run-time remains consistent when an exception is thrown through C code

There are more caveats (the C code might leak or so), but in principle this seems to be workable. I assume other compilers have similar flags.

[gnzlbg]

The library in question never uses setjmp and is completely agnostic to the unwinding method. I do not propose to support setjmp . In fact, I’m strongly opposed to having any form of setjmp compatibility in Rust.

So if Rust does not call setjmp, how can the C library unwind into Rust ?

[kornel]

By calling a Rust function that calls panic!().

[gnzlbg]

What happens if that C code calls C++ code that calls C code that calls Rust that panic? Do we unwind across C++ calling destructors?

[kornel]

In case of libjpeg and libpng that’s out of scope — never happens.

[gnzlbg]

In case of libjpeg and libpng that’s out of scope — never happens.

Then the proposed attribute should be called #[libjpeg_unwind] and not #[unwind].

[kornel]

#[c_unwind] works for me.

[gnzlbg]

I’d be strongly against that.

[gnzlbg]

The C code just needs to let the panic “pass through”. From my reading of what is described above, this is okay if the C code is compiled with the right flag.

AFAIK this only works if we propagate Rust panics as C++ exceptions, and the C code was compiled with C++ exception support, and then catch C++ exceptions and convert them back to Rust panics because the Rust panic implementation does not need to match the C++ exception implementation.

[gThorondorsen]

This is not accurate. The library in question never uses setjmp and is completely agnostic to the unwinding method.

(emphasis mine)

The C code just needs to let the panic “pass through”. From my reading of what is described above, this is okay if the C code is compiled with the right flag.

AFAIK this only works if we propagate Rust panics as C++ exceptions, and the C code was compiled with C++ exception support, and then catch C++ exceptions and convert them back to Rust panics because the Rust panic implementation does not need to match the C++ exception implementation.

(idem)

For reference, here is a CraneLift issue hinting at such an alternative implementation for panics. Brief summary: a compiler can define the ABI of Rust functions to have two return addresses, one for the normal returns (as is usual) and one for panic!s (this one is new).

A Rust compiler implementing the above would have no trouble calling or defining extern "C" functions, except for panics out of the extern fns it defines. And as far as I know, a C library is unlikely to support being unwound through such a calling convention.

[FenrirWolf]

That cranelift issue is interesting. I’d seen it mentioned that Rust wants the freedom to tweak unwinding independently of other language implementations, so it’s nice seeing an example of how that freedom would actually be utilized.

[retep998]

AFAIK there are multiple incompatible unwinding ABIs on Windows (SJLJ and SEH). When targeting e.g. the msvc targets which use SEH a C library can still use SJLJ (SetJmpLongJmp) for unwinding since those are standard C APIs. If we assume the unwinding ABI of the target to be SEH, which is what we should do on that target, the problem that’s being discussed here would still not work because the C libraries use SJLJ instead.

You can link against a DLL compiled by any toolchain, independent of which toolchain you’re using. So even if you’re using msvc, the DLL you’re calling into may have been produced by a mingw toolchain using dwarf exception handling. That said, all native code on windows has SEH unwinding tables because Windows itself needs them, and so SEH unwinding will always unwind correctly through any native code, but whether destructors will fire is determined by the SEH handler that is used. Rust uses the same handler that VC++ does, so if unwinding was allowed across the C++ Rust boundary, C++ exceptions would trigger Rust destructors and Rust panics would trigger C++ destructors. Also setjmp and longjmp in VC++ use the SEH exception stuff, they’re not some separate unwinding mechanism.

[Lokathor]

Background: I have a program (repo), and it’s normally run by compiling the lib into a dynamic library that exports C ABI functions, and then the binary loads up said dynamic library and makes calls into it. So we have Rust code calling other Rust code through the C ABI via a dynamic library.

Question: If the dynamic lib panics back up into the binary, is it UB or not?

EDIT: After discussion with @gnzlbg on Discord it seems like the probable answer is “that’s UB don’t do that”, but anyone should feel free to add more info if they have more info.

[gnzlbg]

@Lokathor I don’t think that can ever work “as is” without setting the Rust panicking implementation in stone forever, which is something that we probably don’t want to do to allow us to improve the implementation over time.

The problem is, that even if the code at both side of the FFI is Rust, it could be Rust compiled with two different toolchain versions, using incompatible panicking implementations. (Through C FFI, we can’t really check that).

On Windows, MSVC breaks ABI compatibility on every release (similar to how Rust releases currently work), so C++ has this same problem: https://stackoverflow.com/a/5108118

If you want error-handling which is portable across multiple compilers/compiler versions/compiler settings, either use return codes or OS-provided exceptions (e.g. SEH on Windows)

If you want to make this work, you need to pick the same panicking ABI at both sides of the FFI and stick to it. This is possible in Rust today: catch all panics, and pass them using a stable ABI, and then panic in Rust at the other side.

The stable ABI to use is… up to you actually. Sure both error-codes and SEH work, but you can also use a pointer to a string (and if the pointer is null, there was no error), or whatever you want. You can also write a proc macro that does the dance for you, but questions like “which error code should it use”, “should all panics use the same error code, or different ones”, (the same applies to raising structured exceptions), are all pretty much application dependent.

EDIT: Even if we could come up with a general way to translate panics across FFI boundaries for some situations, that won’t necessarily make panicking through FFI a good idea in practice. Lots of things that can’t be checked need to align for it to work (e.g. what happens if the two Rust crates linked use the same panic implementation but a different memory allocator due to a different toolchain version / linking issue, and one crates tries to free the panic payload of the other crate, etc.).

[BatmanAoD]

pass them using a stable ABI

Can SEH be manually triggered and caught in Rust using the MSVC toolchain? On other platforms, can either GNU or LLVM exceptions be thrown?

[gnzlbg]

Can SEH be manually triggered and caught in Rust using the MSVC toolchain? On other platforms, can either GNU or LLVM exceptions be thrown?

Right now, panic is implemented by doing what the native platform does (the Rust run-time calls C++ code that throws an exception), so if the question is whether this can be done, the answer is yes, that’s what panic and catch_unwind do. OTOH, if the question is whether this can be done independently of how the Rust run-time is implemented, AFAICT the answer is no. Unwinding across FFI is UB, so one can’t call a C++ function that does this. One could throw and catch an exception inside an inline assembly block, but letting it escape (e.g. just throwing a SEH on windows) would be UB as well (right now destructors would be invoked, but if the Rust panic impl changes, that wouldn’t be the case anymore).

[Lokathor]

@gnzlbg in retrospect this makes sense, but also seems to indicate that a no_std dynamic library would potentially be in quite a bit of trouble.

Just trying to understand the possible cases here, it seems like:

• std:
  • The panic happens
  • The panic_hook is executed
  • The panic runtime is executed, which is either an unwind or an abort
• no_std:
  • The panic happens
  • The program immediately jumps to the panic_handler function and must never return.

Is this correct?

Assuming so, here’s two annoying questiond that Rust probably needs an answer to because it can happen:

• What happens if std rust code calls into a dynamic lib with no_std and a panic_handler? My guess, the lib uses its panic_handler and never returns
• What happens if std Rust calls a std dynamic lib? will the right panic_hook get used? even across compiler versions? Guess: it’s probably totally broken if that happens?

[jcranmer]

Lots of things that can’t be checked need to align for it to work (e.g. what happens if the two Rust crates linked use the same panic implementation but a different memory allocator due to a different toolchain version / linking issue, and one crates tries to free the panic payload of the other crate, etc.).

All of the EH ABIs specify that the exception record contains some field that is used to destroy the exception record. You could check the exact function pointers to figure out if the Rust exception record could have a mismatched memory allocator issue.

[gnzlbg]

@jcranmer

You could check the exact function pointers to figure out if the Rust exception record could have a mismatched memory allocator issue.

If this function is exactly a call to free, this dynamic check can work.

Otherwise, what happens if this function is statically linked into each dynamic library ? Two copies of the function will have different function pointers, yet they might be the exact same function. I’d guess we could maybe force these to always be dynamically linked, not sure.

[cuviper]

The lang team is requesting users to participate in forming the solution:

Issue: Default behavior of unwinding in FFI functions