Option's FFI safety and guarantees for ABI compatibility with nonnull optimizations

language design
1

Enums like Option are special, since some nonnull optimizations (e.g., Option<&'static usize>) make it FFI safe and ABI compatible with other types (e.g., *const usize). But there are some problems:

  1. These nonnull optimizations have very few guarantees. For example, even though ManuallyDrop is #[repr(transparent)], Option<ManuallyDrop<&'static usize>> (I know &'static usize doesn’t need dropping; this is just a minimal example) doesn’t have any guarantees that it has the same size as *const usize. I know in practice it does, but there’s a difference (at least at the semantic level) between what Rust does in practice and what it (publicly) guarantees.
  2. These other types aren’t FFI-safe and may not have the ABI that was intended. Continuing the same example above, Option<ManuallyDrop<&'static usize>> is not FFI-safe and there aren’t guarantees it has the same ABI as *const usize.

I would like to fix this. I would like to propose:

  • If Option<T> is FFI-safe and ABI-compatible with U, AND
  • if X is a struct type that is #[repr(transparent)] and it’s underlying representation is T, THEN
  • Option<X> is FFI-safe and ABI-compatible with both Option<T> and U.

Additionally, this should extend to all Option-like enums (not just the Option provided by the std/core). Here, “Option-like” would mean an enum that is eligible for nonnull optimizations such that Enum<&'static usize> is FFI-safe and has the same ABI and representation as Option<&'static usize> and *const usize.

Are there any concerns with this? Is there a better way to formalize this?

I can draft an RFC if necessary.

2

Option isn’t special in any way, any enum that looks like Option has the same properties as Option.

So we could generalize this to say, any type (like Option<T>) that (currently) does null-pointer optimization when T is a non-nullable pointer will now be guaranteed to do null-pointer optimization if T has the same representation as a non-nullable pointer (such as ManuallyDrop<&_>).

This should enable Option<ManuallyDrop<&_>> to be safe to pass across ffi boundries as well as many other types.

3

I think you want to phrase this in terms of explicit "ManuallyDrop is transparent", instead of vague “ah yes same layout”. There’s a lot of layout niche optimizations, but we probably want to be very clear on exactly which are FFI-safe.

4

Oh cool, I thought Option was the only one whitelisted for FFI usage, but indeed custom enums work too. In that case I'll revise my OP to include other types.

5

Yes, that is what I meant, I couldn’t think of a good qay to phrase it

6

That's more or less what I was trying to express. I revised my second bullet point to say "if X is a struct type that is #[repr(transparent)] and it’s underlying representation is T" in an attempt to clarify that. If there's a better way to express this, or if I should revise another part, let me know.

7

I don't know if it is better, but #[repr(transparent)] structs (and maybe unions) are wrappers around an inner type; so I would say:

If W is a #[repr(transparent)] wrapper around T, then ...

I would also add that although undefined layouts w.r.t the type parameters seem to be a desirable feature, what the OP's proposition shows is that there is also an interest in having #[repr(transparent)] lead to a "structurally equal" equivalence relation ≡:

  • for all types <T, W>, we note W → T when W is a #[repr(transparent)] wrapper around T;

  • we then note ≡ the transitive, reflexive and symmetric closure of →

With that in mind, it would be great if a type's properties w.r.t its type parameters remained the same modulo structural equality, such as the type's (enum-optimized) layout, which would solve the OP's legitimate request:

  • for all types <T, U>, and for all type constructors F, T ≡ U ⇒ F<T> ≡ F<U>

This would match most people's intuition about layout properties I think.

3 Likes
8

If the only-thing you want to guarantee is that option-like repr(X) enums have the same ABI as their non-ZST field, that could be a single RFC. Whether that should be guaranteed for repr(Rust), or whether we want to allow repr(transparent) on these type of enums, is up to the writer of the RFC to make a case for.

If this guarantee does not prevent any optimizations for repr(Rust) option-like enums (AFAICT it is actually an optimization), I think I’d prefer to just guarantee that for all of them. We want to guarantee that all “option-like repr(Rust) unions have the same layout and ABI than the only non-ZST union field” anyways, so this would go hand in hand with that.

OTOH, there is an RFC open about allowing repr(transparent) for option-like unions that’s in FCP. If that RFC is merged, then it would feel inconsistent to not be able to use repr(transparent) on option-like enums as well for the exact same reasons given in that RFC (better error messages mostly).

1 Like
9

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.