# Option's FFI safety and guarantees for ABI compatibility with nonnull optimizations

**URL:** https://internals.rust-lang.org/t/options-ffi-safety-and-guarantees-for-abi-compatibility-with-nonnull-optimizations/9784
**Category:** language design
**Created:** [April 8, 2019, 4:07pm UTC](https://internals.rust-lang.org/t/options-ffi-safety-and-guarantees-for-abi-compatibility-with-nonnull-optimizations/9784 "2019-04-08T16:07:39Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![mjbshaw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/mjbshaw/32/5103_2.png) [@mjbshaw](https://internals.rust-lang.org/u/mjbshaw)
#### Post date: [April 8, 2019, 4:07pm UTC](https://internals.rust-lang.org/t/options-ffi-safety-and-guarantees-for-abi-compatibility-with-nonnull-optimizations/9784/1 "2019-04-08T16:07:39Z")

</div>

Enums like `Option` are special, since some nonnull optimizations (e.g., `Option<&'static usize>`) make it FFI safe and ABI compatible with other types (e.g., `*const usize`). But there are some problems:

1. These nonnull optimizations have very few guarantees. For example, even though `ManuallyDrop` is `#[repr(transparent)]`, `Option<ManuallyDrop<&'static usize>>` (I know `&'static usize` doesn’t need dropping; this is just a minimal example) doesn’t have any guarantees that it has the same size as `*const usize`. I know in practice it does, but there’s a difference (at least at the semantic level) between what Rust does in practice and what it (publicly) guarantees.
2. These other types aren’t FFI-safe and may not have the ABI that was intended. Continuing the same example above, `Option<ManuallyDrop<&'static usize>>` [is not FFI-safe](https://rust.godbolt.org/z/orKKRV) and there aren’t guarantees it has the same ABI as `*const usize`.

I would like to fix this. I would like to propose:

- If `Option<T>` is FFI-safe and ABI-compatible with `U`, AND
- if `X` is a `struct` type that is `#[repr(transparent)]` and it’s underlying representation is `T`, THEN
- `Option<X>` is FFI-safe and ABI-compatible with both `Option<T>` and `U`.

Additionally, this should extend to all `Option`-like enums (not just the `Option` provided by the std/core). Here, “`Option`-like” would mean an enum that is eligible for nonnull optimizations such that `Enum<&'static usize>` is FFI-safe and has the same ABI and representation as `Option<&'static usize>` and `*const usize`.

Are there any concerns with this? Is there a better way to formalize this?

I can draft an RFC if necessary.

---

<div class="post-metadata">

### Author: ![RustyYato](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/rustyyato/32/13627_2.png) [@RustyYato](https://internals.rust-lang.org/u/RustyYato)
#### Post date: [April 8, 2019, 6:17pm UTC](https://internals.rust-lang.org/t/options-ffi-safety-and-guarantees-for-abi-compatibility-with-nonnull-optimizations/9784/2 "2019-04-08T18:17:36Z")

</div>

`Option` isn’t special in any way, any enum that looks like `Option` has the same properties as `Option`.

So we could generalize this to say, any type (like `Option<T>`) that (currently) does null-pointer optimization when `T` is a non-nullable pointer will now be guaranteed to do null-pointer optimization if `T` has the same representation as a non-nullable pointer (such as `ManuallyDrop<&_>`).

This should enable `Option<ManuallyDrop<&_>>` to be safe to pass across ffi boundries as well as many other types.

---

<div class="post-metadata">

### Author: ![mcy](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/mcy/32/6512_2.png) [@mcy](https://internals.rust-lang.org/u/mcy)
#### Post date: [April 8, 2019, 6:32pm UTC](https://internals.rust-lang.org/t/options-ffi-safety-and-guarantees-for-abi-compatibility-with-nonnull-optimizations/9784/3 "2019-04-08T18:32:27Z")

</div>

I think you want to phrase this in terms of explicit "`ManuallyDrop` is `transparent`", instead of vague “ah yes same layout”. There’s a lot of layout niche optimizations, but we probably want to be very clear on exactly which are FFI-safe.

---

<div class="post-metadata">

### Author: ![mjbshaw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/mjbshaw/32/5103_2.png) [@mjbshaw](https://internals.rust-lang.org/u/mjbshaw)
#### Post date: [April 8, 2019, 6:49pm UTC](https://internals.rust-lang.org/t/options-ffi-safety-and-guarantees-for-abi-compatibility-with-nonnull-optimizations/9784/4 "2019-04-08T18:49:15Z")

</div>

> [@RustyYato](#):
>
> `Option` isn’t special in any way, any enum that looks like `Option` has the same properties as `Option` .

Oh cool, I thought `Option` was the only one whitelisted for FFI usage, but indeed [custom enums work too](https://rust.godbolt.org/z/uSlVI5). In that case I'll revise my OP to include other types.

---

<div class="post-metadata">

### Author: ![RustyYato](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/rustyyato/32/13627_2.png) [@RustyYato](https://internals.rust-lang.org/u/RustyYato)
#### Post date: [April 8, 2019, 8:01pm UTC](https://internals.rust-lang.org/t/options-ffi-safety-and-guarantees-for-abi-compatibility-with-nonnull-optimizations/9784/5 "2019-04-08T20:01:02Z")

</div>

Yes, that is what I meant, I couldn’t think of a good qay to phrase it

---

<div class="post-metadata">

### Author: ![mjbshaw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/mjbshaw/32/5103_2.png) [@mjbshaw](https://internals.rust-lang.org/u/mjbshaw)
#### Post date: [April 8, 2019, 8:30pm UTC](https://internals.rust-lang.org/t/options-ffi-safety-and-guarantees-for-abi-compatibility-with-nonnull-optimizations/9784/6 "2019-04-08T20:30:25Z")

</div>

> [@mcy](#):
>
> I think you want to phrase this in terms of explicit " `ManuallyDrop` is `transparent` ", instead of vague “ah yes same layout”.

That's more or less what I was trying to express. I revised my second bullet point to say "if `X` is a `struct` type that is `#[repr(transparent)]` and it’s underlying representation is `T`" in an attempt to clarify that. If there's a better way to express this, or if I should revise another part, let me know.

---

<div class="post-metadata">

### Author: ![dhm](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/dhm/32/4879_2.png) [@dhm](https://internals.rust-lang.org/u/dhm)
#### Post date: [April 13, 2019, 8:23pm UTC](https://internals.rust-lang.org/t/options-ffi-safety-and-guarantees-for-abi-compatibility-with-nonnull-optimizations/9784/7 "2019-04-13T20:23:21Z")

</div>

> [@mjbshaw](#):
>
> If there’s a better way to express this

I don't know if it is better, but `#[repr(transparent)]` `struct`s (and maybe unions) are _wrappers_ around an inner type; so I would say:

> If `W` is a `#[repr(transparent)]` wrapper around `T`, then ...

* * *

I would also add that although [undefined layouts w.r.t the type parameters seem to be a desirable feature](https://github.com/rust-lang/unsafe-code-guidelines/issues/35), what the OP's proposition shows is that there is also an interest in having `#[repr(transparent)]` lead to a "structurally equal" equivalence relation `≡`:

- for all types `<T, W>`, we note `W → T` when `W` is a `#[repr(transparent)]` wrapper around `T`;

- we then note `≡` the transitive, reflexive and symmetric closure of `→`

With that in mind, it would be great if a type's properties w.r.t its type parameters remained the same modulo structural equality, such as the type's (enum-optimized) layout, which would solve the OP's legitimate request:

- for all types `<T, U>,` and for all type constructors `F`, `T ≡ U ⇒ F<T> ≡ F<U>`

This would match most people's intuition about layout properties I think.

---

<div class="post-metadata">

### Author: ![gnzlbg](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/gnzlbg/32/3858_2.png) [@gnzlbg](https://internals.rust-lang.org/u/gnzlbg)
#### Post date: [April 16, 2019, 8:09am UTC](https://internals.rust-lang.org/t/options-ffi-safety-and-guarantees-for-abi-compatibility-with-nonnull-optimizations/9784/8 "2019-04-16T08:09:22Z")

</div>

If the only-thing you want to guarantee is that option-like `repr(X)` `enum`s have the same ABI as their non-ZST field, that could be a single RFC. Whether that should be guaranteed for `repr(Rust)`, or whether we want to allow `repr(transparent)` on these type of `enum`s, is up to the writer of the RFC to make a case for.

If this guarantee does not prevent any optimizations for `repr(Rust)` option-like enums (AFAICT it is actually an optimization), I think I’d prefer to just guarantee that for all of them. We want to guarantee that all “option-like `repr(Rust)` `union`s have the same layout and ABI than the only non-ZST `union` field” anyways, so this would go hand in hand with that.

OTOH, there is an RFC open about allowing `repr(transparent)` for option-like `union`s that’s in FCP. If that RFC is merged, then it would feel inconsistent to not be able to use `repr(transparent)` on option-like `enum`s as well for the exact same reasons given in that RFC (better error messages mostly).

---

<div class="post-metadata">

### Author: ![system](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/system/32/14092_2.png) [@system](https://internals.rust-lang.org/u/system)
#### Post date: [July 15, 2019, 8:09am UTC](https://internals.rust-lang.org/t/options-ffi-safety-and-guarantees-for-abi-compatibility-with-nonnull-optimizations/9784/9 "2019-07-15T08:09:25Z")

</div>

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.
