Pre-RFC: Transparent unions

language design
1
  • Feature Name: transparent_unions
  • Start Date: 2019-02-13
  • RFC PR:
  • Rust Issue:

Summary

Allow #[repr(transparent)] on unions that have exactly one non-zero-sized field (just like structs).

Motivation

Some union types are thin newtype-style wrappers around another type, like ManuallyDrop<T> and MaybeUninit<T>. Both of these types are intended to be used in the same places as T, but without being #[repr(transparent)] the actual compatibility between these types and T is left unspecified.

Making these types #[repr(transparent)] would be useful in certain cases. The following example is from the author's own experience, and is not meant to be the canonical/best motivation of this feature. It is just an example with which the author is most immediately familiar with.

The objrs crate defines macros that transform pure Rust code into being ABI-compatible with Objective-C, allowing the two languages to interoperate directly without any intermediary bridges. For various reasons that will be ommitted for brevity, sending a message to an object from Rust requires going through a #[inline(never)] trampoline:

#[inline(always)]
pub fn print(x: u32, y: u32, z: u32) {
    return __objrs_msg_send_print(core::mem::MaybeUninit::uninitialized(),
                                  core::mem::MaybeUninit::uninitialized(),
                                  x, y, z);
}

#[inline(never)]
#[doc(hidden)]
#[allow(non_upper_case_globals)]
extern "C" fn __objrs_msg_send_print(_: core::mem::MaybeUninit<objrs::runtime::Class>,
                                     _: core::mem::MaybeUninit<*const [u8; 6usize]>,
                                     x: u32, y: u32, z: u32) {
    extern "C" {
        #[link_name = "OBJC_CLASS_$_ClassName"]
        static CLASS: objrs::runtime::objc_class;
    }
    #[link_section = "__DATA,__objc_classrefs,regular,no_dead_strip"]
    #[export_name = "\x01L_OBJC_CLASSLIST_REFERENCES_$_.ClassName"]
    static REF: objrs::runtime::Class = objrs::runtime::Class(unsafe {
        &CLASS as *const _
    });
    let this = unsafe { core::ptr::read_volatile(&REF as *const _) };

    #[link_section = "__TEXT,__objc_methname,cstring_literals"]
    #[export_name = "\x01L_OBJC_METH_VAR_NAME_.__objrs_meth.print"]
    static METH_NAME: [u8; 6usize] = *b"print\x00";

    #[link_section = "__DATA,__objc_selrefs,literal_pointers,no_dead_strip"]
    #[export_name = "\x01L_OBJC_SELECTOR_REFERENCES_.__objrs_sel.print"]
    static SEL_REF: &'static [u8; 6usize] = &METH_NAME;

    let msg_send: unsafe extern "C" fn(objrs::runtime::Class,
                                       *const [u8; 6usize], u32, u32, u32);
    msg_send = unsafe {
        core::mem::transmute(objrs::runtime::objc_msgSend as *const ())
    };
    let sel = unsafe { core::ptr::read_volatile(&SEL_REF as *const _) } as *const _;
    return unsafe { msg_send(this, sel, x, y, z) };
}

The public user-facing function is print, which delegates to the trampoline __objrs_msg_send_print. The use of core::mem::MaybeUninit for the first two arguments to __objrs_msg_send_print allows the compiler to better optimize the trampoline since the parameters fully match up between __objrs_msg_send_print and msg_send. No register spilling or shufflig is required. This generates the following optimal assembly:

__objrs_msg_send_print:
    movq L_OBJC_CLASSLIST_REFERENCES_$_.ClassName@GOTPCREL(%rip), %rax
    movq (%rax), %rdi
    movq L_OBJC_SELECTOR_REFERENCES_.__objrs_sel.print@GOTPCREL(%rip), %rax
    movq (%rax), %rsi
    jmpq *objc_msgSend@GOTPCREL(%rip)

If you remove the two MaybeUninit parameters (so the arguments to print and __objrs_msg_send_print are identical), suboptimal register shuffling is required:

example::__objrs_msg_send_print:
    movl %edx, %r8d
    movl %esi, %ecx
    movl %edi, %edx
    movq L_OBJC_CLASSLIST_REFERENCES_$_.ClassName@GOTPCREL(%rip), %rax
    movq (%rax), %rdi
    movq L_OBJC_SELECTOR_REFERENCES_.__objrs_sel.print@GOTPCREL(%rip), %rax
    movq (%rax), %rsi
    jmpq *objc_msgSend@GOTPCREL(%rip)

This kind of optimization is only possible if MaybeUninit has the same representation/ABI as its underlying type. Making MaybeUninit #[repr(transparent)] would make optimizations like this much more reliable.

More generally, these unions are wrappers around a type T that are intended to change its behavior but not its' ABI or representation. Permitting #[repr(transparent)] on unions would allow them to better fulfill that end.

Guide-level explanation

A union may be #[repr(transparent)] in exactly the same conditions in which a struct may be #[repr(transparent)]. Some concrete illustrations follow.

A union may be #[repr(transparent)] if it has exactly one non-zero-sized field:

// This union has the same representation as `usize`.
#[repr(transparent)]
union CustomUnion {
    field: usize,
    nothing: (),
}

If the union is generic over T and has a field of type T, it may also be #[repr(transparent)] (even if T is a zero-sized type):

// This union has the same representation as `T`.
#[repr(transparent)]
pub union GenericUnion<T: Copy> { // Unions with non-`Copy` fields are unstable.
    pub field: T,
    pub nothing: (),
}

// This is okay even though `()` is a zero-sized type.
pub const THIS_IS_OKAY: GenericUnion<()> = GenericUnion { field: () };

Reference-level explanation

The logic controlling whether a union of type U may be #[repr(transparent)] should match the logic controlling whether a struct of type S may be #[repr(transparent)] (assuming U and S have the same generic parameters and fields).

Drawbacks

  • #[repr(transparent)] on a union is of limited use. There are cases where it is useful, but they're not common and some users might unnecessarily apply #[repr(transparent)] to a union.

Rationale and alternatives

It would be nice to make ManuallyDrop<T> and MaybeUninit<T> both #[repr(transparent)]. Both those types are unions, and thus this RFC is required in order to allow making those types transparent.

Of course, the standard "do nothing" alternative exists. Rust doesn't strictly require this feature. But it would benefit from this, so the "do nothing" alternative is undesirable.

Prior art

See the discussion on RFC #1758 (which introduced #[repr(transparent)]) for some discussion on applying the attribute to a union. A summary of the discussion:

https://github.com/rust-lang/rfcs/pull/1758#discussion_r80436621 nagisa: "Why not univariant unions and enums?" nox: "I tried to be conservative for now given I don't have a use case for univariant unions and enums in FFI context."

Specify #[repr(transparent)] by nox · Pull Request #1758 · rust-lang/rfcs · GitHub eddyb: "I found another important usecase: for ManuallyDrop<T>, to be useful in arrays (i.e. small vector optimizations), it needs to have the same layout as T and AFAICT #[repr(C)] is not guaranteed to do the right thing" retep998: "So we'd need to be able to specify #[repr(transparent)] on unions?" eddyb: "That's the only way to be sure AFAICT, yes."

Specify #[repr(transparent)] by nox · Pull Request #1758 · rust-lang/rfcs · GitHub joshtriplett: "In terms of interactions with other features, I think this needs to specify what happens if you apply it to a union with one field, a union with multiple fields, a struct (tuple or otherwise) with multiple fields, a single-variant enum with one field, an enum struct variant where the enum uses repr(u32) or similar. The answer to some of those might be "compile error", but some of them (e.g. the union case) may potentially make sense in some contexts."

Specify #[repr(transparent)] by nox · Pull Request #1758 · rust-lang/rfcs · GitHub pnkfelix: "However, I personally do not think we need to expand the scope of the feature. So I am okay with leaving it solely defined on struct, and leave union/enum to a follow-on RFC later. (Much the same with a hypothetical newtype feature.)"

In summary, many of the questions regarding #[repr(transparent)] on a union were the same as applying it to a multi-field struct. These questions have since been answered, and I see no problems with applying those same answers to union.

Unresolved questions

None (yet).

Future possibilities

Univariant enums are ommitted from this RFC in an effort to keep the scope small and avoid unnecessary bikeshedding. A future RFC could explore #[repr(transparent)] on a univariant enum.

7 Likes
2

I was worried it'd be something more complicated, but if it's as simple as that this sounds great!

1 Like
3

Definitely in favor!

One correct though: ManuallyDrop is not a union, and it already is repr(transparent).

1 Like
4

Is MaybeUninitialized the only use case for a union with one non-zero-size field? (Though I’m not sure what this would say. Is making at a special case better or worse than extending the rule to other such unions, however unlikely they are?)

1 Like
5

I am very confused by the example given. I apologize for digging into something explicitly not intended to be “canonical” or “the best example”, but I do not see any strong relation to the pre-RFC at all. If I understand correctly, then

  • the MaybeUninit arguments are not used at all, neither by callers nor callees, they are only added to make the actually-used arguments end up in more desirable registers
  • this effect could be achieved more naturally with more direct control over the ABI of __objrs_msg_send_print
  • alternatively it could also be achieved with a different (non-union) type as well
  • The status quo already achieves the codegen you want
  • Though the current codegen is not guaranteed, if it ever changes then objrs will just take a performance hit, not start hitting UB
    • (Worth noting that the codegen guarantee of repr(transparent) is about ABI compatibility, not about the actual thing you care about, good register allocation.)

If that’s correct, then I feel like this example is not just “not the best example” but at best tangentially related to repr(transparent), and I’d really like to see an example for which repr(transparent) is actually important.

6

Don’t want to bikeshed too much, but an alternative would be to allow #[repr(transparent)] on any union, and give it the representation of its first member or something (which would have to be large enough). Thus, the following union would have the ABI of a u64, which would make it a way to pass around a pair of u32s that’s more efficient in certain cases:

#[repr(transparent)]
union Foo {
    one_u64: u64,
    two_u32s: TwoU32s,
}
#[repr(C)]
struct TwoU32s(u32, u32);

Edit: Or if, say, you ran out of registers for a function’s integer arguments, and instead of putting the rest on the stack as usual, you decide you want to stick them in floating-point registers for some reason :slight_smile: :

#[repr(transparent)]
union Bar {
    floating: f64,
    integer: u64,
}
2 Likes
7

Oh man, I didn't realize it had changed. It used to be a union. Thanks for the heads up. I'll update my local copy of the text to remove mention of ManuallyDrop.

It's the only case that comes to mind, but I'd be surprised if other use cases don't exist.

Correct.

Yes, but there is no mechanism in Rust to have more control over this. Inline asm! might work inline asm! is never going to get stabilized, and even if it did I'm not confident I could make it work reliably (because the method needs to match the C calling convention so it can jmp directly to objc_msgSend, which means I'd need to match the C calling convention, which would require knowing the ABI of all the arguments, none of which is currently possible).

I haven't been able to find any (non-union) type that works. Everything has to be initialized before it is used in Rust. ZSTs don't work because they aren't actually passed as arguments. The only way to have an uninitialized value is to use MaybeUninit (or an equivalent union).

That's totally true, and there is no guarantee that the code generator will generate the optimal assembly for this. But having T and MaybeUninit<T> match at the ABI level makes this kind of optmization much more likely to succeed (otherwise, if T's ABI is to be passed in a register but MaybeUninit<T>'s ABI is to be passed on the stack (with a pointer to the memory location), then this attempted optimization will not just fail but actually backfire).

I agree. I have some other FFI-related ideas I need to explore to see if they're better justifications for this RFC, but I'd be happy to hear any other use cases others might have.

That's a really interesting idea. I'm personally hesitant to reuse #[repr(transparent)] for this. I'd prefer something new like #[repr(type = DesiredRepresentationType)] (where, for your examples, DesiredRepresentationType would be u64 for Foo and f64 for Bar), proposed in a separate RFC.

9

Why shouldn’t unions with one non-zero sized fill not always be automatically “transparent” (have the same ABI as the non-zero field)? EDIT: too many negatives, the question is why do we need to add repr(transparent) at all? We should do this automatically.

We had this problem with MaybeUninit not propagating the ABI of SIMD types in libcore, and IIRC we fixed it by making unions with one non-zero-sized field always propagate the ABI of the type. What does repr(transparent) add on top of that?

EDIT: If you want the non-transparent behavior, you can do struct Aggregate(MaybeUninit<f32x4>); .

1 Like
10

What about types like [0; f64] with zero size but non-zero alignment?

11

Same answer as for structs, I assume: repr(rust) is free to do whatever optimizations the compiler wants, but if you want to rely on it you should say on what you're relying.

7 Likes
12

Unless we guarantee that repr(rust) does this, in which case, adding repr(transparent) becomes unnecessary.

So my question is still, why should we add repr(transparent) instead of guaranteeing that repr(rust) does this? Which optimizations are enabled by not guaranteeing "repr(transparent)"-like behavior for repr(rust)`?

In particular, libstd is already relying on repr(rust) doing this. We can change that if we have to add repr(transparent) though, but we already do have some experience with making repr(rust) guarantee this. What have we learned from this experience? Have we learned of any optimizations that have become or will be impossible because of this? Can anybody think of any? How important are they? Etc.

I'm not sure what value does adding a second solution to this problem add. If the value is "future-proofing", then an RFC should make a point of what are we "future-proofing" for, e.g., which optimizations will this enable for repr(rust) unions. If the answer is "none", then IMO the case for adding this is very weak, it is just one extra thing people will need to remember when using unions that we could do automatically instead.

Maybe even worse than that, we will be doing this automatically for them anyways, but if they silently rely on that they would be relying on unspecified behavior while this is something that we could just easily specify. At that point we might want to consider adding a lint that detects these types of unions and tells users to add repr(transparent) to avoid relying on unspecified behavior.

13

repr(transparent) is still useful because it will show an error when you intend something to be transparent, but accidentally violated one of the constraints (like having only one non-ZST field).

6 Likes
14

That should be its main selling point.

15

Thanks everyone for the input. I have revised the text and opened a merge request to rust-lang/rfcs.

1 Like
16

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.