repr(C) AIX Struct Alignment

compiler
41

I have filed an issue against LLVM in the hopes that this can be pursued upstream: [AIX] LLVM data layout is incorrect unless overridden by clang · Issue #133599 · llvm/llvm-project · GitHub

1 Like
42

Overall that's great news then, the target is not as busted as we thought. :slight_smile:

Ad-hoc overalignment is still annoying and different from how we document repr(C) works, but it's no worse than the MSVC issue.

43
2 Likes
44

Uh, how sure are we about this?

I found this example

struct Struct1 {  
  double a1;    
  char a2;
};

Apparently this is supposed to have size 16, because "the member with the largest alignment requirement is a1; therefore, a2 is padded with 7 bytes". If we just set double to have alignment 4, we'll give this struct size 12.

45

What about if you do __alignof__(some_struct.field_of_type_double_with_4_byte_alignment)?

I ask because the documentation of __alignof__ for this compiler says (emphasis mine):

The __alignof__ operator is a language extension to C99 and C++03 that returns the position to which its operand is aligned. The operand can be an expression or a parenthesized type identifier. If the operand is an expression that represents an lvalue, the number that is returned by __alignof__ represents the alignment that the lvalue is known to have. The type of the expression is determined at compile time, but the expression itself is not evaluated. If the operand is a type, the number represents the alignment that is usually required for the type on the target platform.

46

One potential approach, to address both this issue and the MSVC x86_32 uint64_t issue, is to split the notion of a type’s alignment into preferred and minimal alignment. These would be defined as follows:

  • Preferred alignment is what core::mem::align_of() returns.
    • Fields of repr(Rust) ADTs are always aligned to their preferred alignment.
      • The same guarantee applies to repr(linear), if that ever becomes a thing (including if it’s eventually decided that repr(C) should mean that).
    • Local variables are guaranteed aligned to at least preferred alignment.
    • Collections like Box and Vec provide (and assume, when constructed with e.g. from_raw/from_raw_parts) preferred alignment.
    • If a type has an ABI-compatible equivalent in the platform C compiler, its Rust preferred alignment will most likely match what that compiler says its alignof/_Alignof/__alignof__ is.
  • Minimal alignment is the minimum alignment required for reading a type to not be UB.
    • References have a validity invariant of always being aligned to at least the minimal alignment of their pointee type (but don’t require more, either as a validity or safety invariant).
    • The minimal alignment a repr(Rust) or repr(linear) ADT is guaranteed to equal its preferred alignment, on all platforms.
    • If a type has an ABI-compatible equivalent in the platform C compiler, and that C compiler provides a standard-compliant alignof/_Alignof, the minimal alignment of the Rust type will match what those C constructs say its alignment is.
    • On almost all platforms (I think x86-32 MSVC and AIX would be the only 2 exceptions so far?), minimal alignment would always equal preferred alignment.
    • A new API, e.g. core::mem::min_align_of(), should be added to allow querying a type’s minimal alignment.
      • core::mem::min_align_of_val() should also be added. On the few platforms where minimal and preferred alignment can differ, this would require an additional entry in trait object vtables.

This proposal conflicts with the currently documented guarantee that core::mem::align_of “returns the ABI-required minimum alignment of a type in bytes”. But that guarantee already doesn’t hold in practice on the 2 problematic targets. The changes described above would not change the return value of align_of() or the alignment provided by any Rust API, so they can’t break any code that works today. (Of course, if repr(C) were to be changed to actually match C, that would be breaking; there is no real way around that.)

47

_Alignof and __alignof__ are not the same.

48

The clang implementation still reports the _Alignof for the struct as 4. The struct may have to be given further padding, but that just brings us back to "repr(linear) is not repr(C)", not some alien universe.

49

I'd prefer an approach that confines the insanity to the two targets that decided they don't need common sense: have align_of return the alignment that all instances of a type respect (i.e., for double on AIX it is 4), but have struct computation sometimes add extra padding at the end. This also helps with other problems, for instance, we also need #[repr(C)] struct Zst([u8; 0]); to add padding to get a size of 1 on MSVC targets (see here).

We need to break with repr(C) == repr(linear) anyway; once we do, this should all be fixable.

I take it that clang does add the trailing padding as required by the docs?

I was about to say this _Alignof value sounds like a clang bug, but presumably if this struct type features as the 2nd field of another struct type, it will indeed be just 4-aligned, so 4 is indeed the actual minimal alignment of that type.

So really the oddity can be entirely described by adding more trailing padding? But... why would one do that?! Is it so that arrays of this type keep the double field 8-aligned if the array itself is 8-aligned, or so?

1 Like
50

I take it that clang does add the trailing padding as required by the docs?

Yessir. You did indeed pull a good catch!

We will have to revise the layout algorithm slightly if we want to adhere to repr(C) on that platform but, as I've discussed, it should be a fairly modest amount of fucking about compared to the possible insanity that we were thinking about before. "Insert padding here and there" is leagues simpler.

So really the oddity can be entirely described by adding more trailing padding? But... why would one do that?! Is it so that arrays of this type keep the double field 8-aligned if the array itself is 8-aligned, or so?

I think that is indeed it, plus maybe a consideration here and there about C++ subobject layout.

51

Only the former is part of the C standard, yes. But does there exist any compiler for which they give different values for the same type?

52

This would be the “cleanest” option, yes, but I’m afraid it would break too much existing code. Especially as the current repr(C) would match neither the new repr(C) nor the new repr(linear) on MSVC.

53

So Clang’s _Alignof doesn’t match the non-Clang compiler’s __alignof__?

54

Yeah, I don't know what a good transition looks like for the 32bit MSVC issue. But if just changing the layout on that target is not acceptable, I'd rather not fix that issue than burden the language with the idea of "preferred alignment".

For everything else, I still think we can

  • add repr(linear), currently identical to repr(C)
  • in a future edition, change the meaning of repr(C) to mean "the C layout for this target"

Maybe we can make repr(C_2027) available in older editions if people want to eagerly opt-in, or so. My hypothesis is that much more code relies on repr(C) being like the system C ABI than it following the documented layout algorithm. And anyway for most targets, repr(C) and repr(linear) will remain the same.

55

For the x86-32 MSVC issue, there is also a performance consideration—AIUI double should be aligned to 8 bytes for good performance, and MSVC will often try to do that, but only guarantees 4. If Rust doesn’t even make an effort to provide 8-byte alignment of double, that could regress performance. (I have no idea how significant this would be)

56

Yes.

1 Like
57

Such as?

58

Not sure how much we should be concerned with performance on what is effectively a retro platform at this point. We can figure out how to over-align some local variables if this actually turns out to be a problem. I don't think this has to affect the language-visible part of the design.

2 Likes
59

Yes, I suppose if both local variables and repr(Rust) over-aligned, that might mitigate the issue…

60

Considering how long this discussion and others have been going on, is there any reason we shouldn’t start (RFCing and) adding repr(linear) now, regardless of what more is learned about C compilers and regardless of whether repr(C) ever changes? Even if it never becomes distinct, it would be useful for making code’s layout intent clearer.

4 Likes