So, here is how C lays out a struct. For each field in the order you defined them in the source:
Determine the size and alignment of the field.
Skip the number of bytes required from the end of the last field to meet the alignment of the next one.
The field's offset from the beginning is wherever you are now.
And, when you're done with that, make the following two adjustments:
The alignment of the struct is the alignment of the most-aligned field in it.
The size of the struct gets rounded up to a multiple of the alignment. This makes arrays work, because we can find the nth array element with n*sizeof(element).
This last step is unfortunate. It makes ((u16, u8), u8) for example take 6 bytes whereas it would otherwise take 4. (A tuple literally like this is not particularly useful, but imagine structs with similarly-size and similarly-aligned fields where the inner struct is a separate type because it is also used independently.)
What do you all think of making types having “two sizes”? One for how much space is needed to store a single value of this type (in a tuple/struct/enum, on the stack, …) and another for how much space (what offset) from an item to the next in an array or slice. The latter could be called “stride”, and only it would be rounded up. This would make the other size of (u16, u8) be 3 instead of 4, and ((u16, u8), u8) could then be 4 bytes.
One issue is what std::mem::size_of should return. It is used by unsafe code at least in the code for core::slice where it is assumed to be the stride. I don’t have an example in mind, but there could be other unsafe code that uses size_of and assumes it has the other meaning. (Perhaps copying raw bytes without using the typed std::ptr::{read, write} functions?) The code of libcore can be fixed whenever we change the language, but it an indication that other unsafe code outside of the standard library could be making the same assumption.
Perhaps we could deprecate size_of in favor of two other functions, pushing users to consciously choose one or the other. But that would be unfortunate, size_of is a nice name.
This should have been done in 2015 at worst, better earlier
Now you can’t remove the trailing padding by default without silently corrupting memory here and there with write(size_of)s.
You’ll have to introduce a new attribute #[no_trailing_padding] enabling the optimization, then push crate authors to support types marked with it in their generic code, then maybe turn it on by default in a few years, and the breakage will still be possible due to non-generic code.
I’d really like to see at least #[no_trailing_padding] supported as soon as possible, this is a trivial change and it doesn’t break anything immediately.
I agree with the thought, having a different size and stride opens up many “packing” optimizations.
Regarding @petrochenkov thoughts, I am not clear how one would use unsafe code to write a value without write. The lack of Rust ABI makes it quite difficult for someone to know the position of fields, and copying the whole struct, padding included, is likely to raise warnings in tools such as valgrind (unless the padding is actually zero-initialized?).
As such, it seems that the only correct usage of size_of would be to use it as stride, and therefore that its behavior should not change (as unfortunate as it is).
It also seems to me that it is not necessary to expose the actually used size. If one wishes to write a specific field, a combination of offset_of! and write should allow to find the field in a struct and write to it (without overwriting its tail-padding).
Of course it could be nice, but once again the information is of limited use without actually knowing the layout of the structure: there may be padding between data members that is uninitialized and should cause valgrind to complain if copying wholesale.
I tried a search on github to try and gauge how size_of is used… but it seems my search-foo is not really up to par (it mostly brought up rust issues).
let mut val = ((0_u16, 0_u8), 0_u8);
let val0 = &mut val.0;
let val1 = &mut val.1;
*val0 = (1, 1)
println!("{}", val1);
Two issues need to be considered:
How does this change affect the codegen of *val0 = (1, 1)? Are we currently using the ability to scribble over padding for better codegen?
Will an unsafe “equivalent” of this code scribble over val1’s data?
I don’t know about the first one, but the second one is definitely plausible with the inner_size_of proposal. Code like ptr::write_bytes(val0, 0, size_of::<T>()) will suddenly be broken.
It’s possible@SimonSapin’s “deprecate both” proposal will work, but there will need to be a hefty transition period where we don’t actually apply the optimization (or it requires a flag).
This data by @camlorn shows occurences of this rounding (I think it’s the end_padding field that some types have). Many are false positives because strangely closures show as a single field with end_padding: 8, but there are about 1000 occurences which look like legit rounding up, which is about 10% of all types with more than 8 bytes. It looks like this issue has real impact.
Now you can’t remove the trailing padding by default without silently corrupting memory here and there with