[Pre-RFC]: Safe Transmute

language design
22

One question I had with regards to this -- do we make any guarantees today about whether padding must be undefined? That is, is it fine to cast known bytes into padding so long as you don't read from them again? (e.g., is mem::zeroed() fine to use for a struct with padding?)

2 Likes
23

This won't work until at least cont-generics lands

24

My understanding is that you can transmute bytes to structs with padding, just not transmute structs with padding to bytes. From what I understand, it's UB to read padding bytes.

3 Likes
25

FWIW I actually made a crate for in-place coercions: coercion

Currently the API exposes two traits: As for types that can be safely transmuted in-place, and Coerce for types where the transmutation isn't necessarily safe. The API also supports unsized types with a mechanism to convert pointers to those types. I literally just ran into a case where a fallible TryAs version of As would be useful on another project, so I might add that as well.

One annoying thing about As is that it doesn't quite match Rust's as operator, as as can also convert (potentially lossfully) between types of different sizes.

2 Likes
26

Some prior art... the ProcSend trait https://docs.rs/interprocess-traits/0.1.1/interprocess_traits/trait.ProcSend.html and the SharedMemCast trait https://docs.rs/shared_memory/0.8.2/shared_memory/trait.SharedMemCast.html

27

This rocks! It'd be awesome to have something like this in the language, specially for people who do a lot of cross-process stuff.

I'll echo what I said in the interprocess-traits issue tracker, and paste my raw thoughts on the proposal here (taken while reading)

  • I find the idea of a trait that cannot be impl'd to be weird; maybe they mean you can only unsafe impl it?
  • ToFromBytes is a horrible name, I think overall they mean that the type is isomorphic to a byte slice.
  • Maybe having the compiler verify type isomorphisms is where this thing is going?
  • All Transmutable types are isomorphic to one another (via bytes) (?)
  • Maybe they should just be two traits, FromBytes and ToBytes
1 Like
28

It would be really neat if we could specify the alignment on or around the type as well. That seems somewhat doable now for the output type using #[repr(align)].

2 Likes
29

I think the ideal way to do this would be with const generics. But even then as far as I know type-level const parameters canā€™t be used in attributes, so we could for example add a magic std::marker::MinAlign<const A: usize> type that is zero-size and behaves as if it had #[repr(align(A.next_power_of_two()))].

30

Alright, I tried it. Itā€™s getting a little off-topic, sorry!

This works today for a concrete type:

pub unsafe fn usize_as_bytes(x: &usize) -> &[u8; std::mem::size_of::<usize>()] {
    &*(x as *const _ as *const _)
}

If you make the type generic, thereā€™s an error message that sounds reasonable. Would const generics improve this?

// error[E0277]: the size for values of type `T` cannot be known at compilation time
pub unsafe fn as_bytes<T>(x: &T) -> &[u8; std::mem::size_of::<T>()] {
    &*(x as *const _ as *const _)
}

Then we can try some tricksā€¦

trait SizeOf {
    const SIZE: usize;
}

impl<T> SizeOf for T {
    const SIZE: usize = std::mem::size_of::<Self>();
}

pub unsafe fn as_bytes<T>(x: &T) -> &[u8; T::SIZE] where T: Sized + SizeOf {
     &*(x as *const _ as *const _)
}

(Playground)

Errors:

   Compiling playground v0.0.1 (/playground)
error[E0277]: the size for values of type `T` cannot be known at compilation time
  --> src/lib.rs:18:43
   |
11 |     const SIZE: usize;
   |     ------------------ required by `SizeOf::SIZE`
...
18 | pub unsafe fn as_bytes<T>(x: &T) -> &[u8; T::SIZE] where T: Sized + SizeOf {
   |                                           ^^^^^^^ doesn't have a size known at compile-time
   |
   = help: the trait `std::marker::Sized` is not implemented for `T`
   = note: to learn more, visit <https://doc.rust-lang.org/book/ch19-04-advanced-types.html#dynamically-sized-types-and-the-sized-trait>
   = help: consider adding a `where T: std::marker::Sized` bound
   = note: required because of the requirements on the impl of `SizeOf` for `T`

The trait and its impl in isolation compile fine, but using it to size an array gives the same E0277 as before. The help: the trait `std::marker::Sized` is not implemented for `T` part looks wrong, though.

2 Likes
31

Using associated consts in types also falls under const generics. The error messages for this have always been questionable.

1 Like
32

Do you have a suggestion for which types of lints would be most useful?

33

I don't believe that this can be effectively done with a library. If we want to derive Transmutable there are certain checks that need to be done at compile time that I don't believe can be. Since size_of and align_of are const, you can indeed get quite far in proc-macros, but if you need to verify that a field (which has a an arbitrary type T) is itself Transmutable you cannot do that without access to type information. I believe only the compiler will be able to do this. Beyond that, supports for safe unions definitely needs language support to work.

34

That's fair, I think we should remove this.

35

You can emit code that fails to compile if the constraint is not met:

The diagnostics are not ideal, but itā€™s possible. The static_assertions packages this (and more) in nicer-to-use macros: static_assertions::assert_impl_all - Rust

36

Good point, though the value of this may be limited by:

No we canā€™t. Initializing a union with as a "short" variant leaves the rest of the bytes (which are padding for the purpose of this variant) uninitalized. (This is how std::mem::MaybeUninit<T> works, itā€™s a union of T and ().) Then accessing a "longer" variant reads uninitalized bytes, which is UB.

So safe unions could only work where all variants have the same size, and have padding at the same locations.

1 Like
37

This design doesn't support transmuting directly between types that aren't transmutable to/from [u8], and so seems inferior to one who does, such as having a TransmutableFrom<U> trait lazy automatically implemented by the compiler.

38

I hope "wrong" #[derive(Transmutable)] is permitted to exist in generic context, because I'd like:

#[repr(C)]
#[derive(Transmutable)]
struct RGB<T> {
  r: T, g: T, b: T
}

to be Transmutable where T: Transmutable and alignment allows it. I can't expres "alignment allows it" with impl ā€¦ where T: Transmutable, so derive needs to be smart about such special bound.

It's also a bit of a shame that Transmutable requires a manual opt-in. I'm afraid that users who use wrapper types will forget to opt-in, and RGB<u8> will work, but RGB<Wrapper<u8>> won't.

Maybe at least #[repr(transparent)] should carry Transmutable over? Or structs with all public fields? (adding private field to all-public-fields struct is a breaking change already).

1 Like
39

I think we could make impl <T: Transmutable> Transmutable for RGB<T> work.

I think we can safely assume that any type T will always have a size that's a multiple of its alignment. That seems sufficient to address any potential alignment requirements there.

40

Can you imagine a way we could make that work? What would it take to make a union of u8 and u32 safe, given that both accept any possible bit pattern?

41

There's a tougher case:

struct RGBA<T, U> { r: T, g: T, b: T, a: U }

which can be shown to be non-transmutable even if T & U are.