As part of Project Safe Transmute, I, @rylev@joshlf and others have drafted a design of a compiler-supported API for safer transmutation. At the heart of the proposal is a trait that is implemented automatically by the compiler for all types where a transmutation is statically guaranteed to be sound, safe, and stable.
Click here for a rustdoc overview of the proposed API.
Click here to read the pre-RFC in its entirety (warning: long!).
Compared to prior art, this proposal contributes a design for truly general transmutation (just like mem::transmute), that does not compromise on safety and does not pose a stability hazard. With this API, it might someday be possible deprecate mem::transmute altogether!
Please let us know what you think! I am especially interested in any use-cases which are not covered by our proposal.
One small thing I'm worried about is NeglectStability.
As of current, it is impossible for me to write safe Rust code which relies on nonpublic details of dependencies (so long as said nonpublic details are actually nonpublic and not just #[doc(hidden)] and exposed for e.g. macros). If I want to reach through the stability barrier, I need to reach for the unsafe transmute.
Given a safe NeglectStability, though, we've added a safe way to rely on implementation details of a dependency. This is still explicitly opted into (probably, if NeglectStability is never inferred, in any way) but scares me.
Specifically because I can write a case where data changes without the representation changing, so the client code wouldn't stop compiling, it would just silently be wrong.
I don't know the exact options required to opt-in to just this, but say I want to stably allow transmuting &Complicated to &ComplicatedHead.
If a consumer neglects stability and transmutes &Complicated to &[u8; 3], they can read my private fields. If I change the order of them in a subsequent release, their code is now silently giving the wrong results.
The ability of using NeglectStability to read (or even change) private fields that could have arbitrary safety requirements thoroughly scares me. I know you address some of this with the talk of implicit constructability, but as of my last reading I'm not yet convinced that every pitfall around private data (within a fixed, defined space) is guarded against in the face of NeglectStability.
I'm not yet convinced that every pitfall around private data (within a fixed, defined space) is guarded against in the face of NeglectStability .
Since constructability is really subtle to get right, the RFC also proposes a simpler, incomplete formulation of constructability. Because this simpler formulation of constructability is incomplete, NeglectStability can pose a safety hazard. We therefore recommend that NeglectStability is initially not a safe transmutation option.
I think your solution is safe (and I'm very impressed by the RFC in general), but is there not a backwards compatibility problem when the compiler transitions from the simplified to the more exact constructability criteria?
As I understand it the simplified criteria allows more cases (at the cost of making NeglectStability an unsafe option only).
But couldn't there be unsafe_transmute (with NeglectStability) calls that are allowed under simplified constructability, but forbidden under the more precise constructability criteria?
Independently I think there is some overlap between this RFC and ABI stability that could be discussed in the RFC:
The RFC requires repr(C), but if there is another stable representation such as discussed in the stable modular ABI thread, it should also work.
derive(PromiseTransmutable) seems to correspond to library ABI stability for this type. Maybe in the future this could thus be required for public types of stable ABI crates. Considering this use case, maybe a more neutral name could be chosen, thus as derive(StableLayoutGuarantee).
There are plenty of situations where you want a safer transmute, but not necessarily a safe transmute. This is where the options system of the RFC comes in. You might want to give up some static guarantees of safety when there is a more powerful runtime check you can perform; e.g., NeglectAlignment unsafely disables the static alignment check.
Without the options system, the only way to perform a conditionally-safe transmutation would be to use the wildly unsafemem::transmute and company.
My hope is that additional options will be added in the future until there is no conditionally-safe use-case that isn't solved by this RFC with the right set of options! A NeglectConstructability option is one such possible future option, but requires a lot of design work to get right.
Someday moving to the complete formulation of constructability should generally not cause backcompat issues.
The exception to this rule is the pub-in-priv trick. The documentation for the stability declaration traits should be clear you should not implement these traits for your type if you are using the pub-in-priv trick to restrict their implicit constructability.
I'm surprised the RFC doesn't mention integer byte order concerns. I think it should be explicit about whether e.g. a u32 <-> [u8; 4] conversion is or isn't considered a “safe transmute”.
There is a section on platform-dependent layouts, but it's not (yet) explicit that endianness is a kind of platform-dependent layout. Our RFC is expressly oblivious to platform-dependent layouts. Safety, in this RFC, is scoped to memory safety. A transmutation from u32 to [u8; 4] might be inconsistent between platforms, but it is not unsafe.
This is a very worthy project. The first obvious thing that strikes me about it is that the reuse of the rust-specific term "transmute" will be potentially confusing in the future. "mem::transmute" and this API are quite different. Talking about "transmute" when it means two different things will be complicated and pretty much guarantee the term is always accompanied with a clarifying adjective, like "unsafe transmute" vs "safe transmute".
I'd suggest giving this a new name, maybe with "cast" in it. This could be an opportunity to use the C++ "reinterpret" terminology, like "ReinterpretInto", "ReinterpretFrom", but anything else will make it easier to discuss. If we want to keep it cute and magic themed, maybe come up with another magical word that also has a connotation of safety.
Could you elaborate? The lense through which I've always seen this proposal (and, before it, typic), has been: What's the trait bound we add to mem::transmute to make it safe? And this RFC's answer is:
(Of course, we cannot literally just slap trait bound on mem::transmute—that would a breaking change. I propose two free functions, safe_transmute and unsafe_transmute, and that we perhaps someday deprecate mem::transmute.)
Maybe? I suspect that would be very uncommon. I don't think we even have a complete sense of all the options that might eventually exist. For instance, a hypothetical NeglectConstructability option would have subtle interactions with !Send, !Sync, UnsafeCell and any abstraction whose safety depends on restricting visibility. Deciding if and how to partition that space of dangerous transmutations between different options is going to be a substantial undertaking.
What's important is that the API surface can handle that future work, and it can: those additional options can be freely added in the future if they're deemed necessary. I would love for this API to eventually get to a place where any use of mem::transmute necessitated by a lack of neglectable checks in the safe API was an indicator of unsoundness.
Is there a way for something like NonZeroU8 to be transmutable via ignoring validity?
Also how does this act in generic code? Will the fixed/fully completed constructibility check be dependent on which module the code is in? (Would transmuting a struct with a pub(crate) field inside that crate work?)
I agree that it can be slightly confusing to talk about when the APIs are not qualified as unsafe vs safe, though I think the RFC does a good job of making it clear when the safe variety is meant and when the unsafe one is. I do, however, like the idea of borrowing terminology from C++ rather than using the prefix safe_ to distinguish it with the existing API. I don't think that that needs to be decided in this RFC, however.
Yes, this is precisely the sort of situation where the proposed NeglectValidity option is useful!
Yes, the full formulation of constructability is intrinsically tied to scope. So, whether you can transmute a struct with a pub(crate) field inside it would depend on where you attempt to do it.
The complete details of this full formulation are not totally fleshed out. It won't be as simple as *assess constructability at the point where TransmuteFrom occurs. TransmuteInto, for instance, is just a blanket implementation over TransmuteFrom—but no end-users types are constructible from the libcore! You really need to assess the provenance of the TransmuteFrom bound.
I don't have anything to add other than overwhelming positive feedback and appreciation for the work that has gone into this. Having this kind of functionality in the language and in std makes me downright giddy. It will be a dramatic improvement to many types of unsafe code, and will also open the doors to writing safe code that we can be sure is correct. Doing this in Rust today is of course possible, but there are so many invariants to check that I often avoid doing it because it isn't worth the risk. But this RFC appears to largely de-risk it, which I think will be a huge boon.
Is there a reason pub(crate) is being considered here, instead of just defining constructibility as having all bare pub files recursively and treating pub(crate) as private? Given this is purely for backwards compatibility, I agree that a complex definition might not be useful. (Crate authors wanting to use these impls can/should just use the PromiseTransmutable* traits instead).
