I thought I’d chime in with some comments on Abomonation; I assume it is the same one we are talking about as no one else should spell it that way (where is the “5” from? typo? lots of secret forks doing cool things? :D).
Abomonation does several horrible things, and I wouldn’t be surprised if the correct answer is not “this should be defined behavior”. But, I thought I would point out some of the things that it does, as different from the problem it thinks it has to solve. I have an interest in something with similar properties being possible, even if the pile of code that is Abomonation needs to be deleted and zeroed over several times.
Abomonation was an attempt to deal with the issue that for data intensive compute with communication (ie multiple workers) one really wants some sort of region allocation in order to efficiently bundle up data, ship them to other workers, and unpack the data on the other end. The “competition” in the HPC space just memcpys plain old data around, the competition in the JVM space takes comically large amounts of time to serialize data (or roll their own off-heap serialized representations), and the competition in the C/C++ space does a heap crawl but then has to strongly warn the recipient that they should not use methods that expect to mutate anything about the interpreted data. Rust seems well positioned to do the efficient thing, safely.
Abomonation’s implementation was the optimistic take that maybe the minimum amount of work would be enough: memcpy on each pointer you find into a new region on the encoding side, and some pointer correction and casting on the receiving side. Other options exist, like CapnProto, but there is an ergonomic hit where types in use need some macro love (this is less a problem if you are serializing your own few types, and more of a problem if you are writing middleware for other people who wouldn’t otherwise know you have to do this for them).
There are a few concrete things I know of that are presently UB with respect to Rust’s guidelines:
-
Abomonation calls memcpy to read from structs which may have padding bytes. The last version of the docs that I read said that reads of undefined values is undefined behavior. This is not what LLVM says (such reads result in undef values, which may become UB if they are used badly, e.g. in division denominators), but I can understand why Rust would be more restrictive (perhaps it doesn’t need to be in this case, though).
-
Abomonation totally ignores alignment, and could produce references to types that are not aligned with properties that their initial allocation had. This can be fixed, but it may mean a copy to shift everything by just a few bytes, or only reading data into known aligned locations.
-
Abomonation totally ignores widths, with the assumption that you are using the same binary to encode and decode the data.
Abomonation also does some other weird things that I’m not even sure if they are clearly defined or undefined. The main one is to (i) make a copy of reachable memory from some typed reference &T, (ii) correct pointers in that copy to reference their corresponding locations in the copy, then (iii) cast the result to a &T and act as if it is valid. The UB spec says something about “it is UB for values to be invalid”, where clearly whether what Abomonation does being UB or not depends on what is or is not valid; I suspect the fact that the values are behind a reference is not a saving grace.
For example:
-
I could imagine some version of this working out just fine for &[u64], which has a well-defined size and alignment and all of that.
-
I would imagine no version should work out just fine for &Rc<T>, because the implementation of clone does some funny business with UnsafeCell that is very unlikely to be faithfully reproduced (maybe it works? maybe it just writes over the strong count location in the underlying &[u8], or maybe it does this and some NOALIAS assumptions elsewhere break).
-
I don’t know whether and how this would work for Vec<T> or Option<T> or other composite types. Each of these have some baked in assumptions about their structure, and may not be valid just because you copied their exact bytes. At the moment it “seems to” work, in that I encode and decode a lot of data and don’t experience problems.
On the encode side of things, I have to imagine there is some version of “copy lots of bytes into a buffer” that isn’t undefined behavior, because there are no particular invariants about the resulting Vec<u8>. The decode side seems to be where things are terrifying, because the intent is to let otherwise oblivious Rust code use a &T as if it were such a thing.
So, it is all a bit scary, and a bit of a mess. I’d love to get some guidance (perhaps as a part of this process, or perhaps as the result) about how to make it UB conformant.
I want to double-extra stress that from my point of view it is important to be able to have some sort of correspondence between in-memory representations and “on-disk” representations, if you want performance from data-intensive computation. repr(C) is one way out, but it has a similar ergonomic overhead to CapnProto: either users need to use it on their custom what-would-otherwise-be- (String, usize) types, or accept that each type is wrapped by the system preventing them from using code that expects a &[(String, usize)], and having them re-write it as taking something like &[Wrapper<(String, usize)>]. Either of these approaches also increase the cost of moving data from a Vec<T> into a Vec<Wrapper<T>>, where we may not be able to just memcpy data any more.
Can I set repr(C) for an entire project? This is something I would like out of the UB spec: some way to reliably know (or insist on) something about the in-memory representation of types in the programs I run.
Edit: Just to re-iterate, it is not important that I insist on a specific in-memory representation, but rather consistency of the in-memory representation, whatever its layout. For example, Rust could hypothetically create behind the scenes a few specializations of types with different layouts, and convert between them with care. This would utterly crush Abomonation’s plan, and would stress me out greatly. Field orders that vary build-to-build (which is apparently in nightly?) are also stressful, but are apparently appealing enough to other people that they are worth doing (can I insist on Abomonation vs Serde serialization numbers in any perf benchmarks they produce?).
Edit 2: Awesome side information: Abomonation’s unsafe is literally the only unsafe code in timely dataflow (that I know of). I think that is pretty amazing for the language.
