Attention Hackers: Filling Drop
The zeroes you are reading today may not be there tomorrow.
As part of a future-proofing effort to pave the way for non-zeroing drop / RFC 320, pnkfelix plans to put together a Pull Request that replacing today’s zeroing drop with a so-called “filling drop.” He hopes to get the Pull Request together in time for the 1.0 beta.
This future-proofing effort is a relatively short-term step towards implementing a feature whose RFC has already been accepted. Therefore, pnkfelix (with other members of the team) decided it did not warrant an RFC (meeting notes). But it is a big enough change to warrant discussion in a forum less transient than IRC; thus, here we are.
(If you do not know what “zeroing drop” is, you might consider reading the how dynamic drop semantics works appendix of RFC 320.)
RFC 320 describes a long-term strategy for replacing zeroing-based dynamic-drop with a non-zeroing variant.
However, community members have raised concerns that if we ship 1.0 with the dynamic drop semantics as implemented today, developers will end up implicitly relying on the zeroing drop semantics.
For example, a developer who wants to cause a certain set of objects to be forgotten (i.e. not have their destructors run) may think that it suffices to simply overwrite the entirety of the memory representing those objects with zero bits.
pnkfelix does not know when the full implementation of non-zeroing dynamic drop will land; namely, it is possible it will happen for 1.0. (He hopes to put effort into such during the six weeks between 1.0 beta and 1.0.)
But non-zeroing drop certainly is not going to happen for 1.0 beta, and pnkfelix would prefer to try to put up a better defense against the scenario outlined in the previous paragraph.
Short-term proposal: Filling Drop
Currently Rust initializes the drop-flag of an object with a nonzero byte (it turns out to be a
0x01) whenever the object itself is initialized. When the object and its contents are moved or destroyed, we zero its memory entirely (including its drop flag). Likewise, when a
Box<T> is dropped and deallocated, we can just zero out the memory where the pointer was stored (no separate drop flag needed).
The core idea of filling-drop is simple:
Continue using some nonzero byte (today it is
0x01) for the drop-flag when initializing an object. (Let us name this marker byte, whatever it is,
Instead of overwriting the object’s memory with a zero byte, use some other marker byte, such as
0x1d. (Let us name this marker byte
When checking the drop-flag to decide whether a value needs to have its drop glue run, change the existing logic so that it skips when it sees a
DONE_DROP(rather than conditionalizing on zero specifically).
In addition to the previous bullet, when checking the drop-flag on debug builds, invoke an
llvm::debugtrapif the drop flag is equal to neither
- We issue
llvm::debugtraprather than panicking because debugtrap is cheap in code size (at least on Intel). However, pnkfelix is open to alternative ideas on this point.
- We issue
(Note that much of the code for the above could be implemented today as a refactoring of the existing code, with no observable change apart from the debug build checking, by just assigning the values
NEED_DROP = 0x01 and
DONE_DROP = 0x00.)
Code using the unstable attribute
#[unsafe_no_drop_flag]and relying on zeroing for their destructor logic will need to do something else
std::memwill expose (unstable) identifiers that provide value for
DONE_DROPat various bit-widths, just so such libraries can get something running again with relatively little fuss. But obviously it would be better (and in the long-term, outright necessary) for those libraries to stop relying on such logic.
initintrinsic will be replaced with two distinct intrinsics:
init_dropped. (We still provide
init_zerorather than just change the semantics of
initbecause there are many pieces of code that rely on zeroing on initialization, e.g. for FFI interactions, and have nothing to do with Rust’s destructor semantics.)
The main goal of filling-drop is to catch libraries that are today relying on memory zeroing.
Our primary interest is to identify libraries that is today writing zeroes themselves to cause destructors to be skipped, and tell them to stop doing this. Likewise, libraries that use
unsafe_no_drop_flag will be flagged (see above note about
A secondary concern is about libraries that assume that other clients that read from dropped memory will always only see zeroes (a ficticious example: a library that passed pointers to dropped memory into a C foreign function that wants null-terminated strings).
- The current filling-drop plan does not future-proof things as much as one might desire for this case, since such a client might only be relying on the fact that the memory gets overwritten by some garbage data, and the fact that it is zero data is irrelevant. But, one cannot please everybody; that library is simply going to break when non-zeroing drop is actually implemented.
Is this strategy dangerous? Of course it is, in many ways!
Here is one example problem: the reason that zero was a particularly good choice for the role of was because a pointer-sized value filled with zeroes is (for our target architectures) the same as the null pointer; i.e. values after zeroing never corresponds to valid memory addresses. This property might not hold for a filling-drop implementation.
- Since we control our allocator, we should be able to ensure for the short-term that at least no
Box<_>will never be mistaken for
DONE_DROP, so this may not be that big of a problem.
- In any case, pnkfelix suspects the easiest way around this will be via selecting an appropriate value for
0x1d1d1d1dmight alias some valid address on a 32-bit system, but
0xfdfdfdfdprobably does not.
Does this accomplish anything?
One might ask: Won’t these libraries that were relying on zeroing-drop just start relying on filling-drop? In particular, couldn’t they just start filling in the memory with the
0x1d byte or whatever
Its a reasonable question, and of couse, yes, we cannot prevent libraries from actively shooting themselves in the foot. However, it seems less likely that someone would start relying on the filling-drop semantics by accident.
Won’t this slow things down?
Probably not by much, but many code sequences can do tricks with zero that are not available when one is using non-trivial values for
pnkfelix plans to measure the cost and make sure that code size is not being blown up by too much.
But the more important thing to note is this:
This is a short term plan.
It is just a future-proofing placeholder for a proper non-zeroing dynamic drop to be added later.
pnkfelix has a prototype branch of
rustc with this change, at drop-flag-hacks.
That url again, for those of you looking at a PDF or screen capture:
pnkfelix believes the prototype might pass
make check, but he has not had a chance to confirm that yet.
- (He only just this evening found a cause of a bug that was causing
rustcitself to segfault on one compile-fail test case, and now sees his prototype has gotten through all of
compile-failon stage2, and is working on the rest of
rpass-full; but he will not have the chance to update this thread for a few days, so the outcome will remain a mystery for you, dear reader)
If you want to help, it would be great if you could try just to clone the drop-flag-hacks repo and try it out on your host platform and your favorite crates.
Or if you want to offer advice on other ways to future-proof the compiler and runtime for nonzeroing-drop, feel free to chime in on this topic thread.
Thanks for reading!