`freeze(MaybeUninit<T>) -> MaybeUninit<T>` for masked reads

I've lost the thread of this conversation.

To recap: I've argued that semantics (1) I listed above, in which we read all the bytes from memory, and then write all the bytes back to memory, either in the same location (freeze_mut) or in a different location (freeze_by_value), are correct.

Do you agree that semantics (1) are correct?

1 Like

This is the MADV_FREE problem. Even though freeze takes care of the potential issues at the LLVM level, the kernal can allocate/deallocate pages of memory. This can manifest as reading the value of a byte (let's say you read 5. You freeze it! You print it to the console), writing elsewhere to the same page of memory, and then reading 0. You can reverse that, deallocating the page of memory. Even with freeze.

You can't get around this with *x = *x, or anything that optimizes to *x = *x, because the compiler will optimize that away. Which is to say as soon as your proposed signature goes from

// breezy cool
fn freeze<T>(v: MaybeUninit<T>) -> MaybeUninit<T>;

to

#[inline] // uh oh
fn freeze<T>(v: MaybeUninit<T>) -> MaybeUninit<T>;

the hermetic seal is broken. There is no (cost free) LLVM IR you can emit to fix this. You could zero out the memory when calling MaybeUninit::uninit for instance, that would work.

This is why I'm asking if we could get a mem::freeze_esque function that works if we're definitely reading from an initialized page. The answer is probably that the kernel is still allowed to mess with you, but also probably that it will work on all major rust targets. Since your target is actually pretty specific, that would be enough to implement masked_load.

Why is it necessary to deal with MADV_FREE here? The user could do any of a number of things that involve interfacing with that kernel in ways that interact with the running process. The user can write to process’ memory map safely anyhow.

1 Like

MADV_FREE is used by jemalloc and it is generally expected that memory returned by the allocator is uninitialized, but not in any other way special. Other ways of messing with the memory are UB at the point of this messing, not at the point of using the memory with which was messed. However MADV_FREE'd memory must be fine for MaybeUninit for jemalloc to be considered sound.

There's some good discussion about this in

General MaybeUninit::freeze is going to have to deal with everything brought up there.

However, the compiler cannot optimize *x = freeze *x away[1], specifically because of things like this. This is actually what case (1) is; LLVM has a freeze operation now, with the desired semantics.

(At least, OP said this. I did not confirm.)


  1. Specifically, if the value of *x is not known to be a non-uninit non-poison value, this has the side effect of setting *x to an arbitrary (but hereafter consistent) value. If *x is known to be not-uninit and not-poison, the operation is known to be a noöp and can still be removed. ↩︎

2 Likes

Note that fast_eq has very strange behavior here, even violating the functional requirements of Eq, since fast_eq(x, x) can return false.

fast_copy can easily be implemented in today's Rust by doing the copy at type MaybeUninit<u64>.

It's kind of a grey area -- the behavior of load_and_freeze can be described as an Abstract Machine transition, but it is a transition that no Rust program can ever take. We need to be careful with such transitions. E.g. if you used inline assembly to read the value of a local variable whose address was never leaked outside that function, then that is also a transition that can be described as an Abstract Machine transition, but it is certainly not something we want to allow.

But it is hard for me to imagine how implementing freeze via inline assembly could cause problems.

It is certainly not documented to be sound. I have no idea whether these intrinsics should be specified as regular Rust loads or as sugar for inline assembly.


I am not fundamentally opposed to freeze, but besides the MADV_FREE trouble, whoever writes the RFC to add this to the language should also consider the arguments laid down here. freeze is subtle because it lets you write non-UB programs that expose the values of deallocated memory.

2 Likes

Could freeze just be unsafe and part of the safety contract would go to only call it on values not mapped with stuff like MADV_FREE?

I don't see how this is the case, maybe I'm missing something? In my example, bytes 0, 4, 5, 6, 7 (the non-padding bytes of the struct) have defined values and bytes 1, 2, 3 have uninitialized values (the padding bytes of the struct). After freezing, I masked out the uninitialized values. The end result of fast_eq is that it compares exactly the non-padding bytes of the struct, as would be expected.

To make sure I'm understanding you right, I think you're saying:

grey area / questionable:

pub unsafe fn load_and_freeze_asm(x: *const MaybeUninit<u64>) -> u64 {
    let result: u64;
    std::arch::asm!(
        "mov {dst}, [{src}]",
        dst = lateout(reg) result,
        src = in(reg) x,
    );
    result
}

probably ok:

pub unsafe fn freeze_asm(x: MaybeUninit<u64>) -> u64 {
    let result: u64;
    std::arch::asm!(
        "mov {dst}, {src}",  // Shame to have to do this noop, but ok...
        dst = lateout(reg) result,
        src = in(reg) x,
    );
    result
}

Did I get that right? Sadly, freeze_asm is illegal in Rust, because MaybeUninit can't be used as an operand in inline assembly.

Indeed, thanks for the reference. For my use cases I see freeze primarily as a mechanism by which I can offer masked_load. And masked_load by design masks out any uninitialized data. So I would imagine having freeze be marked unsafe with its safety requirement being that callers do not expose uninitialized data to safe code. Derived functions such as masked_load could be safe functions.

1 Like

Yes, this is the key point. LLVM's freeze operation has an actual semantics in the LLVM abstract machine: it turns poison and undef values into non-deterministic-but-defined values. It can only be optimized away if the compiler can prove that the input is neither poison nor undef.


  1. Specifically, if the value of *x is not known to be a non-uninit non-poison value, this has the side effect of setting *x to an arbitrary (but hereafter consistent) value. If *x is known to be not-uninit and not-poison, the operation is known to be a noöp and can still be removed. ↩︎

To emphasize that this is not true if there's a freeze there, here's Alive2 generating a counterexample:

(Of course, if there isn't a freeze then Alive2 will confirm that removing it is correct: https://alive2.llvm.org/ce/z/oJNRv_.)

1 Like

Argh, sorry. I should have read the code more carefully. You are right.

No, both of these would be questionable. Any time you do anything with inline assembly that has effects on Rust-visible state (such as the value of local variables, or the contents of memory that Rust knows about), if those same effects would be impossible to create using pure Rust, that is questionable.

Inline assembly cannot be used to extend Rust's ability to act on its own state. This is crucial to ensure correct compilation, since the compiler, when optimizing function X, will justify its correctness for an arbitrary Rust program calling that function -- not an arbitrary assembly program.

I don't see how that can work, since freeze by definition makes it non-UB to expose the contents of uninit memory -- and unsafe is for UB only.

I agree your use of freeze is fine from a security perspective. But I don't think we can use unsafe to ensure such a property, similar to how we do not unsafe to mark potential memory leaks. Rather, if we add freeze, we should probably carefully document that there is a general expectation that the values picked by freeze should not leak into the rest of the program.

Indeed. So when previously you talked about "two possible semantics that one could choose for fn freeze(&mut MaybeUninit<T>)", that was a misnomer -- the semantics of freeze in the Rust Abstract Machine is to turn uninit bytes into arbitrary init bytes. What you described were possible implementations of that semantics.

5 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.