Volatile and sensitive memory

OK, let me make sure I understand:

Given this declaration of a C function:

uint64_t volatile *f();

Is this the proper Rust FFI declaration for this function?:

extern {
    fn f() -> *mut u64;
}

Can we assume that as long as Rust code never attempts to load or store from the result of f, the compiler will never attempt any loads or stores?

Let’s say I want to allocate a buffer that Rust will never load or store from. Is there a way to do that? For example, consider:

struct Foo {
   buffer: std::cell::UnsafeCell<[u8; 64]>
}
impl Foo {
   fn new() -> Foo {
     Foo {
       buffer: std::cell::UnsafeCell::new([0u8; 64])
     }
   }
}

fn main() {
   let _ = Foo::new();
}

It seems like the compiler is allowed to insert spurious loads and stores into the buffer. Is there any way to avoid that? In particular, I would like to pass this buffer to some non-Rust code so that it can use it for whatever it wants, but I want Rust to be responsible for the allocation. Is this possible? In C, we would use volatile uint8_t buffer[64].

Perhaps this is a better way of phrasing the second part: What is the proper FFI declaration for this?:

struct Foo {
    volatile uint8_t buffer[64];
};
#[repr(C)]
struct Foo {
    buffer: ???
}

Yes

Correct

I don't understand your example, why would you ever need to do that? As I said before, volatile access is only useful for memory-mapped I/O.

The case of ACCESS_ONCE in Linux is a bit special since it is relying on volatile loads being atomic. What you really want there is a relaxed atomic load, which volatile reads happen to provide.

In your example the buffer is allocated on the stack and needs to be zero-filled by Rust, which involves storing to it. Could you maybe post a longer example with a C equivalent to show how you envision this buffer being used? As I said before, volatile is only useful for memory-mapped I/O.

#[repr(C)]
struct Foo {
    buffer: [u8, 64]
}

And accessing the contents of the buffer using only raw pointers and volatile operations. Note that this means that you can't create a reference to Foo, only a pointer, if you wish to guarantee no spurious reads are generated.

Here is the real-life code example: https://github.com/briansmith/ring/blob/b95fd0986ac46dc8c5caf1bd85a2526c8b507958/src/aead.rs#L180

In my case, I want the buffer to contain AES encryption keys. It is OK for it to start out being zero-initialized, however once the extern (assembler) code writes the encryption key to the buffer, I want to guarantee that the Rust compiler won’t allow the contents of the buffer (the secret key) to leak out everywhere.

How about this, without the #[repr(C)]:

struct Foo {
    buffer: [u8, 64]
}

Would the same still hold? Or, does repr(C) add the volatile-ish semantics?

Oh, wait. I didn't see this part. i'm looking for a way to do it while still allowing safe (no use of unsafe) references to Foo, as in C.

The same still holds. Volatile semantics only apply to loads and stores, not the data itself. This is the same in C. If you read the C standard, it only specifies requirements for volatile loads and stores, and nothing about volatile data:

If you want to guarantee that the Rust compiler doesn't leak the key then you need to make sure you never dereference a pointer to the key or create a reference to it. You also need to make sure the key is never located in a local variable managed by Rust. You need to allocate some heap memory to hold the key and only dereference that pointer from assembly code which clears its stack frame and registers when it is done.

Keep in mind that Rust is allowed to perform all these optimizations because the only way that key data can be leaked is through reading uninitialized data (left over from an old stack frame for example), which is undefined behavior. This is less of a problem in Rust than in C because reading uninitialized data is only possible using unsafe code.

[quote="Amanieu, post:26, topic:3188"] This is the same in C. If you read the C standard, it only specifies requirements for volatile loads and stores, and nothing about volatile data:

A “volatile object” is an object that was defined using the volatile modifier. Literally “volatile object” means “volatile data”. The C standard guarantees the above semantics for loads and stores through a pointer-to-volatile that actually points to a volatile object. In particular, if you have:

extern volatile uint8_t buffer[64];

Then the C compiler isn't allowed to add extra loads and stores to buffer.

But, if you have

extern uint8_t buffer2[64];
[…]
uint8_t volatile *p2 = buffer2;

Then even loads and stores through p2 can be optimized if the compiler recognizes that p2 is pointing to a non-volatile object (as it is in this case). The compiler can also add spurious reads and even spurious writes to buffer at any time. For example, the compiler is allowed to touch the buffer in order to prime the memory cache.

While this may be true with a strict interpretation of the C standard (and even then I'm not sure), in practice no compiler will do such a thing. Clang translates accesses to volatile objects into LLVM volatile loads and stores, and does not tag data as volatile in any way. This translation is based on the type of the pointer rather than any data it may refer to, so in your example Clang would never generate spurious loads to the buffer, even if p2 is dereferenced.

See Prevent the compiler from converting "secure" memcmp() into canonical one. by Dmitry-Me · Pull Request #102 · openssl/openssl · GitHub for a counterexample.

I'm not trolling here. My goal is to figure out to what extent I can implement crypto code in Rust vs. C vs assembler. My goal is to make my crypto library ring 100% Rust + my own DSL, but I need a way to have a volatile uint8_t buffer[N] buffer in Rust. It can be of any type, but I'd like to find a type that Rust promises it will never peek into for any reason, in order to avoid side-channel attacks—timing attacks in particular. My concern is that volatile_load and volatile_store don't actually capture all the semantics of volatile in C as there is still no way to declare that a particular field or variable is off limits to the optimizer in safe (doesn't require unsafe) Rust code.

As I said in my previous post, the only way to guarantee that Rust (or C for that matter) will never peek at your data is to allocate it on the heap and only dereference the pointer to it using assembly. If you ever read data from the buffer from Rust or C then there is no way to prevent the compiler from leaving any data it has read on the stack or in registers after the function returns. Writing to the buffer (without reading any of it) might be doable but you might as well write that in assembly too so that you are sure no data is leaking.

Playing tricks with volatile or optimization does not provide any guarantee that the compiler will not break this code in the future. Fundamentally, the problem is that data has become inaccessible from the language's point of view (gone out of scope) and the only way to see that data again is by triggering undefined behavior. Compilers are fully within their rights to assume undefined behavior never happens, which makes any attempt to work around this within the Rust/C language futile in the long run.

Slightly off-topic, but for that specific case you can make the compiler "forget" where a pointer came from by using inline assembly: Compiler Explorer

You can work around the unsafe issue with a wrapper type that exposes a safe interface around volatile_{load, store}.

Regardless of what the compiler does, technically there is no guarantee that the malloc implementation itself doesn’t go around randomly reading contents of previously allocated buffers just to annoy you, so I suggest going as low level as possible and using mmap or platform equivalents. This has the advantage of letting you use mlock to prevent paging, if you want…

I suspect that whatever the C standard says, LLVM is never going to do any optimizations on volatile accesses in IR - indeed, it looks like GCC’s similar behavior is basically a bug. But if for some crazy reason it someday tries to, using mmap should also ensure it can’t do so in your case, because it wouldn’t be able to detect you’re not writing to a “real” volatile object - whatever that means outside of C-land.

However, Amanieu is completely right that the compiled code is likely to leave data lying around in registers and/or the stack; not to mention that there’s no guarantee the optimizer won’t change constant-time code to non-constant-time. Not sure if it’s still maintained, but have you seen nadeko?

If Rust doesn't allow spurious reads of volatile memory, then it must not link executables that use a malloc implementation that works like this.

Anyway, I did some research and I learned more about how LLVM works. Basically, AFAICT, LLVM doesn't keep track of which objects are volatile and which ones aren't. Instead, it just doesn't touch anything unless it is touched by a non-volatile load or non-volatile store. But, once a non-volatile load or non-volatile store is seen, it feels free to start optimizing those loads/stores which might result in a load/store getting moved to a different point in the program than it appears in the program's source.

Now, the question is whether this is guaranteed by the Rust language, or whether it is an implementation-specific behavior of rustc. In the case of Clang, we can see that the above behavior is not guaranteed by the C11 language spec; that is, Clang's behavior is much more conservative than what the C language spec requires.

Now, here's the thing I find strange: A huge part of the value proposition of Rust is that it uses its type system to prevent common types of bugs in error-prone aspects of programming. It seems to me that a useful constraint is "this memory must only be accessed via volatile loads and stores, not normal loads and stores"; that is, there should be some way to use types to force the exclusive use of volatile reads and writes and/or to disable the * dereferencing operator for volatile objects in favor of explicit volatile_load, volatile_store, nonvolatile_load, and nonvolatile_store methods. Yet, the current volatile load/store proposal does not take advantage of Rust's type system to help programmers avoid mistakes at all. This doesn't seem “rustic“ to me.

Interestingly, I think one could easily use the volatile_load and volatile_store intrinsics and the rest of the features of Rust's type system to create such a safe interface to volatile memory in libstd similar to UnsafeCell. In fact, I have to say I'm not even sure if that is what UnsafeCell is supposed to be; it seems like UnsafeCell is something similar but different, as it doesn't use volatile_load and volatile_store internally.

I'll try to put together such a thing and see if it makes sense. If somebody has already done so, or knows why it's a bad idea, I'd love to hear about it.

Spurious reads are irrelevant for what you are trying to do. If at any point you read from the buffer in Rust (or C) code then the compiler is allowed (and very likely will) leave copies of the data that was read on the stack or in registers after the function returns. This is not a spurious read, it's just that the compiler isn't clearing the local variable that you read the buffer into. The only way to avoid this is to only dereference the pointer to the buffer from assembly code.

Any data "leaked" from the buffer can only be recovered by reading uninitialized data, which is undefined behavior. Rust guarantees that undefined behavior is not possible in safe code.

You can easily create a wrapper type that always uses volatile loads and stores to access an inner value, but that still won't solve the problem I mentioned above. I'll repeat it again: volatile is only useful for memory-mapped I/O.

Zinc has VolatileCell. However note that it's only mean to be used for memory-mapped I/O registers.

I am trying to understand (and make sure it is defined) how volatile loads and stores work in the memory model. Are you sure spurious reads are irrelevant to that?

As far as the crypto stuff I'm working on is concerned, this doesn't matter. I'm not trying to defend against Heartbleed-style attacks. I'm more trying to defend against side channel (timing) attacks. In that sense, a spurious read isn't a problem exactly, but a spurious if buffer[n] == 0 { ... } else { ... } would be a problem.

This is not true, because of side channels, especially timing. I'm not concerned about the “read of uninitialized data” scenerio; that is what Rust's type system protects against.

Based on what you said above, I don't think one can do so using the current primitives and the current (lack of) guarantees, if you want to have a safe (doesn't require users of the wrapper to use unsafe) API.

But my question is: Why doesn't libcore provide that wrapper, instead of the less-safe volatile_load and volatile_store? I.e. why is the standard library choosing to provide an error-prone API instead of a “rustic” API?

So, in C, one can use volatile arrays to do what I want w.r.t. preventing loads/stores really easily. I understand that in Rust one can't do that because Rust doesn't have any concept of a volatile object like C does. My interest isn't in making Rust be like C—though I think comparing to C is helpful since C's memory model is currently better defined—but rather I'm just looking for a way to have some data that Rust won't leak information (through side channels) about the contents of, while still allowing Rust code to carry around the data without needing to use unsafe. I understand that Rust's memory model currently doesn't support this. But, how can we fix that? I understand you don't want to borrow C's volatile to fix it, but what would you be willing to do instead?

Yes, that's exactly what I think Rust's interface to volatile memory should be like. However, according to what you wrote above, Zinc's VolatileCell isn't guaranteed to work correctly, right? In particular, it doesn't store value on the heap in memory that is only manipulated by assembly language code, so rustc might insert spurious reads and spurious writes, right?

If you create a reference (that’s it, &T) and access it, then the compiler can read from the reference for its entire lifetime when it wants to. The story with &mut is quite similar, but these references can be invalidated pretty easily - I am still not sure what a reasonable invalidation semantics would be.

This rule does not apply to references that are immediately coerced to a raw pointer (e.g. &foo[0].bar: *const _ does not allow the compiler to create accesses). It is still not clear how to make this work, but I suspect that even a purely syntactical rule could work.

Currently, we don’t really have a way of allocating “some memory” behind a raw pointer on the stack. @ubsan thinks that he’s got some rules, but we didn’t agree on them. We should eventually come up with something, but there’s no agreement yet. You can mmap and use the returned raw pointer.

I suppose that doing

    let mut buf: [u8; 16] = unsafe { mem::uninitialized() };
    let buf_ptr: *mut [u8; 16] = &mut buf;

should be safe, but we still didn’t decide on the rules.

Because it is not the point of libcore is not to include wrappers for every programming pattern on Earth. While libcore does include a fair bit of lang-items, a volatile memory wrapper as you suggest would not be one. There's no particularly convincing reason for it to be in libstd rather than in any other crate.

With things being such as they are, if we designed something ourselves and put it in libstd, a few people would go and complain that our wrapper does the Wrong Thing.

There is nothing inherently wrong with needing unsafe code to support some functionality not included in libstd.

OK, let me ask the most basic question: Does everybody agree that Rust guarantees that zinc’s VolatileCell is a correct implementation of volatile memory? From reading various things above, there doesn’t seem to be consensus that it is correct.

Here’s the code again: