Volatile and sensitive memory

language design
81

That the standard does not go out of its way to state this is exactly my point. The standard does not go into detail when it believes that pointers plucked out of thin air are things that it "owns", except when these pointers were casted from integers that also came from a priori valid pointer, or when that cast is a null pointer constant. There is some muttering about "implementation defined", i suppose, which merely makes this marginally better.

In practice, yes, the compiler proving anything like this would never happen, but the point I was making (which others in this thread agree on) is that Rust should take care to not repeat this specification mistake. After all, something like miri for C's abstract machine could prove such facts.

1 Like
82

That the standard does not go out of its way to state this is exactly my point.

Without knowing which parts of the C standard caused your misconception, it is hard to identify how to do better in Rust. The C standard does not mention anything about the global observability at the root of the misconception, so it is hard to tell where it comes from.

In practice, yes, the compiler proving anything like this would never happen, but the point I was making (which others in this thread agree on) is that Rust should take care to not repeat this specification mistake.

The proposed validity invariant for raw pointers just tells you that all bit-patterns of a raw pointer representation are valid and distinct values, and that has the same "problem" as C. There are dozens of ways to construct a pointer (transmute, union type punning, extern static, an FFI call, casting an usize, inline assembly, ...) and the how does not really matter from the validity point-of-view at least.

Your complain seems to be that the C standard does not have explicitly written anywhere that casting an integer to a pointer is ok. In Rust the validity invariant says that constructing a pointer in any way from any bit-pattern (modulo maybe uninitialized) is ok, but this is apparently not clear enough since C does this as well when it talks about value representations (e.g., intr * ptr; is ok, and constructs a pointer using uninitialized memory, which is kind of "worse" than constructing a pointer from an actual integer value).

I am not convinced that actually naming all programmatic ways in which a pointer can be constructed in the language spec is useful, and it does introduce many problems (e.g. we probably don't want to talk about how linking precisely happens in the spec). For the particular case of the usize as ptr, we have to talk about that when we specify the semantics of as, so we can maybe just link to the validity invariant of raw pointers at that point, which basically says that the cast works fine for any usize value, but this should be clear for users that understand validity already.

1 Like
83

C does not go out of its way to say under what circumstances it considers memory as being initialized beyond some "implementation defined" handwaving. That is the problem I'm referring to. It is not a misconception on my part, but rather a potential (admittedly unreasonable, but that is how such things tend to be) reading of the ISO standard. Rust should go out of its way to be very clear about when it considers "external memory" to "not be its problem".

This isn't about creating pointers and ignoring them; C is perfectly happy for any well-aligned address to be turned into a pointer. The issue is about when an lvalue can be created out of that pointer, and whether that lvalue can be coerced into a value. Creating a wild pointer is completely valid, but dereferencing it is not (since the pointer's bits are uninitialized).

I'm not sure I agree with this. I think that "creating this type of pointer is ok by omission" should not be how you deduce that "forging and using a pointer pointing at some MMIO device is ok". Whether this requires a complete catalogue of all ways to create pointers to initialized memory is not a question I have sufficient background to answer, though.

(Separately, I don't buy that pretending the linker does not exist is even a good idea... C++ goes to extreme, nonsensical lengths to do this, resulting in very confusing one-definition rules.)

1 Like
84

(That's not correct, even creating a dangling pointer without using it is UB in C -- at least if you do it via ptr arithmetic, I am less sure about casts but I think it is the same.) But we are digressing very far from volatile semantics, could we get back to that please?

85

@RalfJung Notice that in this particular example the safety invariant on the pointer arithmetic operation is violated, and therefore, such programs have UB before the result of the arithmetic operations attempts to materialize a new pointer, and we therefore do not even guarantee that a pointer is produced at all.

@mcy

C does not go out of its way to say under what circumstances it considers memory as being initialized beyond some "implementation defined" handwaving.

C11 6.7.9.10 (initialization) is quite specific, and so is C11 J.2.1 (undefined behavior).

The issue is about when an lvalue can be created out of that pointer, and whether that lvalue can be coerced into a value. Creating a wild pointer is completely valid, but dereferencing it is not (since the pointer's bits are uninitialized).

C11 6.5.3.2.4 (address and indirection operators) is quite specific about that as well.

I think that "creating this type of pointer is ok by omission"

The C standard allows creating pointers from any bit-pattern (including uninitialized memory), and the different C operations on pointers have different pre-conditions on their arguments, e.g., * requires the pointer to point to some memory (not be null or uninitialized), and for that memory to be a valid allocation, or else the behavior is undefined.

Rust specifies things with a similar structure as C: it specifies some general properties about pointers (layout, validity, safety, etc.), and then the different operations have extra safety requirements (e.g. ptr::read() requirements are in broad strokes very similar to C's *: if you want to read from a pointer, that pointer better be "readable").

Neither C nor Rust mention in the description of these operations the things that do not matter for their correctness (there is an infinite amount of such things, e.g., whether the allocation was performed via a Rust API, or whether today is Tuesday). From a spec point-of-view, I don't think mentioning these things is strictly necessary, although it is sometimes nice to have footnotes about this (we should try to make the Rust spec more "approachable" than C's, so that user can actually read it). There should definitely be some text introducing users to how memory works in Rust, and this should be definitely covered there. Maybe it might even be worth covering in the API docs of the raw pointer module?

1 Like
86

6.7.9p10 merely specifies that such memory (of automatic storage duration, which I believe is what we're referring to) is "indeterminate", which is a flavor of "implementation defined" handwaving. It is only specific when referring to static or thread_local allocations, which are not what we're talking about. Regarding Annex J, which one of those requirements are you referring to?

6.5.3.2p4 asks whether you know that the pointer points to an object (and that the pointer has compatible type if that object has a type). I believe that this just points back to the issue where it is unclear when a region of the address space begins pointing at an object, other than initialization modes known to C.

I think I'm a little worn out on arguing about this though, and after a lot of thinking I'm finding it harder to state what I actually want unambiguously... so I propose tabling this for now?

87

indeterminate is just another value that every memory locations can have in C (just like in Rust "uninitialized" as a value is a thing): uninitialized automatic variables, padding, pointers to deallocated objects, etc. all have indeterminate values. J2.1 says that it is UB to use those:

The behavior is undefined in the following circumstances: [...] The value of an object with automatic storage duration is used while it is indeterminate

This is something that holds for all C programs and is not implementation-defined.

I believe that this just points back to the issue where it is unclear when a region of the address space begins pointing at an object, other than initialization modes known to C.

Gotcha. I don't know if this helps, but a C object is defined as a memory region that can be used to store values. If one materializes a pointer to a memory location that can be read, that pointer always points to a C object. For example, for MMIO, the OS maps some register into the process address space for the whole duration of the program, so the memory region in which that register is mapped is a C object with a static lifetime. The value actually stored in that object doesn't matter much, since even if it is indeterminate, e.g., because the region is uninitialized, one is allowed to at least read it.

The C standard does not specify the behavior of reading from a pointer that does not refer to a C object. A pointer that does not refer to a C object does not refer to a data region that can store values, so I can't imagine a case where such a read might be a meaningful thing.

Materializing a pointer out of thin air in both C and Rust isn't really that different from extern declarations. For example, with a extern { static FOO: i32; } the programmer is stating that there is a memory region with static lifetime at the address FOO that's always readable. In C, if this is not the case, the behavior is actually instantaneously undefined (in Rust, however, extern declarations are safe). When one creates a pointer from an integer and reads it, the programmer is making a very similar assertion, but with a different lifetime since it suffices for that memory location to be live when one attempts to read it. Does this make sense?

3 Likes
88

The understanding I've always taken in C/C++ is that forging pointers that C/C++ didn't initialize is UB, but in practice you either don't care or you pass --im-a-kernel-author-i-know-what-im-doing-thanks or whatever.

You are correct, forging pointers is always UB. When I deal with fancy memory addresses, I just write a linker script that sets some symbol to that fancy address (probably wreacking more interesting things like odr, but eh). Arguably rust should treat forged pointers the same way (if they didn't come out of something legit, they should be UB to dereference).

But in my interpretation of C/C++, doing so is legal. At least I have yet to have someone point me at a part of the spec that would forbid this.

So the exact requirements comes down to a few parts. First, pointers are scalars (and definately not narrow character types), so you can't read a pointer with an indeterminate value. 2nd, for pointers produced by casting from an integer/integral type (using a C style cast, or in C++, a reinteperet_cast), the pointer is only valid to dereference if the integer value was obtained by casting a pointer, without any truncation between the initial cast and the return cast. (There are other requirements, such as strict aliasing, and alignment)

1 Like
89

How would you imagine to specify MMIO where writing to the MMIO address triggers a DMA transfer? Which is, in a sense, a weird call to some sort of memcopy style function.

90

An "indeterminate value" is the C and C++ spec's wording for, roughly, undefined bytes, like the contents of an uninitialized variable. Casting an integer to a pointer does not produce an indeterminate value.

Those are the conditions under which the C++ spec guarantees that a pointer will be valid to dereference. But it leaves other cases implementation-defined, not undefined. To quote C++17:

  1. A value of integral type or enumeration type can be explicitly converted to a pointer. A pointer converted to an integer of sufficient size (if any such exists on the implementation) and back to the same pointer type will have its original value; mappings between pointers and integers are otherwise implementation-defined. [ Note: Except as described in 6.7.4.3, the result of such a conversion will not be a safely-derived pointer value. —endnote]

(The note about "safely-derived pointer values" sounds ominous, but C++ "pointer safety" is just an extension to support garbage-collected implementations; real implementations typically have get_pointer_safety() == pointer_safety::relaxed.)

Or to quote the equivalent clause from the C spec:

  1. An integer may be converted to any pointer type. Except as previously specified, the result is implementation-defined, might not be correctly aligned, might not point to an entity of the referenced type, and might be a trap representation. [67]

[67] The mapping functions for converting a pointer to an integer or an integer to a pointer are intended to be consistent with the addressing structure of the execution environment.

Again, implementation defined.

On most implementations, it's perfectly valid to write an MMIO register address as an integer and then cast it to a pointer, e.g.

volatile int *x = (volatile int *)0xdeadf000;

There are exceptions, like CHERI, where the hardware prevents you from forging pointers in general.

2 Likes
91

You mean, because that's basically reading some other unrelated memory? Yeah, that's... weird.

For this code to be correct, the programmer needs to put in suitable fences, so I suppose the spec will be similar to concurrency or signal handlers and require fences to avoid UB due to "racy accesses".

92

It's pretty normal for a DMA as far as I know. The workflow is something like:

  • Write source into one address
  • Write destination into another address
  • Write activation settings into a third address (word count to copy and such)
  • The DMA notices and kicks in (which might or might not halt the CPU while it happens, depending on hardware).

Any memory model suitable for Rust in the long term has to support this (possibly modeling it as a spawned thread, or similar).

2 Likes
93

The other common DMA pattern is to have some kind of Ring Buffer or Linked List that is shared with the Hardware where you place descriptors for DMA buffers which contain the pointer to the buffer as a physical address and a length of that buffer and then the hardware will take entries off of this when it needs to use them (for example RX for ethernet) or once you place them there it will start processing them (TX for ethernet).

For GPUs there is usually a ring buffer which is processed by the command processor which can have various structured commands which may include references to other buffers to process or buffers containing data needed for the commands.

In these cases you have memory which is shared with the hardware in a sort of ownership transfer fashion that has strict rules, usually once you have given memory to the hardware it is not yours until the hardware indicates it is done.

3 Likes
94

You didn't mention any fence. I cannot see how this can possibly be sound without a fence, unless all involved writes (including the data writes) are volatile. Did you omit some details or do you think this should be sound without a fence? (I was told by hardware people that requiring a fence in such a situation is expected and acceptable.)

95

You didn't mention any fence.

Ah, yes, you probably need a fence too I suppose.

I cannot see how this can possibly be sound without a fence, unless all involved writes (including the data writes) are volatile.

I don't know what "including the data writes" means here. Volatile is about telling the compiler "you really do have to do that, just do it", and I can't imagine that the DMA unit would, say, copy 100 words but skip the 56th one because it somehow decides that it didn't need to do that one. In that sense, I suppose I would say that all modifications that the DMA unit makes should be considered to be volatile. If I follow your meaning.

Did you omit some details or do you think this should be sound without a fence?

Yes, sorry! The device I normally use that has a DMA unit is old enough (manufactured 2001) that there's no out-of-order execution, no memory cache, and no parallelism. Accordingly, it has no fencing. Sometimes I do end up forgetting the additional requirements that a newer device would have to follow.

96

Note that fences may be needed to prevent compiler reorderings too...

97

Yeah, even if there’s no fencing required at the assembly level, at the source level you definitely need a compiler fence between the writes into the buffer (assuming they are non-volatile) and the volatile writes that kick off the DMA. This is true in C as well.

1 Like
98

I was referring to the (presumably) usually non-volatile writes for the actual data to be transferred. By naive understanding of this process is that it goes something like:

  • normal writes for the data (e.g., the network package to send)
  • fence
  • volatile write putting the address of the data into some MMIO register

As others said, this sounds insufficient. Specifically, the compiler is allowed to reorder the following two lines, if it can show that x and y do not alias:

// x: *mut i32, y: *mut i32
x.write(42);
y.write_volatile(13);
1 Like
99

In othe words, without a fence the compiler is permitted to start the DMA before putting the data in the DMA-associated buffer.

1 Like
100

Oh I see what you're saying. Yeah I'd never have written that code. The DMA in my case is used for ROM to video / sound memory. Never even occurred to me that you'd try and DMA to normal rust memory.

Does core::sync::atomic::compiler_fence(Ordering::SeqCst) count as a sufficient memory barrier for this?

1 Like