Two Kinds of Invariants: Safety and Validity

[RalfJung]

@CAD97

(Under this model,) It is valid to have an &mut T to a partially initialized T , but it is not safe to do so (as safe code could use the uninitialized part of the value).

That is what I am proposing, yes. The terminology of validity vs. safety I think is useful independent of what we decide for references, (a) to be able to even have this discussion and (b) because there will be other differences between validity and safety.

@CAD97

I like that this distinction of safety/validity also creates an actual reasoning behind where to end a unsafe {} block. In a safe function, all types must invariantly be safe. In an unsafe function or unsafe block, types must be valid but may temporarily be not safe. By the time the unsafe block/fn ends, however, everything must be safe again.

I do not think we want the extent of the unsafe block to have any direct consequences.

However, I think this distinction is useful to define what unsafe code is. It is essentially code that works with unsafe data (data that is valid, but not safe). This goes back to what I wrote two years ago about how far out unsafe "leaks".

@CAD97

How does auto-deref interact with safety/validity? Would it just be a (safe) use (thus be part of safety), or since the compiler does it automatically, part of validity? Since Deref can be user implemented, I suppose it can’t really be part of validity…

All of these discussions, as far as I am concerned, are happening on the MIR -- so autoderef is just a method call like any other.

@CAD97

Would it make sense to introduce optimizations in safe code only that rely on safety of references additionally to validity? If I’m not mistaken, that would allow applying the un-ref optimization to safe functions but not unsafe functions.

See what I wrote above about it being hard to define where unsafe code ends.

@ExpHP

the boundaries to safe code are not the end of an unsafe block or an unsafe fn . They are at the end of safe, public fn s.

Yes! Maybe I should clarify in the post

@ExpHP

True, and calling a function with mismatched signature is certainly UB, there is no other option. However, just holding a function pointer with bad signature that is not being used – should that already be UB? That is what validity of the fn type is about.

Is there some sort of optimization that could be made possible by making this UB? If not, it seems silly to make it UB…

I cannot think of any.

@earthengine

I am glad to see that some of my previous ideas in comments on @nikomatsakis 's last blog post was presented in this blog. Specifically, the difference between ! and &! .

I have been sitting on these thoughts for quite a while and wished I had them already spelled out for that discussion -- but then, that discussion certainly helped me to sharpen my thoughts and make them more precise.

@earthengine

means (u32,!) is like ! , it can never be valid.

Agreed.

Notice, however, that I glossed over one issue in that post: When I said all data must always be valid, I was only referring to data the compiler considers initialized. In

let x : &'static i32;

Certainly x must not be valid.

@CAD97

I don’t think it’s much of a burden to require arguments to a safe function to be in a safe state.

It is quite a burden. Unsafe code does things like calling size_of_val on uninitialized references -- I found an instance of this in Arc::drop_slow and I am sure I would have found more if I would have went on looking. See the PR implementing MaybeUninit for a whole bunch of examples where passing &mut T pointing to uninitialized data to known safe functions is very useful.

I am a fan of saying that a safe function must only be called with safe data, ever -- but first of all we still cannot make violations of this UB (because safety cannot be checked), and secondly this is surprising to lot of people.

@earthengine

But I would expect that, until in a static analysis shows that a value t is fully initialized, we should not allow it being sent to or return from other functions, even in unsafe code.

That is totally the case. You can only move a value if the compiler thinks it is initialized, which means at this point is must be valid or else you have UB.

@earthengine

If you want to write a function that receives uninhabited values, use indirection like &! .

If you want to write a function that receives uninhabited values, stop what you are doing.

If you want to write a function that receives uninitialized values, e.g. to initialize them, use &mut T. This will usually be generic code. (That is assuming we adapt my proposal for validity, so your &mut T must be valid but that does not require you to find a valid T.)