Pre-RFC: Unsafe reasons

Summary

Add a way to check at compile time that all unsafe assumptions has been met.

Motivation

Today, unsafe code is clearly separated in Rust, which is a good thing. However, when writing unsafe code, one cannot easily verify they do not forget some assumption that is required for the code to not be undefined behavior. The only way to check assumptions is the documentation of the unsafe functions you call, but while this is always necessary you can still easily forget some condition.

Worse, if a function's preconditions are updated (for example, in a breaking dependency update), your code will still compile, even though it may not meet the new preconditions. And you might get undefined behavior at runtime, which we by all means want to prevent.

Guide-level explanation

When writing unsafe functions, the unsafe keyword can optionally accept a list of reasons. They are enclosed in parentheses, like the following:

unsafe(reason1, reason2) fn my_unsafe_fn() { /* ... */ }

When calling unsafe functions, you can optionally supply a list of reasons. The list of reasons must match the list on the function's definition. For example, you can call the above function like the following:

unsafe {
    my_unsafe_fn().unsafe(reason1, reason2);
}

If you miss a reason, the compiler produces an error:

error[E0000]: unsafe reason `reason2` does not appear in the call
 --> src/main.rs:4:5
  |
4 |     my_unsafe_fn().unsafe(reason1);
  |                          ^^^^^^^^^ provided reasons list
  |
  = note: missing unsafe reason may mean you forgot to handle a precondition of the function
  = note: consult the function's documentation for information on how to avoid undefined behavior

You can continue declaring and calling unsafe functions without reasons. If you have an unsafe function with reasons that you call without reasons, the compiler won't complain (it might lint/error in a future edition).

Unsafe reasons are supposed to help you make sure you don't forget any precondition. An unsafe function should include all preconditions it has in its reasons list. For example, the following function:

pub unsafe fn to_bool(v: u8) -> bool {
    unsafe { std::mem::transmute(v) }
}

Should be written like the following (of course you can change the reason name):

pub unsafe(v_is_0_or_1) fn to_bool(v: u8) -> bool {
    unsafe { std::mem::transmute(v) }
}

Of course, transmute() has preconditions - and therefore eventually will include reasons - too, so if you want to be more safe you can specify them (imaginary reasons, see unresolved questions):

pub unsafe(v_is_0_or_1) fn to_bool(v: u8) -> bool {
    // SAFETY: `u8` doesn't have uninit bytes, and the only invariant of `bool` (besides no uninit)
    // is that it's 0 or 1, which is our precondition.
    unsafe { std::mem::transmute(v).unsafe(no_unmatching_uninit, invariants_kept) }
}

For maximal safety, the list of reasons on function declaration should closely match the unsafe function's safety documentation, and the justifications for the reasons on calls should be explained in the // SAFETY comment.

The compiler provides an optional lint, unsafe_without_reasons, that will be fired on calls to unsafe functions without reasons provided, if the function has reasons in its declaration. The lint is allow by default (but this may change in an edition).

Reference-level explanation

The syntax of function declarations is extended like the following:

   Function :
      FunctionQualifiers fn IDENTIFIER GenericParams?
         ( FunctionParameters? )
         FunctionReturnType? WhereClause?
         ( BlockExpression | ; )
   
   FunctionQualifiers :
      const? async? ItemSafety? (extern Abi?)?
   
   ItemSafety :
-     safe | unsafe
+     safe | unsafe DeclarationUnsafeReasons?

+  DeclarationUnsafeReasons :
+     '(' IDENTIFIER ( , IDENTIFIER )* ')'

Call syntax is extended like the following:

+  CallUnsafeReasons :
+     '.' 'unsafe' '(' IDENTIFIER ( , IDENTIFIER )* ')'

   CallExpression :
-     Expression ( CallParams? )
+     Expression ( CallParams? ) CallUnsafeReasons?

   MethodCallExpression :
-     Expression . PathExprSegment ( CallParams? )
+     Expression . PathExprSegment ( CallParams? ) CallUnsafeReasons?

When calling a not-unsafe function, it is an error to provide unsafe reasons.

When calling unsafe functions:

  • If the function does not have reasons declared, calling with reasons will cause an error. This is done so adding reasons to an existing function that was without reasons won't be a breaking change.
  • If the function does have reasons, and there is a reason from the declaration that is absent in the call, the compiler will emit an error.
  • If there are unnecessary reasons, either in declaration or in call (duplicate or reasons that are not declared in the function declaration), the compiler won't err but will lint with the new lint unnecessary_unsafe_reasons.
  • If the function is declared with reasons but no reasons are provided in the call, the compiler will emit an allow-by-default lint unsafe_without_reasons. Users that want to make sure they have reasons coverage can turn this lint on. This is done so that adding reasons to a function that didn't have reasons won't be a breaking change. In a future edition this lint might be promoted.

Despite the fact that the reasons call syntax includes the unsafe keyword, it does not replace the role of an unsafe block. Calling an unsafe function without unsafe block will remain an error, even if reasons are provided (but see Rationale and alternatives).

The type for an unsafe function with reasons (its FnDef) contains the reasons (so the compiler can track them where it's called), but function pointers do not have reasons. This is based on the assumption that if someone does something that involves multiple unsafe functions (otherwise they wouldn't need fn pointers), their reasons can be different enough that it will be hard for the compiler to unify them, and the programmer already has greater scrutiny on such "general (or somewhat general) precondition" calls. So when coercing FnDefs to fn pointers to unsafe reasons are lost.

Drawbacks

The first drawback is always that it complicates the language.

The second is that this makes the syntax for calling unsafe functions more heavy. While reasons are optional, we do encourage people to put reasons in their unsafe code. For that reason I believe that while reason names need to be clear, they also need to be short and it's okay if what a reason means is only clear for the documentation: as said above users are encouraged to accompany reasons with // SAFETY comments. Reasons are mostly there to check preconditions weren't forgotten.

Rationale and alternatives

The most important question when it comes to alternative is where we put the reasons in the call. This RFC proposes to put them in the call, but an alternative is to put them on the unsafe block (unsafe(reason1, reason2) { ... }). This the major advantage that this is less heavy, particularly because it does not require one to repeat the unsafe keyword and attach the reasons to each call. However, this also has major disadvantages: first, some users like to have big unsafe blocks that cover the entire area that is infected by them (including safe code that the unsafe code relies on), and for them putting the reasons on the unsafe block will mean it'll be far from the call. Second, when you make unsafe blocks that contain multiple calls it may not be clear which reason belongs to which call. But the worst thing is that if we take this approach, adding reasons to a function will become a breaking change, because the user may already have reasons for other calls in the same unsafe block. This will mean libraries will be hesitant to adopt unsafe reasons, and even libstd will be forced to do it at the same time for all functions in libstd at the same time as stabilization of unsafe reasons, or never do it.

Another alternative is taking the opposite direction, and make call with reasons to not need an unsafe block. I'm avoiding this change because the community has been split in the past about whether adding more fine-grained unsafe control is a good idea, but it's definitely something to consider.

Of course there is also the alternative of doing nothing, but I believe the advantages for unsafe code writers outweigh the disadvantages, especially since unsafe code is already niche and should already have greater scrutiny. This also cannot be done by a macro due to the need of semantic information about the called function.

We can make fn pointers type with unsafe reasons (that coerces to one without). The reason I do not propose this, as outlined in the Reference-level explanation, is that I assume that when people use unsafe fn pointers their preconditions are different enough that the compiler won't be able to unify them.

Prior art

There are few languages that have a safe/unsafe model (e.g. C#), but I'm not aware of any language that implements the proposed feature.

Inside the Rust community I believe this idea was thrown somewhen, but I don't think it was discussed a lot.

A somewhat related discussion is whether unsafe should be more granular - that was discussed e.g. here and here.

Unresolved questions

What should be the syntax for reasons on call?

How precise should reasons be? This applies both as a general advice and to stdlib functions. For example, the documentation for std::ptr::read() requires that src is properly aligned and points to a properly initialized value, is not null, is dereferenceable, and is non-racing. Should they all be separate reasons, or should we group some, or perhaps omit some uncommon ones (e.g. that the read is non-racing)?

Should we have a lint for functions declared without unsafe reasons? Some users may want to make sure they don't forget to add them, but on the other hand for some (like FFI, where everything is unsafe) reasons don't help much. It can be an allow-by-default-lint, though.

Future possibilities

One future possibility mentioned above is to make the unsafe_without_reasons lint warn-by-default (or stronger) in an edition.

4 Likes

Seems like an interesting idea to flexibly capture invariant changes at compile time, thanks for writing it up!

I appreciate making the underlying assumptions more explicit by acknowledging the safety pre-conditions.

Another area to consider, possibly as a future possibility (i.e. ensuring this design doesn't preclude it) is when unsafe code relies on assumptions in safe code. When a crate of mine was getting a safety review, they proposed both SAFETY comments that specify how it depends on invariants of safe code and Safety-usable invariant comments on my safe code. These two-sided comments I felt were at higher risk of going stale or being left out because there wasn't an unsafe involved on the other side. Being able to declare these relationships would be a big help.

In writing this, I've used words like "precondition" and "invariant" and this is making me wonder if we should consider a form of contracts for this (yeah, probably opening a can of worms there). Recently, the standard library gained the ability to run unsafe-related asserts on debug builds. Imagine if those asserts were broken out into functions that the caller had to acknowledge when declared as an unsafe pre-condition but could also be attached to other items and used in other ways.

@epage While I appreciate these ideas, and would love for them to be applied, I specifically designed this proposal to be minimalist. There are many ways to validate unsafe code, ranging from debug assertions to formal verification. If we will search for the best possible outcome, we may get nothing in a never-ending search, which is arguably the worst outcome. So unless there are concrete plans for how such contracts system will work, I prefer to stay on the minimalist side.

At work we are already trying to achieve something like that with const assertions :face_with_hand_over_mouth:

That sounds interesting - care to share how and how it goes?

I've seen such safe code being called "robust" here in IRLO, but never once in another place. But I would love for this to be a thing, so I always repeat this concept in the hopes this catch on.

Here is an example usage: Allocator::allocate is safe, but it also provides guarantees that unsafe code can rely upon, so it is robust. Allocator::deallocate is unsafe, because it requires guarantees from the caller (some which a previous call to allocate can help to fulfill).

1 Like

From the tail end of your comment, I'm assuming you are only referring to my comment on contracts, especially since the other part (safety-usable invariants) I suggested as a future possibility so we don't preclude it.

A form of contracts should probably be called out in the Alternatives.

I can understand going with a minimal solution but we also should consider whether the language as a whole is expected to go in a related direction and see if we can align efforts so we don't add one off features that don't fit in with future plans. I can't say whether a form of contracts is of interest to T-lang but attention should at least be drawn to it as an alternative and maybe talked towards interacting with in a future possibility.

1 Like

Fair enough.

Nothing too special, we have boolean consts with different guarantees, and assert them before using unsafe (sometimes).

It emerged because of our bindings to C rtos and, with user-privided header files, bindgen generates constants, and we can't have certain rust APIs if user will, for example, allow timer callbacks to be preempted - which would be observable from C constants at compile time.

Such system is too simplistic to model this proposal; it can only model conditions known at compile time. On the other hand, it is better when can be used, because it guarantees working code and this proposal only "guarantees" more scrutiny.

You are right.

I am wondering if

unsafe(reason1, reason2) fn my_unsafe_fn() { /* ... */ }

unsafe {
    my_unsafe_fn().unsafe(reason1, reason2);
}

is equivalent to

unsafe fn my_unsafe_fn<const REASON_ONE: bool, const REASON_TWO: bool>() {
    const {
        assert!(REASON_ONE);
        assert!(REASON_TWO);
    };
} ;

unsafe {
    /// Blah blah reason one
    const REASON_ONE: bool = true;
    /// Blah blah reason two
    const REASON_TWO bool = true;

    my_unsafe_fn<REASON_ONE, REASON_TWO>();
}

(This is just an idea)

Kind of (and this is what I originally thought you meant), except unsafe reasons do not need to monomorphize and also have a clearer (IMO) syntax. Oh, and errors from them show in cargo check, not just cargo build, and are not cryptic.

In regards of post-monomorphization errors, I guess next year it will be possible to have custom types in const generics, so accept zst instead of bool.

Or even just accept a zst in arguments.

unsafe fn my_unsafe_fn(_: ReasonOne, _: ReasonTwo) {} ;

unsafe {
    /// Blah blah reason one
    let reason_one = ResonOne;
    /// Blah blah reason two
    let reason_two = ReasonTwo;

    my_unsafe_fn(reason_one, reason_one)
}

// Note: Instead of type for different reason, 
// there might be just one type `Reason`. And 
// there also might be `const fn` constructor, it 
// might even take `&str`, and even might be 
// `unsafe` itself. It will even remove the need.
// for `my_unsafe_fn` to be `unsafe`, as it can't 
// be called without calling other unsafe 
// function constructor.

Oh, it may be possible to have a helper trait:

trait True {}
struct Proof<const P: bool>;
impl True for Proof<true> {}

unsafe fn my_unsafe_fn<const REASON_ONE: bool, const REASON_TWO: bool>() 
  where Proof<REASON_ONE>: True, Proof<REASON_TWO>: True 
{}

Another benefit of constants, is that documentation can be attached to them, so rustdoc and rust-analyzer will show it.

Okay, arguably this is a strong alternative. There are two reasons why I think my proposal is still worthwhile: a) verbosity. This way requires generating at least one struct for each function, and may be alien to developers, while my proposal is more natural and less verbose. b) backwards compatibility. With your idea we cannot allow checking stdlib functions, for example, or even just other crates without a very disrupting breaking change, as adding parameters is definitely a breaking change.

(NOT A CONTRIBUTION)

Yes. When you play with this idea, you realize you start to want functions which are generic over invariants, you want invariants which can be discharged no more than once, etc, and quickly you realize that you've just reinvented the capabilities of the type system.

Of course, there are reasons of annotational burden why unsafe invariants are not expressed as ZSTs usually, and its possible to imagine syntactic sugar which makes tracking invariants with the type system more desirable.

3 Likes

If we want to be the most general possible, yes. But if we want to cover 90% of the cases (maybe more) with minimal disruption, my proposal makes more sense. Of course you can call that "a syntax sugar for the type system", but this looks more like something of its own. Anyway I will add this to alternatives.

TBH, I think needing them in the middle of the call like that will defacto result in people being unwilling to actually list out the reasons in nice separable ways, because it'll just be so ugly to see them all through the middle of the logic.

I do like the direction of having to meet all the requirements -- not just the ones you remembered -- but think it should work differently so while you still need to cover them, you don't need to repeat them endlessly.

For example, I think of code like this:

let a = ptr::read(x);
let b = ptr::read(y);
ptr::write(x, b);
ptr::write(y, a);

And, looking at the good safety comments on those functions, I'm really not convinced that I want to have to see it as

unsafe {
    let a = ptr::read(x).unsafe(valid_for_read, properly_aligned, initialized_for_type);
    let b = ptr::read(y).unsafe(valid_for_read, properly_aligned, initialized_for_type);
    ptr::write(x, b).unsafe(valid_for_write, properly_aligned);
    ptr::write(y, a).unsafe(valid_for_write, properly_aligned);
}

when that still didn't cover the "oh yeah and you better not extra-drop those return values from read".

My current thinking instead is I'd prefer something like sketched here:

Where rather than needing to repeat things, maybe I can instead have

unsafe
    [properly_aligned(x), "this came from a reference so it must be"]    
    [properly_aligned(y), "this came from a reference so it must be"]    
    [...]
{
    let a = ptr::read(x);
    let b = ptr::read(y);
    ptr::write(x, b);
    ptr::write(y, a);
}

where the function was defined with something like

unsafe
    [properly_aligned(ptr), "it must be normally ABI aligned for its pointee type"]
    [readable(ptr, 1), "must be able to read one element of its pointee type"]
    [...]
fn read<T>(ptr: *const T) -> T;

and the compiler makes sure I "proved" the necessary things to be able to call it, doing proper handling for the variable names involved.

(Maybe readable(in_ptr, count) and writeable(out_ptr, count) would exist on copy_nonoverlapping, for example, as well, to cover cross-parameter requirements.)

4 Likes

Without the fact that your syntax allows more precise requirement (targeting specific variables), this is the same as my first stated alternative, with the same drawbacks and advantages.

I am also not pleased with the syntax, but I don't have better idea.