Conditions for unsafe code to rely on correctness

If you made a hard rule that unsafe code can't trust other safe code for correctness, it would lead to some absurd conclusions.

Vec is a safe API, so you couldn't trust vec.len(), but you couldn't even verify it, because < and == are Ord and Eq safe trait implementations (these are in the standard library, but if that's the deciding factor, it's not about safe/unsafe API distinction any more, but a more general opinion about which code you find trustworthy).

It's still wise to have defense-in-depth assertions around safety-critical code, and minimize assumptions you have to rely on without verifying. But you can't check everything. At some point you just have to write code that depends on libraries and the compiler being correct.

4 Likes

I don't have a function but the unsafe trait bytemuck::Pod was missing crucial requirements in its documentation for a while initially. I don't know of an explicit crate misusing it intentionally but since derive was introduce later there are plausibly some unintentional misuses out there. I'd say that advisories for unsoundly weak safety requirements on unsafe traits, just like unsoundly weak preconditions on exposed unsafe functions, are in scope of RUSTSEC; in terms of process they should be something verified during upgrade but due to the way tooling works informational isn't the right qualifier here either. That would show up before the upgrade, not as an artifact of 'you should have checked that' when running cargo update to introduce a newer version into a lockfile.

If this is to be actionable, we'd need to be able to detect the version any specific impl was coded against (via an annotation that identifies a version, not a symbol) and compare that to the current one in use—then require an update to the annotation as the resolution of the advisory. I suppose minimum required version is an approximation but I'm not fond of the mix of SemVer vs. semantic requirements of a crate (because bumping dependencies goes against the goal of shipping the fix to as many downstream users as possible if they can't consume newer versions).

1 Like

Thanks for the replies! Based on them, I believe the thread is actually about solving the following (apparent) contradiction.

To save space, I'll use the convention that x_y is the item y of crate x and that N is the type of natural numbers. All scenarios are from the point of view of the crypto crate.

Let's call Extern the following scenario:

/// Returns the `i`-th prime number (panics on zero).
pub fn primes_get(i: N) -> N; // it's really pub use primes::get;

pub fn crypto_test() {
    unsafe { assert_unchecked(primes_get(1) == 2) }
}

Let's call Intern the following scenario:

fn crypto_get(i: N) -> N { /* correct implementation */ }

pub fn crypto_test() {
    unsafe { assert_unchecked(crypto_get(1) == 2) }
}

Let's call Param the following scenario:

/// Safety: `prime(i)` must return the `i`-th prime number for `i` positive.
pub unsafe fn crypto_test(prime: fn(N) -> N) {
    unsafe { assert_unchecked(prime(1) == 2) }
}

Let's call Trait the following scenario:

/// Safety: `get(i)` must return the `i`-th prime number for `i` positive.
pub unsafe trait Primes {
    fn get(i: N) -> N;
}

pub fn crypto_test<P: Primes>() {
    unsafe { assert_unchecked(P::get(1) == 2) }
}

In the Param and Trait scenarios, crypto_test is abstract over the function returning the i-th prime number (through prime and P respectively).

In the Intern scenario, crypto_test is not abstract over crypto_get (it's a concrete implementation).

However, in the Extern scenario, we need to look at how the crypto crate depends on the primes crate:

  • If it depends with = then it's not abstract. It's equivalent to the Intern scenario.
  • If it depends with ^ then we need to check the presence of a top-level Cargo.lock file:
    • If there is one (e.g. because we know the final binary), then it's not abstract. It's equivalent to using = which is equivalent to the Intern scenario.
    • If there is none (e.g. because we look at the crypto library only), it seems to me that crypto_test should be considered abstract over primes_get since there's not yet any concrete implementation.

In that very last case, is crypto_test sound in the same way it is in the Trait, Intern, and even Param[1] scenarios?

  • If yes, what's the argumentation and how does it compare with the argumentation for Trait, Intern, and Param?
  • If no, what's the rationale to accept such code while rejecting the Trait (resp. Param) scenario if it would use a safe trait (resp. a safe function)?

Note that it is safe to use a [patch.crates-io.primes] section in the top-level Cargo.toml (either with the official git repository or with a local path to safe code). This essentially means the final binary may use any possible safe implementation for primes_get, making it similar to the Param scenario but where the argument is provided at binary-level instead of each call-site.

I suspect the answer is of practical matter, because crypto_test is theoretically unsound in the Extern scenario (there is a safe binary that triggers UB because of crypto). The ecosystem simply accepts such unsoundness because the alternative would be worse (kind of like it accepts the File and Command APIs being unsound because there's no unsafe keyword to track environmental UBs: it's not practical for the theoretical benefit).

So this suggests that unsafe code can indeed rely on the correctness of external items (for internal items it's obvious), not because it is theoretically sound to do so, but because there's no reasonable alternative yet (maybe one day something based on cargo-vet could work, the same way a paranoid setting in cargo could ask to type cargo build --unsafe="executions of the compiled binary won't have environmental UB" or refuse to build).


  1. One needs to extend the notion of "sound" to unsafe APIs for that scenario ↩︎

2 Likes

To me it's very simple: unsafe specifically describes a requirement on the caller. It is part of the interface between the caller and the callee.

It doesn't describe any other kinds of risks or dependencies.

Here crypto_test doesn't have a requirement for its caller. So crypto_test doesn't need to be unsafe.

Same here. No requirement on the caller.

Here the caller provides prime and there is a restriction on what kind of prime they can provide. Hence: unsafe.

Here, all possible Ps are OK because all Ps always satisfy the criterion (otherwise there is UB in the implemention of P and we assume no UB). Hence: no requirement on the caller.

If the trait wasn't unsafe, then it is no longer true that all Ps satisfy the criterion. Hence: there would be a restriction on crypto_test's caller, so then we'd need the function to be unsafe.

No because primes_get is not provided by the caller of crypto_test. It's similar to using your own function that you haven't written yet. It doesn't matter to whether the function is unsafe because the order in which you write the functions is not something that concerns the caller.

No. It would only be unsound if primes_get is buggy. crypto_test's unsoundness would then be primes_get's fault. As long as primes_get is correct, crypto_test is sound.

4 Likes

Then on what does crypto_test have a requirement? Someone or something needs to make sure primes_get behaves according to crypto_test expectations. That's a requirement that is not fulfilled by crypto_test (or the crypto crate if one needs to extend the scope of unsafe, like is the case in the Intern scenario).

The ultimate thing that can fulfill this requirement is the top-level crate (the one linking primes and crypto together). Either you thread this requirement up the call stack until main making it a requirement for all those callers, or somehow you find a way to bypass that and directly put a requirement on the top-level crate (I could imagine a notion of unsafe crate that has requirements on the linking step, this has come up in the past for things like no_mangle).

These are different types of requirements.

unsafe APIs describe requirements for a specific call. We could call it "runtime" requirements, except technically P is provided at compile-time, so not sure what to call it, but you could imagine P being provided at runtime.

The crypto crate requires that you link it to a working prime crate. That's a very different sort of requirement, not something that unsafe deals with. It's not a requirement on the function caller, it's a requirement on the build system as a whole.

1 Like

There are many ways to make crypto_test "sound" in the Extern scenario:

  • Mark it unsafe by requiring that primes::get is correct. This is not practical because this will contaminate the whole call-stack.
  • Make the crypto crate pin the primes crate to a version where get is correct. That's suboptimal because there are probably many version with a correct primes::get.
  • Mark the crate unsafe by requiring that it is linked with a version of primes with a correct get. We don't have this yet (it doesn't seem hard to have though). The final binary must discharge this requirement during linking. (This can be enforced by cargo.)
  • Each crate assumes its direct dependencies to be correct. That's today's situation.

You seem to be focusing of the first option. Marking a function unsafe is not the only way to make it "sound".

The OP was asking about conventions in the current ecosystem, so I concentrated on the existing language.

Now you're proposing some new language feature of unsafe crates -- I don't quite understand how this would work, so I didn't comment on that. It seems like a completely different meaning of unsafe.

No, you concentrated on a specific fix (marking crypto_test as unsafe in the Extern scenario) that AFAICT no-one suggested. I still played the game in my first reply because that's indeed a theoretical solution worth documenting, and because you were rejecting that solution (which you came up with) for wrong reasons.

Also I realize I forgot to put the pinning option in the list. I'm adding it now. The fact that I've also added new alternatives to see the spectrum of solutions, was only to show that marking the function unsafe is probably one of the worse.

I am confused. I wasn't providing a fix. A fix for what problem?

Crate A's soundness may rely on crate B's correctness. I don't think that requires fixing, it's by design. This doesn't automatically make A unsound.

If you replace crate B by some broken replacement, then A might no longer be sound. That's by design, A is only designed to work with a correct B.

1 Like

A fix to the problem that a safe program can use the crypto crate to trigger UB.

This is not the problem. This is actually today's most practical solution. More precisely: a crate may assume its direct dependencies to be correct. This assumption does not hold, but we hope that it holds enough for the ecosystem to stay healthy. (That's probably still better than the situation in C, although the "hope" aspect is comparable.) Also I'm sure tooling will improve to better track this issue (of getting UB because of distant safe code being incorrect).

I'll update the original post with this conclusion. I think it answers my question (at least for me).

I think that's like saying "it is safe to edit the primes crate source code that cargo downloaded and cached locally, therefore primes_get may be any safe function" or "it is safe to use a patched version of cargo that changes the source code of the crates it compiles in some specific way". This might be technically true if you hyperfixate on a narrow definition of "soundness", but the results of this reasoning are something between excessively verbose and absurd, so I think this is not a useful extension of the concept of soundness.

It is entirely theoretically sound, if we use the right definition for "sound" in a theory that encompasses things like the build system and the file system -- which no existing theory of soundness does, to my knowledge (and it's also not clear to me that one would get much value out of building such a theory).

I don't think it obvious does. One such theory considers the crypto crate to be a functor (a crate taking a module parameter named primes and returning a module). In that case (the Extern scenario), the crypto crate is abstract the same way the crypto::test function is abstract in the Param and Trait scenarios. Why handle one kind of abstraction differently? One thing I could imagine, is in the Extern scenario, you can pin the version to not be abstract, or you can trust the primes author to be the same within the version requirement you chose (usually ^). It's harder to put such requirement on a caller, hence the possible distinction between those abstractions. But this doesn't seem like a very strong theoretical argument to me.

By choosing to model crypto as a functor, you already chose a model that corresponds much more to "crypto that is generic over primes" than "crypto against a particular primes". Since in Rust we typically think of crates we depend on as a specific crate, not some arbitrary implementation of an interface, I would say that's an odd choice for a model of Rust crates. We'd use a trait if we wanted to be generic over the implementation. (IOW, a functor in ML-style languages corresponds to a trait-based generic implementation in Rust.)

2 Likes

How would you model that? In particular I'm curious how "a specific crate" won't get confused with "a specific implementation" (like in the Intern scenario).

We can introduce the Sealed scenario to help with comparisons. It's the same as the Trait scenario but the trait is safe and "sealed". What "sealed" means depends on how crates are modeled, but it may help demonstrate how not having a specific implementation means having an abstraction. It might be a "controlled" abstraction (to be defined), but it's still an abstraction.

A specific crate is a specific implementation. So I'd just model this as basically including the source code of all dependencies -- i.e., verifying my code against a particular version of the crates I depend on. Obviously that ignores what happens on crate updates, but I think it's fine to treat that as an orthogonal problem.

Es we established above, Rust treats code generic over traits and code depending on another crate very differently. Any formal model that ends up modeling both of these in the same way thus fails to capture how the language is actually used.

So that would be a 4th answer to the conditions for a crate to be claimed sound when its unsafe code relies on correctness of its dependencies.

Answers that exist today (from least safe to most safe):

  • Assume the dependencies to be correct (regardless of implementation). This assumption does not necessarily hold since the actual implementation (in the final binary) may be incorrect. This seems to be the current convention.
  • Pin the dependency tree while determining whether the crate is sound. The crate can either rely on the implementation of those dependencies or rely on their correctness only (which can now be verified). This pinning assumption does not necessarily hold when the crate is used in the final binary.
  • Pin the dependency tree in the manifest. Same as above except the claim now always holds (modulo patching). The problem is that the crate is less usable (pinning is an antipattern).
  • Wait until the final binary is known and check soundness for the dependency tree of that final binary. This needs to be done for each final binary (could be shared through cargo-vet).

Only the first option really answers whether a crate is sound in isolation. The other options need to look outside (either only upstream for the next 2, or also downstream for the last one). That's a consequence of the notion of crate not being properly modeled.

There are also answers that don't exist today (and address a crate being sound in isolation):

  • A crate can be marked robust (or a subset of its items), to indicate that its correctness (or that of those items) can be relied on by unsafe code. Since this marker is part of the type system and thus the crate API, it is tracked by semver. This marker can also just be at documentation level, for the same effect, but with poorer tooling support.
  • A crate can be marked unsafe to indicate that it has safety requirements to link. The requirement here would be which dependencies (or which subset of their items) need to be correct for safety purposes.
  • Some TBD modeling of crates, that is not an abstraction over its dependencies.

The first 2 options are obviously cumulable (since dual), and which crate is marked depends on a case by case basis (the standard library is robust, but some random crate relying on some other random non-robust crate needs to be unsafe). Also note that correctness doesn't play a particular role. Any property can be used for the robust and/or unsafe marker.

Also those first 2 options don't fundamentally change from the current 4 options. They simply give a proper explanation of the safety contracts between crates, and the possiblity to select it on a crate by crate basis rather than a single rule at ecosystem level.