Safe Library Imports

[dhm]

I’ve ported lodepng from C to Rust, and its slice-ified deflate is 2 to 10 times slower than best (unsafe) deflate implementations.

The ideal solution here is for consumers of the library to choose what they prefer: is security/the safety of you application far more critical than its speed? (e.g. a non time-constrained application, or an internet-facing server). Then use the safe version of the lib. Is the security constraint less important than its speed (e.g. videogames)? Then do pick the unsafe version of the lib.

Given #[cfg(feature = "no-unsafe")] and cfg_if!, I think it is really possible for a library author to expose a library where users can opt in/out of unsafe, even if it requires/implies a runtime cost (e.g. RefCell, Rc, Option pattern matching, …):

Cargo.toml:

[dependecies]
"some-lib" = { version = "0.6.9", features = ["no-unsafe", ], }

That’s remarkably different from “safe-only” languages that pay performance and/or complexity penalty (things like GC and/or stricter immutability) in order to be able to do more useful things safely

It could be nice to attempt to opt-in this mode for untrusted dependencies when safety is paramount, while keeping the unsafe of other dependencies (not just ::std obviously) for the speed.

Empowering library users by giving them the choice.

[kornel]

That puts a burden on library maintainers to maintain two versions.

If you require safety and can accept performance penalty, then use WASM in a sandbox. That puts the burden on you, not literally everyone else in the ecosystem.

[Ixrec]

@dhm This again seems to make the assumption that the presence of unsafe in a library always, without exception means client applications using it are less safe.

I still don’t understand this position, so let’s try this:

• If the unsafe code was known for a fact to be perfectly sound, would you still want the library to maintain a no-unsafe feature?

• If some new unsafe code optimization is proposed, and it’s not known to be sound, would you want the library to add it anyway alongside a no-unsafe feature until soundness can be proven or disproven?

I think most of the comments here (or at least mine) are made from the perspective that if unsafe code is not known to be sound, it should never be published in the first place, but if it is known to be sound, then it’s not a threat, and there doesn’t seem to be any sort of “probably sound” middle ground, so there’s no use case for a no-unsafe feature. Which part of this would you disagree with?

[dhm]

That puts a burden on library maintainers to maintain two versions.

If library maintainers want to earn the trust of more skeptical users, it is obvious that some kind of effort is required:

• they sacrifice performance to gain trust, by not using unsafe;

• they “prove” (or hint a plausible sketch of a proof) that the unsafe used is sound (or get someone or something to do it for them (i.e. auditing));

• they accept to “double” the work required to maintain some functions / structs by providing two versions of each: a reliably fast one and a reliably safe one (that is, for instance, what I have started doing);

• or they do not do any of the above (because performance is paramount for the library to be interesting and because they do not have the time for the alternatives); in that case users should be warned / told about the library maintainers’ choice.

use WASM in a sandbox

That is is a genuinely interesting alternative, I’ll look into it. That makes me think there should be a Rust Security Book, with its own section about using rust in a WASM sandbox.

[dhm]

(I thank you @Ixrec for your good attitude in trying to come to an agreement).

In my (maybe not so) humble opinion (aside: the very fact of qualifying oneself as humble is a contradiction xD):

• My concern with unsafe is that it is dangerously close to C. For many this is not an issue, since many programs have been written in C without any vulnerabilities whatsoever. Yet, where I work (security-related field), C is the bread and butter of most sneaky / harder-to-spot bugs: stack & heap overflows, integer arithmetic with pointers (e.g. through indices), use-after-free, data races… Of course, there are other sources of bugs: eval/include-ing user input, file accesses (read/reveal secrets, write/corrupt system data), but those are less sneaky (sometimes they are even related to evil intent from the programmer (e.g. a malicious package / dependency)). The point of all this is that C / unsafe bugs are easy to miss. And many people do not realise at which point / extent that is true.

  I, for instance, have used unsafe in a crate trying really really hard to be careful and using tests etc. and yet I used ::std::mem::uninitialized for a user-given generic type T. Even if I was never reading such a value, it was still UB since the value could hold a value bit-layout-incompatible with the type (e.g., value 3_u8 for a bool, or a null reference) and the compiler may use that type-provided layout guarantee for some exotic optimization . I did not know that at that time

  On the same level, I have seen people attempt to prevent bugs by zero-ing secrets (e.g., on drop), and even go as far a recommending it, without any special attention paid to generics: back to wrong-bit-layout UB.

  TL,DR: UB is so incredibly easy to get when using unsafe that I do demonize it. I’d rather demonize it than naively trust it.

• Specially when there are usually safe alternatives for what the code is attempting to do, by using Rcs, RefCells, more Options. And in many cases the runtime cost is negligeable.

• On the other hand, I do trust attempts at fixing the latent danger of unsafe, not only when it has been proven safe (obviously), but also when it has been “thoroughly” audited:

• If the unsafe code was known for a fact to be perfectly sound, would you still want the library to maintain a no-unsafe feature?

It does not seem necessary to me;

• If some new unsafe code optimization is proposed, and it’s not known to be sound, would you want the library to add it anyway alongside a no-unsafe feature until soundness can be proven or disproven?

That would be my dream. But, more realistically, it should at the very least be added as a major semver change (“safe API -> potentially unsafe API” = “API change”). Once proven sound, it could be retro-added as a minor semver change.

[Ixrec]

That sounds like we actually mostly agree on this stuff, and the disagreement is mostly a matter of tone and emphasis.

In particular, where you say “demonize” in your post, I’d say “the bar for declaring a piece of unsafe code “known-sound” should be extremely high”. That’s obviously very different from and not at all in conflict with my “don’t demonize” meaning “don’t discourage library authors from using known-sound unsafe code”.

(again, if anyone can think of better slogans/shorthands, I’m all ears)

As a quasi-conclusion: I would think it falls under the purview of the security working group’s efforts to audit and minimize unsafe throughout the ecosystem to also decide:

• exactly where we should set the bar for declaring unsafe code to be “known-sound” (e.g. a test suite with some amazing coverage metrics that fully passes the stacked borrows model)
• if there are any unsafes where a no-unsafe feature would be appropriate for whatever reason (e.g. its soundness depends on unresolved issues with the unsafe code guidelines, so it’s impossible to prove sound or unsound for now)

Unless someone has a concrete example they’d like to dig into, I think we’ve probably exhausted the philosophical discussion for now.

[dhm]

Well said!

[granthusbands]

I think most of the comments here (or at least mine) are made from the perspective that if unsafe code is not known to be sound, it should never be published in the first place, but if it is known to be sound, then it’s not a threat, and there doesn’t seem to be any sort of “probably sound” middle ground, so there’s no use case for a no-unsafe feature. Which part of this would you disagree with?

There’s the question of who knows it. If the maintainer/publisher knows it to be sound, that doesn’t help my knowledge of its soundness. Whereas code that doesn’t use unsafe is sound (wrt to types and memory).

I’d say “the bar for declaring a piece of unsafe code “known-sound” should be extremely high ”

That’s fair. Of course, being able to avoid unsafe then still has merit, as it saves a lot of work meeting and checking this extremely high bar.

Unless someone has a concrete example they’d like to dig into, I think we’ve probably exhausted the philosophical discussion for now.

I think there’d be mileage in discussing what unsafe is really meant to be. It seems that anything that becomes a common unsafe pattern probably creates a need for an equivalent non-unsafe language feature, but that’s probably a discussion for a whole new thread.

[Ixrec]

There’s the question of who knows it. If the maintainer/publisher knows it to be sound, that doesn’t help my knowledge of its soundness. Whereas code that doesn’t use unsafe is sound (wrt to types and memory).

And that gets us right back to web-of-trust tools

I think there’d be mileage in discussing what unsafe is really meant to be. It seems that anything that becomes a common unsafe pattern probably creates a need for an equivalent non-unsafe language feature, but that’s probably a discussion for a whole new thread.

I think it’s generally agreed that common unsafe patterns are good candidates for addition to the language or std (I’m not sure if you meant “language feature” to include std), although not every legitimate, known-sound use of unsafe is something that could be made into a language feature (especially FFI and embedded stuff). But yeah, definitely a new thread if there’s anything more to say there.

[granthusbands]

And that gets us right back to web-of-trust tools

Sure, though they’re much more complicated than restricting unsafe and types with system access.

[kornel]

By “demonizing unsafe” I mean treating all uses of unsafe blocks as unacceptable by default, because of what they might be, and not based on what they actually are. That’s fear and prejudice.

I’m fine with solutions that are based on trusting authors to use unsafe properly and/or code audits to actually check what the code is doing. That is healthy skepticism and rejection of code that is actually dangerous.

[kornel]

Going back again to the original post: the official zlib library is thoroughly reviewed and fuzzed. It has an OK track record (some bugs, but nothing major in the last decade). I’d consider it acceptable to use, despite being 100% “unsafe” with 40000 lines of code.

Currently, I can use this library without any problems. I can build my libraries on top of it and have gzip that is pretty fast and safe in practice.

But if use of this library would cause my libraries to be marked as dangerous, banned from some “safe” contexts, then I wouldn’t use it. I just don’t want to deal with this. Instead, I would use a slower “safe-subset-Rust” gzip library. It would be loss of performance for no real gain in safety, just because of social pressure, not technology.

[granthusbands]

By “demonizing unsafe” I mean treating all uses of unsafe blocks as unacceptable by default, because of what they might be, and not based on what they actually are. That’s fear and prejudice.

That’s very emotive language. Note that the security working group’s mission includes:

• Most tasks shouldn’t require dangerous features such as unsafe . This includes FFI.

So a little fear might be a good thing.

But if use of [gzip] would cause my libraries to be marked as dangerous, banned from some “safe” contexts, then I wouldn’t use it.

A fair point. I would still personally prefer to have a no-unsafe option, but some ways to moderate it may also be useful. I’d expect in the long term to be able to say ‘I trust zlib’ without having to trust other unsafe or native libraries. The conditional compilation options mentioned earlier would allow you to use no-unsafe zlib or native zlib depending on what the including crate trusts, wouldn’t they?

[bascule]

For what it’s worth, I made a pretty detailed proposal for how to implement a feature like this by tagging/associating usages of unsafe with cargo features, with the goal of allowing crate consumers to specify which dependent crates are allowed to transitively consume unsafe features of other crates:

Crate capability lists

I’ve been thinking along similar lines, but specifically around the problem of restricting usage of unsafe. I posted some initial thoughts here: https://groups.google.com/d/msg/cap-talk/t9al5hjN19U/XzHfR1peBAAJ I’ve thought about posting a “Pre-Pre-RFC” about this, but I guess I can start by spitballing here. Unsafe Features Synopsis: Extend the existing idea of cargo features with a special notion of “unsafe features” which can be used to whitelist usage of unsafe in dependencies (and their…

[kornel]

For your original use case of importing unaudited libraries and knowing they can’t access files or network, the safety seems to be all-or-nothing. If FFI was allowed, how would Rust ensure it can’t call fopen? How could FFI work while still forbidding arbitrary pointer dereference?

[granthusbands]

If FFI was allowed, how would Rust ensure it can’t call fopen ?

FFI is as dangerous as unsafe, so one would typically not want to allow it. If one wanted file access, one would allow import of a well-known module that can read/write files. That module being allowed FFI still isolates it from the layer you don’t trust.

If you’re talking about the FFI for zlib, you’d have a zlib module that you allowed FFI/unsafe, which you might permit as a well-known library, but you wouldn’t have to give the png library that.uses it the ability to use FFI/unsafe itself. Ideally, there’d be one common zlib that could have an unsafe version and a safe version and you’d import that from the png lib and not have to care whether you’re in a context that allows unsafe.

[kornel]

FFI is as dangerous as unsafe, so one would typically not want to allow it.

Right, that’s why I’m confused by:

Most tasks shouldn’t require dangerous features such as unsafe . This includes FFI

If semantics of FFI don’t change (e.g. it’s not turned into emulation/sandbox), then it is identical to allowing unsafe. Will it just have a different syntax without the unsafe keyword, but the new syntax will get identical treatment as the unsafe keyword? What’s the point?

Of course you could vet crates and allow them (that’s the trust/auditing approach I think is the best solution), but there the distinction between FFI and unsafe is not meaningful, you still have arbitrary pointer dereference allowed, and vet that it’s not abused.

[granthusbands]

If semantics of FFI don’t change (e.g. it’s not turned into emulation/sandbox), then it is identical to allowing unsafe

It looks like an FFI call requires unsafe, separate to the definitions. I think what the group is trying to say is that one can’t just say “Using unsafe for just FFI is fine”; they’re saying (I think) that FFI is as unsafe as other unsafe things and so won’t get a free pass.

[granthusbands]

For what it’s worth, I made a pretty detailed proposal for how to implement a feature like this by tagging/associating usages of unsafe with cargo features […] Crate capability lists

Indeed. With the later modification that some of the unsafe in std is simply trusted by default, it could be a good part of the solution. Though the FFI in std would then probably be trusted and you’d want another restriction to prevent file and network access and such, by default.

[bascule]

std would then probably be trusted and you’d want another restriction to prevent file and network access and such, by default.

I was suggesting using the subsystems as std as the rough modeling dimensions for “unsafe features”:

rust-lang/rust

You can imagine things like file and network access requiring, at the very least, a std/io unsafe feature is enabled.

For FFI, I’m okay with the existing API remaining unsafe forever, however I would be interested in seeing it wrapped in something higher-level which can provide safe abstractions. We’ve seen a few things of this nature spring up for doing language-specific bindings. All that said, if you had such a safe_ffi crate which wrapped the unsafety, this same mechanism could be used to control which crates have “safe” FFI access.