Safe Library Imports

Rust has one very simple reason for not including an effect system: the strangeness budget. Getting people over the borrowck hump is hard enough; teaching them to use an effect system is also very hard; overcoming both of them would be exponentially more difficult, since they’d have to learn both of them at once.

The thing that always annoys me about discussions like this, as you’ve already said, is that it’s very easy to fire off shots in it without very much investment, without necessarily arguing in good faith, and while refusing to admit to their real motive. So I’ll put it out there right now:

I think Rust should sacrifice infosec in this case in the name of expediency. The concept of the “weirdness budget” articulates why I think it’s expedient; not having an effect system means that nobody has to be taught how to use it. Not having an effect system is essentially catering to the lowest common denominator, since it still leaves open language-agnostic opportunities for supply chain management for those who really need it.

4 Likes

The original request in this tread was about a PNG library. Speed of PNG decoding and encoding is almost entirely dependent on the gzip implementation, which in turn greatly benefits from optimized memory copies (unsafe) and SIMD (unsafe).

I’ve ported lodepng from C to Rust, and its slice-ified deflate is 2 to 10 times slower than best (unsafe) deflate implementations.

So “just don’t do funny stuff” type of import would be very unsatisfying. You’d probably want to allow the PNG library to use unsafe zlib.

2 Likes

Rust has one very simple reason for not including an effect system: the strangeness budget

A fair point. I should note that I thnk my proposal is a lot simpler than an effect system, both in effort and strangeness, but the strangeness may still be higher than you’d like. And the performance of safe rust for file parsing is much worse than I expected, making the cost higher.

I think Rust should sacrifice infosec in this case in the name of expediency.

I can appreciate this point of view; there are all sorts of budget in language design.

The simplest variants of this could be voluntarily followed (as scottmcm noted) and ignored by the greater community (as it involves flagged imports and compile time conditionals and could use an existing no_unsafe feature), which I think makes it less of a sacrifice.

It still feels a little like batting it down without due consideration, especially as you describe it as an effect system, which it is only superficially similar to. But I do understand my bias as the proposer might always make it seem that way.

Would a complete worked example help, do you think? (as a github fork of a crate and some documentation)

That is worse than I expected and may indeed be too much of s performance drop for safety for most people. (Edited in:) One could trust a gzip library to use unsafe code without trusting a png library that uses it to use unsafe code, but that’s a big ask.

Yeah, it would need to be finer-grained to avoid this problem. You’d then hypothetically want to use a flag to only allow particular non-core imports (or none) but still allow unsafe code. It would still protect againt programmer error, to some extent, but not against malicious intent.

(Meta: I only now discover how to quote someone properly. The quote button in the toolbar should perhaps give a helpful hint, in the preview pane, about selecting text in the page.)

It has been pointed out that Effects keep coming up because my description may lead people in that direction. What I’m actually talking about is more like module-scoped capabilities. It’s perhaps best described in sequence.

Imagine you’ve got some Rust code in which you cannot import any libraries at all and cannot use unsafe. With such code, as long as Rust itself is not broken, you cannot do anything but compute (and allocate supporting memory). There’s no file access, network access or system access of any kind. But there’s also no vector classes, no mathematical operations beyond the basic operators, and a number of other missing features.

There is a subset of the standard library that provides vector classes, mathematical operations and other things. These provisions still don’t allow file, network and runtime-bypass abilities and so are safe for anyone to use, even if some users are malicious.

Then there are parts of the standard library (and unsafe) that do provide malicious or exploited actors with a lot, like the aforementioned abilities to access files, access the network and bypass runtime safety.

If you could specify that some library only had access to the safe subset that could work in isolation from an OS, and whatever additional libraries you whitelist, then you would know it could not either drain your bitcoin wallets or accidentally allow others to do so.

Note that allowing access to some library that can access files (like a timezone library accessing the tzinfo file or a logging library outputting to a known location) is fine. As long as Rust’s type system is working, that file access can still work, even while the caller has no other way to access files. Similarly, you can provide a png library with callbacks that grab data from a file, without it being able to access files itself.

All of the access to libraries and similar is through mod and therefore, I’m suggesting restriction attributes on mod that would restrict what further sub-modules a module can load, with an easy way of saying ‘only the known-safe ones’. So your main module could import what it likes, but it could restrict sub-modules to importing only specific things. The current Rust type system would take care of the rest.

It’s worth noting that #![no_std] actually goes some way towards this, but eliminates a number of useful base libraries. Splitting std exports just a little could facilitate both better embedded programming and this proposed new feature.

Those who don’t care about any of this could just continue importing all of std and carry on as normal. It only restricts those who choose to have the extra safety.

I think there’s a lot of confusion, discussion scope creep and talking past each other going on, so let me take a step back and make explicit a bunch of stuff I was probably far too terse or implicit about in my last post (and I think is also being hinted at in everyone else’s posts).

Solving Security and “Shooting Down” discussion

Security is hard. There is no One True Threat Model that applies to all users of Rust. There is no single tool, or single feature that can solve all the security issues that matter in practice. Perhaps more importantly, these tools/features/threat models often cannot be discussed in isolation from each other because all these concerns intersect and overlap very heavily.

Like any other challenging problem with no easy answers, there is a long history of people proposing overly simplistic solutions that ignore part of the complexity and in practice would cause more problems than they solve. Explaining why they’d cause more problems than they solve is also very difficult, and it’s often impossible to truly convince everyone. In particular, the people suggesting these solutions often get so defensive that they stop thinking about what’s actually best for the Rust language and ecosystem in favor of just trying to “win the argument” (this happens on a lot of internals threads, not just security-related ones).

“Mantras” like the ones you listed are what happens when the same argument has to be repeatedly invoked against new equally flawed proposals on a regular basis. The intent is not (or at least it’s never supposed to be) about “shooting down” propsoals or “shutting down” discussion, but only to avoid wasting everyone’s time rehashing things that “everyone” knows already. Of course, the newcomer making this familiar and flawed proposal is not part of the “everyone” that already knows it’s familiar and flawed, which leads to this perception of undeserved hostility.

We’ve clearly crossed the line where this shorthand is leading to everyone talking past each other instead of avoiding wasted discussion. Which is why I’m about to bite the bullet and rehash some stuff (or to borrow your phrase, do our share of the “emotional labor”).

Philosophical Objections

“Demonizing Unsafe”

For historical context, I believe this term originates in [Pre-RFC] Cargo Safety Rails and I’ve probably helped popularize it by using it in posts like [Pre-RFC] Cargo Safety Rails and Crate capability lists

Here are some of the key sound bytes:

  • “While I agree that evaluating the quality-level of a program including its dependencies is a good goal, I think focussing on unsafety as a way to do so is somewhat naive.”
  • “I believe that the majority of unsafe code is not a problem - it is small, self-contained, and innocuous.”
  • “unsafe code is not the only source of bugs. If you care so much about safety/quality that you would check every dependent crate for unsafe code, you should almost certainly be checking them all for logic bugs too.”
  • “If we treat known-sound unsafe code differently from safe code, that makes known-sound unsafe code seem more dangerous than it really is, and makes safe code seem more safe than it really is.”
  • “there’s a very strong risk of any audit framework/tooling like this unintentionally leading to crate authors being discouraged from using any unsafe code for optimization, even when the soundness of that unsafe code is uncontroversial … we’d be causing more harm than good if we introduced a system that did have this problem in practice (if nothing else, it risks encouraging the idea that security is at odds with performance).”

As in those threads, I’m assuming it’s uncontroversial that there is such a thing as “known-sound unsafe code”, that we as a community are capable of identifying it, and that it should not be treated any differently than safe code by the ecosystem (even though the compiler and core language obviously do need to treat it differently).

Hopefully it’s now obvious why this is a legitimate objection to the proposal in this thread, just as it was in those past threads. If “safe imports” were added to the language, then it becomes a breaking change for any library to replace safe code with (even known-sound) unsafe code. While I hate to sound like I’m shutting down proposals, we have to be able to express objections to proposal, and I really do believe that would cause far more harm than good.

If anyone can think of a better shorthand for this position than “don’t demonize unsafe”, I’d love to hear it.

To be super clear, I do think the presence of unsafe is a good heuristic for where to focus personal or community auditing efforts. I also think that the bar for “known-sound” ought to be very high, and I agree that a minority of people in the Rust community can meet that bar. But none of that contradicts the rest of this section.

One True Threat Model

In an even broader sense, the real problem with many of these past threads is that they’re effectively proposing that we hardcode one specific threat model into cargo or crates.io or (in this case) the Rust language itself.

This is where the mantras of “false sense of security”, “not the real problem”, “static analysis is the answer”, and “trust audits are the answer” come in.

What we should really say is:

  • only the application developer can decide what their threat model should be
  • obviously, there is no single threat model that is correct for all apps all the time
  • less obviously, there is also no good “default” or “lowest common denominator” threat model that is “good enough” for “most” apps
  • we should avoid designs that would give application developers the impression that they don’t need to pick a threat model, or that we’ve solved that problem for them and they don’t need to think about what their threat model is
  • we should develop tools and features that enable you to enforce whatever threat model you care about

Or at least that’s what I have in mind when I say things like that. I think we all agree that there are important threat models where ruling out file system access, network access or un-audited unsafe code would be extremely valuable, but there are also other important threat models more interested in side channels, timing attacks, DoS attacks and so on. We simply shouldn’t hardcode any one of these threat models.

Incomplete/Unclear Proposal

How would we actually use these features?

Another common problem that I think gets in the way of productive discussion is a failure to articulate how security-related proposals for tools or language changes would actually get used in practice. Because of the obvious practical reality that nobody can clean-room reimplement all of their dependencies, or manually audit all of their dependencies, what workflow you have in mind becomes extremely important.

For instance, with the proposal in this thread, the only usage I’m aware of is to simply mark all your imports as “safe”, see which ones fail to compile, then remove those “safe” annotations and then audit those crates. That’s essentially the same workflow as running a web-of-trust tool on your project to produce a list of dependencies that need auditing. If tools like cargo-crev ranked the un-audited crates by things like “contains unsafe”, you get all the same benefits of focusing on more-likely-to-be-dangerous code without “demonizing” any of the crates with known-sound unsafe (assuming we can define “known-sound” in terms of community audits, which is a big assumption).

So in addition to the abstract philosophical objections given above, I’m also just not seeing how a language feature like this would be a practical improvement over an external tool. I think this is a big part of the reason many are responding by saying this solves the wrong problem or other tools do a better job.

Did you have some other workflow in mind? Are there any imports you wouldn’t always want to apply “safe” to? Did you want to imbue “safe” with some other semantics beyond sometimes failing compilation? When you need a crate that uses unsafe, would you do something other than audit it for soundness?

Why all the talk of effect systems?

I think this happened because your original post said “they cannot access files or the network”, and it’s not at all clear how this could possibly be implemented (in a robust enough way to provide any meaningful security guarantees) without a full blown effect system. A more recent post of yours also said “If you can get static assurances from the type and build system, you don’t need trust audits and further static analysis” which raises the same question. And as I was writing this, your new post responding to my accidentally posted incomplete draft of this post is making further claims of this sort.

So while I agree that your original post intended to be a far simpler proposal than an effect system, you’ve been consistently making claims that conflict with that intent. Importantly, a lot of your responses to the other objections seem to rely on these claims that are hard to make sense of without an effect system. I think this is the biggest reason why so many posts in this specific thread seem to be talking past each other, but it’s not really relevant to the broader issues that this thread has in common with many past threads, and that’s why this is the shortest section.

Moving Forward

Fundamentally, I believe that “solving security” is too complex of a problem to be solved by simply having people submit proposals and then debating each proposal in isolation the way this forum is currently set up. This is not about whether the “pro-safe import” people “lose” to the “anti-safe import” people; that’s completely the wrong way to frame this kind of discussion, yet it is how it gets implicitly framed by the very format of “one user posts a pre-RFC thread, everyone else posts responses to it”.

And that is why my first instinct when presented with this thread was to link everyone to the working group. This is exactly the kind of thing working groups are for: Look at a complex problem, discuss a wide range of possible solutions with input from as many interested parties as possible, and decide as a group which solutions are worth pursuing and which aren’t, based in part on past experience and which other solutions are out there to avoid redundancy or confusion.

I happen to believe the specific proposal that started this thread is not part of the ideal set of solutions. But I also think that’s far less important than the idea that it makes no sense to discuss this specific proposal in isolation. There are absolutely ideas in this proposal that should be investigated thoroughly, and as far as I know they already are being investigated in other ways.

For example, I think we’re all interested in ways to express “these dependencies cannot use the network”. But we don’t seem to have any concrete proposal for how to define that, and some proposals for enforcing it have the obviously problem that they make adding any kind of network feature a backwards-incompatible change. However, perhaps we could:

  • make all crates that access the network on purpose tag themselves in some way
  • get the whole Rust community to agree on a certain tagging system
  • teach web-of-trust tools to show these tags in their output
  • make one of the audit criteria for web-of-trust tools checking that you’ve been tagged correctly and none of your unsafe code circumvents that and so on
  • state that the intended workflow is not forbidding network usage at compile time but instead a CI job that fails if new network users ever appear in your dependency tree

I think there’s probably a viable solution in here. But as you can see, this faint sketch of an idea is already at least five separate proposals, and I believe that’s typical of security issues. That’s why I think threads discussing single proposals in isolation are not a good way to make progress. If I had more free time or relevant expertise, I’d probably join a working group myself.

I do think that threads discussion problems in isolation might be worthwhile though. For instance, a thread just focused on how to define “network access” could at least work out whether there are any usefully tool-able definitions of that phrase. And personally I’d really like to see a thread about describing people’s intended usage of transitive language subsetting features like this; as discussed above the only usage I know of is “audit every crate using X”, but that makes a lot of these security-related feature proposals obviously redundant, so they must have something else in mind.

11 Likes

I don’t think the latter part is uncontroversial. The whole crux of the “demonize unsafe or not” discussion seems to be exactly that controversy.

I’d love having better tools for unsafe tracking, auditing and so forth. And I’m pretty sure I’m not the only one.

1 Like

Ah, yeah that’s bad wording. I should’ve been more explicit there.

So there are definitely projects that will want to “treat unsafe code differently” in the sense of auditing those crates or forbidden new ones entering their dependency tree without audits and so forth. That’s obviously fine, and is one of the many use cases that we want to enable.

What I meant to say was that we shouldn’t treat unsafe code in any way that would “unintentionally lead to crate authors being discouraged from using any unsafe code for optimization, even when the soundness of that unsafe code is uncontroversial”. And to be extra clear, we absolutely want to discourage using unsafe code when its soundness is not crystal clear for whatever reason.

Is that still controversial?

2 Likes

Phrased like that I agree it’s a good principle, yeah.

Thanks for clarifying!

1 Like

I didn’t know of this “known-sound unsafe code” that one has to write when writing Rust code. It would indeed be a huge barrier to writing code without unsafe. Are there any resources that describe it? Why has Rust not adapted to make it unnecessary?

If I need to spend a lot of time engaging with or being part of a relevant working group, how would I go about that?

hardcode one specific threat model

That’s not what this is. This is directly applying the principle of least authority to module imports. Rather than letting every module load anything it likes, we instead choose what it can load (and whether it can use unsafe, because that’s a bypass). There are many threat models it affects and many it does not. It is a tool, and one that provides more guarantees that many of the other tools under consideration.

I think we all agree that there are important threat models where ruling out file system access, network access or un-audited unsafe code would be extremely valuable

Agreed.

For instance, with the proposal in this thread, the only usage I’m aware of is to simply mark all your imports as “safe”, see which ones fail to compile, then remove those “safe” annotations and then audit those crates.

When finding crates, it would be a category that you’d search inside first. If you find a crate for your purposes that has ‘least authority’, you’d likely use it in place of one that does not. The crate may offer extra features if you give it more authority.

Are there any imports you wouldn’t always want to apply “safe” to?

You’d want that for most imports. If you’re importing a library that runs plugins for a paint package, you might not restrict it. But it, in turn, would likely restrict the plugins.

Did you want to imbue “safe” with some other semantics beyond sometimes failing compilation?

I’d want packages to be able to contain optional unsafe code or file or network access code, accordingly compiled or not. A first version could just forbid it all, though.

When you need a crate that uses unsafe, would you do something other than audit it for soundness?

After auditing it, I’d have to fix the version and add myself to whatever mailing list might inform me of security flaws in that version. Or I’d fork/pull-request an unsafe-optional version. The trust/audit systems under discussion elsewhere could help in this case, as long as sufficiently trusted people have had time to audit it.

effect system

Effect systems affect all function types. This proposal does not alter any type signatures.

consistently making claims that conflict

not part of the ideal set of solutions

These in particular feel like rejection before understanding, especially given that you still don’t know how this proposal differs from an Effect System.

That’s part of my problem. It seems common for people to form that belief before even properly understanding what is being proposed, and to stick to it.

It feels like no matter what I do, some people here will decide that what I’m describing is actually something I’m explicitly saying it’s not, or that it cannot do what I know it does from direct experience, and is hence unsuitable, without/before apparently asking questions that will aid understanding.

I wonder whether some sort of real-time chat might help, to avoid so much talking past each other. But while people are comfortable making authoritative negative claims about a proposal they don’t yet understand, it is hard to see a way forward.

This problem has been addressed for other languages, such as the Caja project for JavaScript and the Joe-E object capability subset of Java. In each case, there was a need to tame the library. Section 4.2.1 of the Joe-E paper describes the process.

I found it interesting how little it affected usability. For example, you control file access by replacing methods that take path names with methods that take open file handles. The result is that you can allow file access without the risk of allowing code to open arbitrary files.

1 Like

This is where I was going with my “75% solution” comments.

There are many interesting, perf-critical crates that one absolutely wants to use unsafe. If one uses them, then one needs to trust them somehow. That might be faith in humanity, faith in the crates.io maintainers to remove broken things, faith in a community reputation, faith in your code audit process, whatever. One does need to trust it, but is good with doing so because it provides enough value to be worth it.

The problem, as I see it, are all the little crates. For example, I have this silly little crate:

Nobody needs this crate. Right now, even I wouldn’t use it if I needed to get permission for things, because it does use unsafe. If you look at it the unsafe is clearly sound, but why would I even bother auditing it when all it does it let me move .rev() calls around?

But if there was a new language feature that meant it didn’t need unsafe, or it could use a well-known trusted crate to do what it needs instead of the unsafe, then I think it would be really nice to be able to argue “look, it’s a low-risk crate” and not need to audit it, to help encourage more crate usage.

I’ll note that in a world where all code needs to be audited, no upgrades can be done without looking at the changes, and thus breaking on new unsafe code is a feature.

2 Likes

Yeah, it does. It has to. Let’s imagine a scenario where this is done entirely at the crate level, with the appropriate trickery of having a std_nonetwork variant that has its types reexported by the std_network etc. Attempting to implement all of this stuff in Cargo without touching the core Rust language itself.

Imagine I’ve written a crate, let’s hypothetically say ammonia, which doesn’t require network support. Let’s say it also depends on content-security-policy. Ammonia shall marked as no-network, and it shall always be marked as no-network, because filtering HTML is supposed to be a deterministic process.

Let’s further imagine content-security-policy has optional networking support. This is also not really hypothetical; the CSP level 3 standard specifies a way to report violations. Presumably, I would make this a Cargo feature (let’s call it network-report-violations) with conditional compilation. Cargo features are supposed to be additive, and I would be careful to ensure that I followed that here, so if one crate that depends on content-security-policy turns on networking support, it shouldn’t affect ones that don’t require that feature. Naturally, this also makes content-security-policy a crate that optionally relies on network.

How should Cargo ensure that ammonia never calls the network-using functions that content-security-policy may or many not have? Obviously, ammonia should not be allowed to turn on the feature within its own Cargo.toml, but if it went ahead and called the network-requiring functions anyway, then it would fail to compile if the feature was turned off, but it would successfully compile if it was turned on by a completely different crate, and it would violate the sandbox by doing so.

1 Like

This is indeed a significant problem, and one I was hoping would be raised and perhaps answered by someone. The question of what to do when the same crate is used more than once with differing permissions is a good one.

It’s perhaps worth noting that ammonia can be included no-network while being explicitly allowed to import content-security-policy with the network-report-violations feature, as a user-level workaround for this multi-import problem. But I’m sure others with more knowledge than me about the lines between cargo imports and rust types could find a better answer.

I’ve had this conversation a few times:

— Rust is thread-safe and automatically manages memory thanks to static analysis!
— What?! That’s not possible! How does it solve <this difficult problem>?
— It doesn’t! :smiley:

Rust as a language is an interesting hack: instead of solving all the hard/unsolvable edge cases, it only solves the easy ones, and leaves the rest to unsafe.

I find this combination incredibly powerful, because it’s safe and easy most of the time, but it doesn’t have any hard limits on how low-level and performant it can get.

That’s remarkably different from “safe-only” languages that pay performance and/or complexity penalty (things like GC and/or stricter immutability) in order to be able to do more useful things safely, and they still must have some hard limits on what they can do.

Rust wasn’t designed as “safe-only” language from the start, so the safe subset is intentionally smaller and not maximally powerful, because it never needed to. So to me “demonizing unsafe” in Rust is a lose-lose scenario: you throw away the powerful/performant part, and you’re left with a half-language that is less powerful than proper “safe-only” languages.

8 Likes

There are ways to handle @notriddle’s problem. One approach is to use a tamed object capabilities library. Just as you limit file access by passing an open file handle instead of a path, you can pass a reference to something that forwards HTTP requests only to a specific set of URLs or that sends all requests to localhost.

1 Like

In my experience, same-language sandboxing of arbitrary untrusted code is incredibly hard and almost always fails. Perhaps the most famous example are Java applets, which have had an almost continuous stream of security vulnerabilities. But I’ve also discovered vulnerabilities in multiple other sandboxing projects. I don’t think it is as easy or reliable as you seem to be assuming.

Yeah, the runtime Java security layer is hugely overcomplicated. It has class loaders, security managers, stack inspectors, protection domains, access controllers, access controller contexts privileged blocks and more all trying to work together to intuit intent, along with easy bypasses in Reflection and Serialization. There were plenty of mistakes both in the design and the implementation, in large part because it is so big. Rust is not Java and the principle of least authority and preventing imports is much simpler.

A large part of what’s being spoken about here is compile-time, not run-time and is already enforced by static typing. Also, the principle of least authority tends to lead to smaller attack surfaces most everywhere, as running code then holds fewer things to be attacked.

Yeah, there will always be vulnerabilities at multiple levels. No matter how good or bad Java’s security layer was, the bypasses using reflection or bad class validation simply bypassed it. That shouldn’t make us give up on good security measures, any more than it makes us give up on memory protection.

It’s still a significant improvement. If you can demonstrate some Rust code that doesn’t use unsafe or import any modules at all, that can do anything at all at the OS level, that might be more convincing.

Basically, every time you reduce the amount of authority available to code (or leaked via other types), you improve the security of a system. You don’t have to be aiming for capabilities; the effort still helps. In a statically typed language like Rust, the majority of the security enforcement is already in the compiler, as it’s just types and visibility of them.

This question still stands. And the general question of what ways forward there potentially are for this kind of compile-time security enhancement. Some sort of Pre-RFC with things fleshed out? Some sort of spike/prototype? A chat with some security folks?

Why not start by taking the question directly to the security-WG? @bascule