Why is a trusted, feature-complete crypto library not a top priority for the Rust community?

[jimmycuadra]

Hello all,

I think that a cryptography library that is well supported and trusted is essential for the development of programs in any language. Crypto primitives are a requirement for a large number of common tasks in our modern, Internet-connected world. Everything from cryptographic ciphers to hashing functions to X.509 intfrastructure is needed for so many applications. It’s been my observation that developing such a library for Rust is being approached, but doesn’t seem to have the sense of urgency from the community that it seems it deserves.

The Rust core team opted not to include cryptographic functions in Rust’s standard library, since a stated goal is to keep a small standard library with just enough to enable any further abstractions and libraries in external crates. I agree with this approach and think it’s a great idea, but it also hurts when there are not “officially endorsed” options for fundamentals like crypto where security and trust are so important. In many of the various applications I’ve started to build in Rust, I find myself being eventually blocked by lack of solid crypto support.

The two projects I’m aware of that have come the furthest towards helping fill this gap are rust-crypto and rust-openssl. rust-crypto is a great project that looks like it will eventually get us some mostly-pure-Rust crypto primitives, but as both the language and the library are new, will naturally take some time to be tested and audited and to become trusted for real applications. rust-openssl appears to be the de facto interim solution, as it provides Rust programs with access to the world’s most-used crypto library. However, the project seems to be managed in sort of an ad-hoc way, with little documentation, no project roadmap, and most development coming from a hodgepodge of pull requests from users that needed one specific thing that hadn’t been implemented yet. It seems like it should be a very high priority for us to at least get rust-openssl into a state where Rust programmers can use OpenSSL functionality safely and reliably without having to shell out to the openssl command line tool.

I realize that Rust is still very young, having reached 1.0 less than a year ago, and that there is a huge amount of ongoing work to improve and further stabilize it, and that not everything can be done at once with only finite resources. I suppose I am mostly curious to hear others’ thoughts on how crypto in Rust should fall on the community’s list of priorities and how we might go about getting ourselves to a better place. I apologize if this seems like a rant or a complaint—I’m very appreciative of everything the Rust core team and community are doing, but I’m hoping to help impassion folks about getting us solid crypto support.

[briansmith]

[I worked on NSS for over three years, and I’m working on a Rust crypto library based on BoringSSL’s crypto primitives: [*ring*](https://github.com/briansmith/ring), webpki and some other stuff.]

it also hurts when there are not “officially endorsed” options for fundamentals like crypto where security and trust are so important.

Be careful what you wish for. If you demand an “officially endorsed” option then you’re likely to end up with politicians picking a library for you, instead of the best technical choice being selected.

In many of the various applications I’ve started to build in Rust, I find myself being eventually blocked by lack of solid crypto support.

What is missing that you need?

I think the problem is lack of awareness of what people are currently building + lack of participation and/or funding by the people who want/need the crypto libs. I recommend that you check out the projects at https://github.com/kud1ing/awesome-rust#cryptography and work with the developers to help them build what you need.

Also, the core Rust teams have a lot of work to do already. If we steal their time to make them pick their favorite crypto library, they will have less time to spend making Rust acceptable for cryptography. For example, in (stable) Rust there’s no way to force alignment like C/C++'s alignas(16), and this impedes progress in making truly bulletproof crypto in Rust. I’d rather have the core Rust teams focus on those core language/library issues so that every kind of library benefits. Accordingly, I think it would be better to shift the discussion to be about what core language/library/toolchain features are needed to build good crypto libraries for Rust.

[jethrogb]

I agree that this is something we’re lacking and needs to be on our list of priorities. I don’t think it should be part of the core team’s responsibilities though.

What’s missing? For example rust-crypto does not do RSA or some other common forms of asymmetric crypto. Of course, to support that, you also really need a good bignum library as well as ASN.1 which are good chunks of coding by themselves.

You’re completely right about the state and quality of rust-openssl. I dare say the code quality is on par with or lower than that of OpenSSL itself. Servo currently uses rust-openssl, so perhaps the Servo developers can spend some of their cycles improving it.

This all is to say that yes, there is support for most crypto operations, but it certainly isn’t robust enough to use in a production environment.

[briansmith]

For example rust-crypto does not do RSA or some other common forms of asymmetric crypto. Of course, to support that, you also really need a good bignum library as well as ASN.1 which are good chunks of coding by themselves.

libsodium does assymetric crypto. ring does RSA, ECDSA (P-256 and P-384), and EdDSA signature verification, and will have signature generation soon. Both also have X25519 and ring also has ECDHE (P-256 and P-384). ring (and webpki) also does all the ASN.1 decoding.

[jimmycuadra]

Be careful what you wish for. If you demand an “officially endorsed” option then you’re likely to end up with politicians picking a library for you, instead of the best technical choice being selected.

Like I mentioned in my original post, I agree with the Rust core team’s decision not to have crypto in the standard library. I was just trying to emphasize that by not doing that, the onus is on the community to do it.

Accordingly, I think it would be better to shift the discussion to be about what core language/library/toolchain features are needed to build good crypto libraries for Rust.

Maybe this post would have been better suited to users.rust-lang.org, since that’s really what I was wanting to discuss.

@briansmith, ring and webpki look awesome! Very exciting stuff. Is it in scope for either of those projects to handle X.509 functionality beyond verification (e.g. certification creation and signing, signing requests, etc.)? One of the specific things I’m interested in doing is managing a private PKI and as far as I know there isn’t anything for this yet in Rust.

[briansmith]

Maybe this post would have been better suited to users.rust-lang.org, since that’s really what I was wanting to discuss.

I agree. I think it would be good to start a thread there.

ring and webpki look awesome! Very exciting stuff. Is it in scope for either of those projects to handle X.509 functionality beyond verification (e.g. certification creation and signing, signing requests, etc.)? One of the specific things I’m interested in doing is managing a private PKI and as far as I know there isn’t anything for this yet in Rust.

The webpki project itself is focused 100% on verification (chain building, chain validation, ocsp validation, CT validation, etc.) that a TLS client would do, and the stuff that the TLS web server would do to facilitate it (stapling, etc.). It won’t have any of the CA infrastructure. If you are looking for an open source solution to the PKI CA infrastructure that doesn’t cost any money, you should consider Cloudflare’s CFSSL, which is written in Go. If you need stuff for managing a PKI infrastructure that Go wouldn’t be suitable for, feel free to contact me privately by email.

[alexcrichton]

Thanks for the post @jimmycuadra! This has definitely been on my mind at least from time to time, and I wanted to write down some of my thoughts on this in addition to those already mentioned here.

From a technical perspective, I would basically just like to have a canonical solution for various crypto needs. If you want to do things like SSL, hashing, or asymmetric ciphers we should have some crate for this in the ecosystem (not necessarily all in the same crate). Unfortunately I don’t think we’ve quite gotten to this point in the ecosystem, but I think that we’re definitely on the way there! My opinion is that once we start to see crates like this emerge we could perhaps consider moving to the nursery and then eventually into rust-lang itself (in accordance with the RFC for rust-lang crates) to have it be a blessed solution for the Rust community at large.

It will likely be quite a long time before we’re comfortable having this kind of functionality in std itself, but the purpose of RFC 1242 was to spell out how the libs team will help manage the ecosystem beyond std as well as even having an almost “extended std” with the rust-lang crates. Progress has been somewhat slow on this front, but we’re still trying to keep the pipeline full!

So talking less long-term, I also have some thoughts about the state of things today. Applications like Cargo and users of hyper need SSL support, and there are already crates doing various kinds of hashing here and there (e.g. crates.io). For this the choice of OpenSSL is usually made (as you’ve noticed), but it comes with the standard drawbacks of what you mentioned, “being OpenSSL”, difficult to find on Windows, difficult to link on OSX, and not even the easiest thing to link to on Linux. I would agree as well that the Rust API of rust-openssl may need an audit, but I wouldn’t necessarily call the situation dire! I suspect that if the libs team talked a bit with @sfackler we could bang out a std-like-quality API in less than a week in addition to drastically improving the usage documentation for how to interface with the library. I know I would personally enjoy auditing the API at least for docs, usage, patterns, etc.

In the near future I’m also pretty excited about projects like @briansmith’s ring, they seem like they’ve definitely got some huge potential in terms of cryptography-at-large in Rust itself. This of course is always a long uphill battle as trusted libs for security have quite high standards (rightfully so).

So with all that in mind, here’s what I think the next steps here should be:

1. Revamp and audit the API surface of rust-openssl. Bring it up to std-like quality and approach it with an eye of stabilization.
2. Try to improve the story for cross-platform crypto (using the native system APIs). For example this may mean a library that uses OpenSSL on Linux, Security.framework on OSX, and the Cryptography API on Windows. I doubt this library would be rust-openssl exactly, and there are bits and pieces of this today in the ecosystem, but it hasn’t all been tied together.
3. Consider moving a crypto-related crate to the nursery. This may be rust-openssl or perhaps even this “cross platform” library, but movement into the nursery will help messaging in terms of “this is the first thing you should try” as well as help improve the API/platform support story over time.
4. In the long term look towards implementing everything in Rust itself (ala ring and friends). Once we’ve got traction of Rust basically everywhere, lots of buy in, and lots of enthusiasm this may “just fall out”

I’m pretty excited to see where crypto goes in the future, and I think there’s even a few things we can do in the short term to greatly improve the experience. Hope that all makes sense!

[briansmith]

Revamp and audit the API surface of rust-openssl. Bring it up to std-like quality and approach it with an eye of stabilization.

Unsurprisingly, we already did this in ring. This is why we have a much better API and implementation than rust-openssl.

I respect the effort of the people working on on rust-openssl, but IMO rust-openssl is a dead-end politically and technically.

Politically, supporting rust-openssl over better alternatives that try to actually use Rust to create safer and more efficient crypto is akin to recommending somebody create a Rust wrapper around Chrome’s Blink instead of using Servo. Instead, it’s better to fix the minor limitations of Rust (e.g. #[align(...)], a clearer definition of the memory model and language semantics, and a gas-style assembler on Windows) so that the libs team can promote Rust crypto libraries that actually demonstrate the advantages that Rust has over C and other alternatives.

Technically, Sodium Oxide (libsodium), parts of rust-crypto, parts of ring, webpki, ring-tls, and other Rust libraries are far ahead of rust-openssl in terms of assurance of safety and/or efficiency, and it would take multiple years of effort for rust-openssl to catch up even if the others stood still. For the same amount of effort, you could instead have absolutely incredible native Rust (and assembly language) crypto libraries.

[jethrogb]

I think you’re vastly underestimating the amount of effort it will take to get the current Rust implementations to be as featureful and as cryptographically robust as OpenSSL, as well as vastly overestimating the amount of effort it will take to build a sane API on top of OpenSSL.

There are two things that make OpenSSL a reasonably good choice for many projects: features and maturity.

The truth of the matter is that people use OpenSSL because it is the only library out there that supports everything they want to do. Sodium is nice but you can’t use it with existing applications. OpenSSL already does everything we want for rust-openssl in terms of functionality.

I agree that in the future it would be very nice to not to have to rely on this bug-ridden library and have pure-Rust implementations, but at the moment many more people are auditing OpenSSL than there would be people auditing some up and coming Rust Crypto/TLS implementation. Beware: being able to have rustc catch memory-safety errors at compile-time does not mean you can suddenly write flawless crypto code.

[briansmith]

[quote=“jethrogb, post:9, topic:3125”] I agree that in the future it would be very nice to not to have to rely on this bug-ridden library and have pure-Rust implementations, but at the moment many more people are auditing OpenSSL than there would be people auditing some up and coming Rust Crypto/TLS implementation.[/quote]

Are you sure? if you look at the recent CVE history and other security-related bug history of OpenSSL, you will see that ring and BoringSSL have generally fixed and/or avoided issues well before they were found or fixed in OpenSSL.

Examples:

• CVE-2016-0701 - Fixed in ring in October 2015. Never affected ring-tls. Fixed in BoringSSL April 2015. Not even reported in OpenSSL until January 2016.

• CVE-2015-3197 - Never affected ring/ring-tls or BoringSSL because it doesn’t do SSLv2. Not fixed until January 2016 in OpenSSL.

• CVE-2015-3193 - Based on OpenSSL’s asessment, we avoided this in ring starting October 2015 when we removed the DH and DSA code. Not fixed in OpenSSL until December 2015.

• CVE-2015-3194 - Completely avoided in ring and webpki because we avoided PSS. OpenSSL wasn’t fixed until December 2015.

• CVE-2015-3195 - This is impossible given the design of webpki, assuming the rust compiler is correct.

• CVE-2015-3196 - Completely avoided in ring-tls by not supporting PSK mode.

• CVE-2015-1794 - Completely avoided in ring-tls by not supporting non-ECC DH key exchange.

• CVE-2015-1793 - We don’t have the tests for this in webpki so I won’t comment on it.

• CVE-2015-1788 - Completely avoided in webpki because we don’t support crazy and unnecessary ECC stuff.

• CVE-2015-1789 - Impossible in webpki because it uses Rust and doesn’t use unsafe (assuming Rust compiler is correct.)

• CVE-2015-1790, CVE-2015-1792 - Completely avoided in webpki because webpki intentionally doesn’t implement the caIssuers AIA extension in certificates. (Incidentally, OpenSSL’s assessment that this doesn’t affect SSL servers and clients is somewhat misleading; it could affect them if they have custom certificate processing code that tries to do ciIssuers AIA fetching.)

There could be problems in ring, webpki, or ring-tls that aren’t in OpenSSL. But, especially in the case of ring, it is unlikely, because we’re for the most part we’re implementing crypto using BoringSSL’s (OpenSSL’s) underlying core crypto code.

Additionally, most (and hopefully all, eventually) changes to the BoringSSL C code in ring get reviewed by Google and integrated into BoringSSL, except for ones that purely remove functionality that Google wants to keep.

Additionally, BoringSSL is reviewed and tested to an incredible level within Google, as far as I’m aware.

I fully admit that ring-tls and webpki need a lot more testing and a lot of review. However, it is much less compared to the remaining corresponding effort needed for OpenSSL, let alone rust-openssl + OpenSSL.

Note that Mozilla and Google (at least, to the best of my understanding) both consider OpenSSL’s certificate validation code (including its ASN.1 parser, which affects a lot of things) to be categorically not fit for purpose in the context of their web browsers. That’s why Mozilla made mozilla::pkix (which is basically the prototype for webpki) and why Google is making their own too. I am sure that Servo will have to replace it with something before Servo is considered production-ready. Note that I created webpki because it was too hard to create good Rust bindings for mozilla::pkix.

[eternaleye]

My opinion is that once we start to see crates like this emerge we could perhaps consider moving to the nursery and then eventually into rust-lang itself (in accordance with the RFC for rust-lang crates) to have it be a blessed solution for the Rust community at large.

I’ll note that personally, I’d consider this a fatally flawed idea. In particular, when issues are found - as with RC4, or MD5, or RSA-1024, or “Export” crypto, or TLS compression, or other stuff found to be weak - it’s crucial to be able to not just disable, but deimplement bad crypto on a short timescale, or you wind up right back at the table with SLOTH and LOGJAM, where dead crypto rises from the grave to threaten the living.

Mixing crypto with stdlib-like stability guarantees is an actively dangerous thing to do. Sometimes, you really do need to just rip out something people are using, because them continuing to use it is worse. Because of that, I feel pulling it into the rust-lang umbrella would send the wrong message - either that the lib offers stability guarantees it doesn’t, or that the broader commitment of rust-lang to stability is weaker than is generally intended.

[jimmycuadra]

I am a bit surprised to read Alex’s thoughts that a crypto library would ever be an official rust-lang crate, let alone in std. Otherwise, I am in agreement with his thoughts. In particular I think it’s important to consider both short term and long term goals. Long term, it’s clear everyone wants pure-Rust-with-some-assembly crypto that covers all bases, but that will simply take time. In the interim, people who want to write application in Rust that do basically any type of networking need something they can use now. I think rust-openssl (or some other replacement for OpenSSL bindings) is the obvious choice just due to the ubiquity of OpenSSL, for better or for worse.

I’ve been contributing to rust-openssl and fixing things as I come across them, but (with all due respect to Steven Fackler and other maintainers) it’s a bit of a wasteland. The code itself is in very bad shape and there is not much activity in the issue tracker. A while back I opened an issue about documentation and project status to try to get some clarification, but it’s sat unanswered. It’s unclear who is really running the show on a project that is pretty essential to the Rust community.

@alexcrichton, I would be very interested in helping with an overhaul of rust-openssl like you describe. Thus far, I’ve hesitated to do things even as simple as adding missing docs because I’m unsure what parts of the existing code were experimental vs. intended to be a stable API. Often I’ve considered just starting my own crate with a nicer Rust API that uses openssl-sys. If the project had some leadership and clear communication around direction, I think it could be drastically improved in relatively short order.

[jimmycuadra]

If you are looking for an open source solution to the PKI CA infrastructure that doesn’t cost any money, you should consider Cloudflare’s CFSSL, which is written in Go. If you need stuff for managing a PKI infrastructure that Go wouldn’t be suitable for, feel free to contact me privately by email.

I am aware of CFSSL and it’s a great project, but this suggestion reinforces that we need the tools to build things like that in Rust! The answer to “can I do X in Rust?” should be “yes” rather than “no, just use language Y.” (Far-forward thinking, of course.)

[briansmith]

I am aware of CFSSL and it’s a great project, but this suggestion reinforces that we need the tools to build things like that in Rust! The answer to “can I do X in Rust?” should be “yes” rather than “no, just use language Y.” (Far-forward thinking, of course.)

The only thing impeding Rust crypto from being great is a lack of funding. People working on Rust crypto are making money doing stuff that has nothing to do with Rust, and then investing it into Rust. That’s not a sustainable strategy. There are multiple people that would build an open-source PKI infrastructure management system in Rust, if you’re willing to finance it. But, otherwise, you’re really better off using CFSSL or something else.

[pczarn]

We can have a crate with all traits needed to use crypto. That way if everyone agrees on an officially endorsed interface, you could easily switch between implementations. Now I’m thinking, would a single interface satisfy the needs of many users and libraries? We can learn from the standard library’s stable hashing interface. One of its major problems is it can’t be implemented in an efficient manner for one-shot hashing algorithms such as FarmHash.

[briansmith]

if everyone agrees on an officially endorsed interface

That’s unlikely to happen.

Also, traits aren’t (always) the best way to abstract over implementation differences, even in Rust. Actually, the more your API avoids exposing traits, the easier and more efficient it is to use, because Rust doesn’t have good support for passing/returning/storing trait objects by value. Because of this, one would have to parameterize the entire API to allow pluggable allocators, which makes something that’s already complicated even more so. Otherwise, the API wouldn’t work with #![no_std] very well.

[sfackler]

I’ve been contributing to rust-openssl and fixing things as I come across them, but (with all due respect to Steven Fackler and other maintainers) it’s a bit of a wasteland. The code itself is in very bad shape and there is not much activity in the issue tracker. A while back I opened an issue about documentation and project status to try to get some clarification, but it’s sat unanswered. It’s unclear who is really running the show on a project that is pretty essential to the Rust community.

Sounds like rust-openssl is an accurate reflection of OpenSSL itself!

In all seriousness, though, some of rust-openssl is a mess, particularly on the crypto side. A lot of this has to do with the crate’s history. I started working on the TLS side ~2.5 years ago since at the time there was literally no way to do TLS stuff in Rust. After considering NSS but deciding not to go with it due to possible bad interactions with green threads (remember those?) as well as absurdly out of date documentation, I wrote up some libssl bindings. The API is IMO okay but not great. There are way too many error types but that’s cleaned up on the breaks branch which’ll land in the next major version bump. The async story’s cleared up there as well. A good amount of the cruft beyond that is in my opinion because of an expansion toward feature completeness. OpenSSL’s TLS APIs have an absolutely enormous surface area, and that tends to get reflected in rust-openssl piecemeal as different people end up needing different features.

The crypto side has been around for a long time. It’s been passed around several times, most recently from @erickt to merge into rust-openssl a bit over two years ago now. I haven’t touched it too much so again, most of the expansion has been piecemeal as people have wanted to do different things.

If you’d like to start thinking about larger scale refactors that would be fantastic - the breaks branch is the right place to put that stuff without needing to worry about back compat. In addition, the impending release of OpenSSL 1.1 will be “interesting”. The APIs are changing pretty significantly (many types are being made opaque, SSlv23Method is going away, etc), that I’m a bit worried about what it would even look like to support 0.9.8 all the way through 1.1.0 in the same library.

[dirvine]

I wonder if anyone has more thoughts on this. In terms of pure rust there seems to be a few options ongoing. It seems rust-crypto is moving again, there are a bunch of bindings (openssl/sodium etc.) and more.

I would like to understand the possibility of a pure rust crypto implementation. Is there an issue with LLVM optimisation and timing attacks, for instance.

In terms of an implementation that can be audited independently, does anyone have more recent thoughts. I wonder about sodiumoxide bindings or rust native implementation of libsodium or moving rust-crypto forward to become a first class citizen in the rust community. It does seem until rust has an implementation that is timing attack resistant that can be audited then we are left with bindings. Anyhow thoughts welcomed and very much appreciated.

[dobenour]

My thoughts:

• ring and ring-tls are the near-term answer. There is literally no way that any Rust crypto lib can match the amount of use or review that BoringSSL gets – and, in cryptography, review matters. This is not image or video processing or HTML parsing where the vast majority of security vulnerabilities are memory-safety violations that are impossible in safe Rust. In cryptography, logic errors are just as likely to be vulnerabilities. To avoid timing attacks, memory allocations, branches, and array accesses must be completely independent of secret data, and in most cases that means independent of all data except length. Hence it is MUCH easier to detect memory safety violations – if they happen once, they will always happen (@briansmith correct me if I am wrong).

• Crypto algorithms get much better speedups from being written in assembly than most. They involve very large amounts of computation compared to memory accesses. There is a very good reason why crypto code is often written in assembler – and it is not just side-channel attacks, but also because the performance can be better. Much better (~60% for AES-GCM using AES-NI vs writing in C with AES-NI intrinsics).

• Rust’s biggest advantage is not for implementing primitives, but for providing safe interfaces to them. Examples include AEAD APIs that statically enforce correct use of nonces.

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.