I saw there has been a lot of discussion about inline assembly, including possibly adding an asm! thing to the Rust language. I think that even if Rust had asm!, there will still be a need to support out-of-line assembly language code. And, it seems relatively easy to support out-of-line assembly language code–easier than adding asm!—by simply ensuring that llvm-mc is included.
My strawman proposal is:
- Ensure that the standalone llvm-mc assembler is always installed, even if the platform already has an assembler like binutils’s gas or Microsoft’s MASM.
- Ensure that
llvm-mcis easy for build scripts (build.rs) to invoke, e.g. by ensuring it is in$PATHwhen the build script starts. - Ensure that the version of
llvm-mcused in Rust is the same across platforms. For example, if Rust 1.7 ships with llvm-mc 3.7.0 on Windows, then llvm-mc 3.7.0 must be used on Linux and Mac. - (Eventually) add support for assembling *.s files directly to Cargo, so that build scripts using the GCC crate aren’t necessary.
Why wouldn’t adding asm! to Rust be sufficient support for assembly language?
-
It is a lot of very tedious work to write assembly language code. By its nature, it is difficult to maintain. Thus, it is useful to share assembly language code across projects as much as possible, in a single form. For example of how having to maintaining real-world assembly language code in multiple formats can go terribly wrong, take a look at the Go version of Intel’s P-256 ECC point multiplication code vs the BoringSSL version of the same code, which is a variant of the OpenSSL version of the code. (I’ve started this thread, partially, to avoid needing to create and maintain a third variant, Rust +
!asm, of that code and ~50,000 lines of similar code.) -
As far as cryptography is concerned, one of the reasons we use assembly language, besides performance, is to ensure that the compiler of the higher-level language doesn’t do things that would leak sensitive information through side channels. In general, we (I) feel more comfortable that this happens when the sensitive code is not touched at all by the higher-level compiler.
-
As far as high-assurance computing is concerned, we want to prove that our tools are correct, prove that our code is correct, and thus prove that the when we feed our code to our tools, the output is correct. This is much easier to do when the assembler is separate from rustc.
-
Usually inline assembler doesn’t support the entire feature set that standalone assemblers support. But, usually there’s a reason that standalone assemblers support those features.
-
It is difficult to understand a program that mixes Rust code, assembler
.macrodirectives, and assembly language code. -
asm!is feature that looks simple but is actually quite hard to do well. It may be better to try other approaches, like the MSVC 64-bit approach of not supporting inline assembler at all, but instead support intrinsics. Supporting external assembly language code reduces the pressure on the compiler team to addasm!before it is ready and before alternatives have been fully evaluated.
Why not just use the native assembler?
-
The native assembler on Windows is MASM. The native assembler on Linux is
gas. The assembly language syntax for each of these is completely different. As a result, the OpenSSL team has written a preprocessing system that takes inline assembly code embedded in Perl (yes, Perl) code and emits the correct assembly language syntax, which is then fed into the native assembler. (Actually, OpenSSL doesn’t support the native assembler on Windows, but only supports nasm, which is even more inconvenient.) Note that this proprocessor code may itself have bugs that make the generated assembly language code incorrect. You can see an example of a fix of a significant bug of this form buried in this BoringSSL commit. Having one assembler syntax per architecture instead of per (operating system * architecture) helps avoid the need for such problematic things. -
The system assembler is often too old. In particular, when Intel and ARM add new instructions, the assemblers need to be updated to take advantage of these instructions. Otherwise, the assembly language code has to encode the instructions as byte sequences. Standardizing on llvm-mc and keeping it up to date minimizes the need for such error-prone manual encoding of instructions. For example, this is what OpenSSL’s assembly language code for AES-NI looks like (note that this is using its Perl preprocessor):
# AESNI extension
sub aeskeygenassist
{ my($dst,$src,$imm)=@_;
if ("$dst:$src" =~ /xmm([0-7]):xmm([0-7])/)
{ &data_byte(0x66,0x0f,0x3a,0xdf,0xc0|($1<<3)|$2,$imm); }
}
sub aescommon
{ my($opcodelet,$dst,$src)=@_;
if ("$dst:$src" =~ /xmm([0-7]):xmm([0-7])/)
{ &data_byte(0x66,0x0f,0x38,$opcodelet,0xc0|($1<<3)|$2);}
}
sub aesimc { aescommon(0xdb,@_); }
sub aesenc { aescommon(0xdc,@_); }
sub aesenclast { aescommon(0xdd,@_); }
sub aesdec { aescommon(0xde,@_); }
sub aesdeclast { aescommon(0xdf,@_); }
Would this be hard to do?
I don’t think so. llvm-mc is already built as part of building llvm as part of Rust. The main risk, AFAICT, is figuring out how well llvm-mc currently supports generating COFF output for the -msvc target. A secondary risk is how to deal with any conflict of Rust-provided llvm-mc with another llvm-mc in $PATH. My suggestion is just to name the rust-provided llvm-mc executable rust-as or rust-llvm-mc or similar, to avoid this.
What projects would benefit from this?
I am mostly familiar with crypto libraries. Any crypto library that wants to maximize performance could benefit from this right away. Especially, crypto code that also wants to have optimal performance on Windows would benefit from this. For example, note how this issue in rust-crypto was closed by disabling the AES-NI optimizations for -msvc targets.
Isn’t llvm-mc too primitive? What about alternatives like gas, nasm, yasm?
Until recently, llvm-mc’s macro support was not that great. However, now it seems to be good enough (or at least there are patches pending which make it good enough) to be useful for sophisticated use cases. And, in particular, I believe it is good enough to reasonably replace the Perl preprocessor for OpenSSL’s code. Also, llvm-mc supports the same macro language for all architectures, so it is possible to write macros that abstract architecture differences, which is useful at least for crypto code. So, despite my initial biases (which were to just use yasm for x86/x64 code, which is actually easier for my projects), I think llvm-mc is the better choice, and that its advantages over other alternatives will only increase over time.
Thoughts?