Custom LLVM calling convention


Rust uses aggregate function arguments and return types (structs and enums) a LOT. The C calling convention is suboptimal in this case, passing the aggregates on the stack even when they could be in registers if the calling convention were different.

Swift (which I believe has the same problem) uses a custom calling convention. It is suboptimal for Rust, though, because it involves a special register used for error handling which Rust doesn’t need. Have there been any thoughts on a custom calling convention specifically for Rust?


Rust does in fact use its own calling convention – but it’s handled sort of “before” the LLVM level. That is, if the Rust type is Foo, we will often tell LLVM to expect a Foo*, essentially.


That is still less efficient than my proposal, in which Foo would be returned in registers unless it is big.


We can pass arguments and results in registers without adding a custom calling convention to LLVM. In fact, rustc already generates reasonable results if you request the C calling convention:

pub struct X(*const i32, *const i32);
pub extern "C" fn f(x: X) -> X { x }


	movq	%rdi, %rax
	movq	%rsi, %rdx

Of course, it’s kind of pathetic that Rust’s default calling convention is worse than the C calling convention… but that can be fixed without a custom LLVM calling convention.


Since Rust uses LLVM fast-call, shouldn’t that be reported to LLVM’s developers?


I meant that rustc is making bad decisions about what structures to pass/return by pointer, not that LLVM is doing anything wrong.


What about a struct with four int64s? That can be returned in registers on x86-64 (at least if the ABI provides enough caller-saved registers), but I doubt that fastcc will do that.

As for whether a custom calling convention is worthwhile: fastcc is still optimized for C-like code, which usually passes aggregates by reference. Rust often passes aggregates by value, so it would be nice to optimize this case. I suspect this is one of the reasons that Swift uses a custom ABI. Note that GHC and an LLVM backend for Erlang HiPE also have custom calling conventions.


Is anything being done or are there plans to fix this? It seems ludicrous that if I want to strongly type an int by wrapping it in a struct all simple functions on it now have to read and write memory. EDIT: No, actually all is fine, I was looking at wrong piece of generated code.


It doesn’t always force it into memory.

pub struct Foo(u32);

pub fn extract(num: Foo) -> u32 {

Gives me:

	movl	%edi, %eax


Ok sorry, ignore my comment, I was looking at the wrong piece of generated code where I was passing it by reference, doh!