[Pre-RFC] Interrupt Calling Conventions

(Most recent version on GitHub)


Summary

Add compiler support for interrupt calling conventions that are specific to an architecture or target. This way, interrupt handler functions can be written directly in Rust without needing assembly shims.

Background

This section gives some introduction to calling conventions in general, summarizes the current support of alternative calling conventions in Rust, and explains why interrupt handlers require special calling conventions.

Calling Conventions

Calling conventions define how function calls are performed, including:

  • how function arguments are passed, e.g. in specific CPU registers, on the stack, as a pointer, etc.
  • how the function returns its result
  • which registers must be preserved by the function
  • setup and clean-up of the stack frame, e.g. whether the caller or callee restores the stack to it's previous state again

Calling conventions are a large part of a function's ABI (application binary interface), so the terms are sometimes used interchangeably.

Current Support

By default, Rust uses an internal "Rust" calling convention, which is not standardized and might change in the future. For interoperating with external code, Rust allows to set the calling convention of a function explicitly through an extern "calling_conv" fn foo() {} function qualifier. The calling convention of external functions can be specified through extern "calling_conv" { fn bar(); }.

The most common alternative calling convention supported by Rust is extern "C", which can be used to interface with most code written in C. In addition, Rust supports various other calling conventions, which are required in more specific cases. Most alternative calling conventions are only supported on a single architecture, for example the "aapcs" ABI is only supported on ARM systems.

Interrupt Handlers

While most functions are invoked by other software, there are some cases where the hardware (or its firmware) invokes a function directly. The most common example are interrupt handler functions defined in embedded systems or operating system kernels. These functions are set up to be called directly by the hardware when a specific interrupt fires (e.g. when a network packet arrives). Interrupt handlers are also called interrupt service routines (ISRs).

Depending on the architecture, a special calling convention is required for such interrupt handler functions. For example, interrupt handlers are often required to restore all registers to their previous state before returning because interrupts happen asynchronously while other code is running. Also, they often receive additional state as input and need to follow a special procedure on return (e.g. use the iretq instruction on x86_64).

Motivation

Since the hardware platform requires a special calling convention for interrupt handlers, we cannot define the handler functions directly in Rust using any of the currently supported calling conventions. Instead, we need to define a wrapper function in raw assembly that acts as a compatibility layer between the interrupt calling convention and the calling convention of the Rust function. For example, with an extern "C" Rust function using the System V AMD64 ABI, the wrapper would need to do the following steps on x86_64:

  • Backup all registers on the stack that are not preserved by the C calling convention
  • This includes all registers except RBX, RSP, RBP, and R12R15 (these are restored by extern "C" functions)
  • This also includes floating point and SSE state, which can be huge (unless we are sure that the interrupt handler does not use the corresponding registers)
  • Align the stack on a 16-byte boundary (required by the C calling convention)
  • Copy the arguments (passed on the stack) into registers (where the C calling convention expects them)
  • Call the Rust function
  • Clean up the stack, including the alignment bytes and arguments.
  • Restore all registers
  • Invoke the iretq instruction to return from the interrupt

This approach has lots of issues. For one, assembly code is difficult to write and especially difficult to write correctly. Errors can easily lead to silent undefined behavior, for example when mixing up two registers when restoring their values. What makes things worse is that the correctness also depends on the compilation settings. For example, there are multiple variants of the C calling convention for x86_64, depending on whether the target system is specified as Windows or Unix-compatible.

The other issue of the above approach is its performance overhead. Interrupt handlers are often invoked with a very high frequency and at a high priority, so they should be as efficient as possible. However, custom assembly code cannot be optimized by Rust or LLVM, so no inlining or copy elision happens. Also, the wrapper function needs to save all registers that the Rust function could possibly use, because it does not know which registers are actually written by the function.

To avoid these issues, this RFC proposes to add support for interrupt calling conventions to the Rust language. This makes it possible to define interrupt handlers directly as Rust functions, without requiring any wrapper functions or custom assembly code.

Rust already supports three different interrupt calling conventions as experimental features: msp430-interrupt, x86-interrupt, and avr-interrupt. They are already widely used in embedded and operating system kernel projects, so this feature also seems to be useful in practice.

Guide-level explanation

In addition to ABIs for interfacing with external code, Rust also supports so-called interrupt ABIs to define interrupt handler functions that can interface directly with the hardware. These ABIs are only needed for bare-metal applications such as embedded systems or operating system kernels. The ABIs are special because they impose requirements on the whole signature of the function, including arguments and return values.

The following interrupt ABIs are currently supported:

  • (unstable) extern "msp430-interrupt": Allows to create interrupt handlers on MSP430 microcontrollers. Functions must have the signature unsafe extern "msp430-interrupt" fn(). To add a function to the interrupt table, use the following snippet:

    #[no_mangle]
    #[link_section = "__interrupt_vector_10"]
    pub static TIM0_VECTOR: unsafe extern "msp430-interrupt" fn() = tim0;
    unsafe extern "msp430-interrupt" fn tim0() {...}
    

    Then place the __interrupt_vector_10 section in the interrupt handler table using a linker script.

  • (unstable) extern "x86-interrupt": This calling convention can be used for defining interrupt handlers on 32-bit and 64-bit x86 targets. Functions must have one of the following two signatures, depending on the interrupt vector:

    extern "x86-interrupt" fn(stack_frame: &ExceptionStackFrame);
    extern "x86-interrupt" fn(stack_frame: &ExceptionStackFrame, error_code: u64);
    

    The error_code argument is not an optional argument. It is set by the hardware for some interrupt vector, but not for others. The programmer must make sure to always use the correct signature for each interrupt vector, otherwise undefined behavior occurs.

  • (unstable) extern "avr-interrupt" and extern "avr-non-blocking-interrupt"

(The above calling conventions are just listed as an example. They are not part of this RFC.)

By using these ABIs, it is possible to implement interrupt handlers directly in Rust, without writing any custom assembly code. This is not only safer and more convenient, it also often results in better performance. The reason for this is that the compiler can employ (cross-function) optimization techniques this way, for example to only backup the CPU registers that are actually overwritten by the interrupt handler.

Reference-level explanation

The exact requirements and properties of the individual interrupt calling conventions must be defined and documented before stabilizing them. However, there are some properties and requirements that apply to all interrupt calling conventions.

Compiler Checks

Interrupt calling conventions have strict requirements that are checked by the Rust compiler:

  • They must not be called by Rust code.
  • The function signature must match a specific template.
  • They are only available on specific targets and might require specific target settings.
  • All other requirements imposed by the implementation of the calling convention in LLVM.

If any of these conditions are violated, the compiler throws an error. It should not be possible to cause LLVM errors using interrupt calling conventions.

Stability

Since interrupt calling conventions are closely tied to a target architecture, they are only as stable as the corresponding target triple, even if the interrupt calling convention is stabilized. If support for a target triple is removed from Rust, removing support for corresponding interrupt calling conventions is not considered a breaking change.

Apart from this limitation, interrupt calling conventions fall under Rust's normal stability guarantees. For this reason, special care must be taken before stabilizing interrupt calling conventions that are implemented outside of rustc (e.g. in LLVM).

Safety

Functions with interrupt calling conventions are considered normal Rust functions. No unsafe annotations are required to declare them and there are no restrictions on their implementation. However, it is not allowed to call such functions from (Rust) code since the custom prelude and epilogue of the functions could lead to memory safety violations. For this reason, the attempt to call a function defined with an interrupt calling convention must result in an hard error that cannot be circumvented through unsafe blocks or by allowing some lints.

The only valid way to invoke a function with an interrupt calling convention is to register them as an interrupt handler directly on the hardware, for example by placing their address in an interrupt descriptor table on x86_64. There is no way for the compiler to verify that this operation is correct, so special care needs to be taken by the programmer to ensure that no violation of memory safety can occur.

Drawbacks

Interrupt calling conventions can be quite complex. So even though they are a very isolated feature, they still add a considerable amount of complexity to the Rust language. This added complexity could lead to considerable work for alternative Rust compilers/code generators that don't build on top of LLVM. Examples are cranelift, gccrs, or mrustc.

Most interrupt calling conventions are still unstable/undocumented features of LLVM, so we need to be cautious about stabilizing them in Rust. Stabilizing them too early could lead to maintenance problems and might make LLVM updates more difficult, e.g. when some barely maintained calling convention is accidentally broken in the latest LLVM release. There is also the danger that LLVM drops support for an interrupt calling convention at some point. If the calling convention is already stabilized in Rust, we would need to find an alternative way to provide that functionality.

The proposed feature is only needed for applications in a specific niche, namely embedded programs and operating system kernel.

Rationale and alternatives

As described in the Motivation, the main alternative to interrupt calling conventions are wrapper functions written in assembly, e.g. in a naked function. This reduces the maintenance burden for the Rust compiler, but makes interrupt handlers inconvenient to write, more dangerous, and less performant.

Alternative: Calling Convention that Preserves all Registers

Many of the advantages of compiler-supported interrupt calling conventions come from the automated register handling, i.e. that all registers are restored to their previous state before returning. We might also be able achieve this using a calling convention that preserves all registers, for example LLVM's preserve_all calling convention.

Such a calling convention could be platform independent and should be much easier to maintain. It could also be called normally from Rust code and might thus have use cases outside of interrupt handling, e.g. similar to functions annotated as #[cold].

Using such a calling convention, it should be possible to create interrupt handler wrappers in assembly with comparable performance. These wrapper functions would handle the platform-specific steps of interrupt handling, such as stack alignment, argument preprocessing, and the interrupt epilogue. Since no language support is required for these wrapper functions, they don't impact the maintainability of the compiler and can evolve independently in libraries. Using proc macros, they could even provide a similar level of usability to users.

While this approach could be considered a good middle ground, full compiler support for interrupt calling conventions is still be the better solution from a usability and performance perspective.

Alternative: Implementation in rustc

Instead of relying on LLVM (or alternative code generators) to implement the interrupt calling conventions, we could also try to implement support for the calling conventions in rustc directly. This way, LLVM upgrades would not be affected by this feature and we would be less dependent on LLVM in general. One possible implementation approach for this could be to build upon a calling convention that preserves all registers (see the previous section).

The drawback of this approach is increased complexity and maintenance cost for rustc.

Alternative: Single interrupt ABI that depends on the target

Instead of adding multiple target-specific interrupt calling conventions under different names, we could add support for a single cross-platform extern "interrupt" calling convention. This calling convention would be an alias for the interrupt calling convention of the target system, e.g. x86-interrupt when compiling for an x86 target.

The main advantage of this approach would be that we keep the list of supported ABI variants short, which might make the documentation clearer. However, there are also several drawbacks:

  • Some targets have multiple interrupt calling conventions (e.g. avr and avr-non-blocking). This would be difficult to represent with a single extern "interrupt" calling convention.
  • Interrupt handlers on different targets require different function signatures. It would be difficult to abstract this cleanly.
  • Interrupt handler implementations are often highly target-specific, so that there is not much value in cross-platform handlers. In fact, it could even lead to bugs when an interrupt handler is accidentally reused on a different platform.

Prior art

The three interrupt calling conventions that are mentioned in this RFC are already implemented as experimental features in rustc since multiple years (msp430-interrupt and x86-interrupt since 2017, avr-interrupt since 2020). They are already in use in several projects and were deemed useful enough that the Rust language team decided to consider this feature for proper inclusion.

There was already a prior RFC for interrupt calling conventions in 2015. The RFC was closed for the time being to explore naked functions as a potential alternative first. Naked functions are now on the path to stabilization, but there is still value in support for interrupt calling conventions, as described in this RFC.

GCC supports a cross-platform __interrupt__ attribute for creating interrupt handlers. The behavior is target-specific and very similar to the proposal of this RFC. The LLVM-based Clang compiler also supports this attribute for a subset of targets.

Unresolved questions

  • What are the exact requirements for stabilizing an interrupt calling convention? What level of stability of the LLVM implementation is required?
  • Is there a way to implement interrupt calling conventions directly in rustc without LLVM support?

Future possibilities

This feature is relatively isolated in limited in scope, so it is not expected that this feature will be extended in the future.

19 Likes

This description looks great to me, including the details about stability.

You may want to add a similar caution about stabilizing interrupt calling conventions for a tier 3 target, since a tier 3 target and its support are more likely to evolve, but we can't evolve a calling convention once stabilized. (That doesn't mean we shouldn't, just that we should exercise caution.)

If the calling conventions already exist, what is this RFC about?

None of those calling conventions ever went through an RFC, and it came up recently that it'd be helpful to have a more precise explanation of them, to put them on a path towards stabilizing them in the future.

6 Likes

Could there be a tiered stabilization for these, similar to other triples?

1 Like

Thanks! I updated the PR with a note about tier 3 targets: Stabilizations for tier 3 should only be done with caution · phil-opp/rfcs@92bf2fe · GitHub

Could you give a few more details on your idea? Do you mean to introduce different stabilization levels such as "guaranteed to work", "guaranteed to build", and "may work", instead of the current stable and unstable levels?

The idea that I had was to mimic the guarantees of the target triples with these new interrupt ABIs.

Since these are closely tied to the target triples it makes some sense to have they be as stable as the triple.

Specifically, it was a thought as a response to the observation that for some platforms the interrupt ABIs might need to change with the platform so this could be a mechanism for allowing the language to do that.

5 Likes

I think we need to differentiate language-level stability from the target tier policy here. Language-level stability means that breaking changes to language features are not allowed after stabilization. This applies to all targets, independent of their tier. The target tier policy is only about the level of official support that a target receives, e.g. through automated CI tests.

The interrupt calling conventions kind of sit in between because they are a language feature on one hand, but specific to a target on the other. For this reason, I proposed that "interrupt calling conventions are only as stable as the corresponding target triple" in this Pre-RFC. What I meant was that there are the following special rules:

  • If official support for a target is dropped, the corresponding interrupt calling convention can be removed from the Rust language, even if it is stabilized. This is not considered a breaking change because no code on other targets is broken by this, since it was not possible to use the calling convention on other targets.
  • Interrupt calling conventions are treated as a platform feature and fall under Rust's target tier policy. That means that the calling conventions are only as well supported as the corresponding target. So on Tier 3 targets, there are no guarantees that corresponding interrupt calling conventions will build, even if they're stabilized. On Tier 2 targets, interrupt calling coventions are guaranteed to build, but no automated tests are run. Only on Tier 1 targets, there is a guarantee that interrupt calling conventions will work.

(I updated the RFC with the above clarifications in Clarify the stability and platform support of interrupt calling conve… · phil-opp/rfcs@a2f8c29 · GitHub)

(emphasis mine)

I don't think that allowing breaking changes to the Rust language depending on the target's support level is a good idea. It would basically mean that a stable Rust compiler does not provide backwards compatibility guarantees on Tier 3 targets anymore.

1 Like

Yes that sounds better.

I meant from your emphasis to mean the "emit" might change but I guess that can already happen in some cases.

2 Likes

Does this calling convention have a stance on the CPU modes when a CPU has more than one mode? For example, on ARM the interrupt handler is entered in Supervisor mode, but most rust code wants to run in User or System mode. A hand-written IRQ handler will switch modes before jumping to the rust level function.

Interesting question! I would not expect a calling convention to handle such application-specific behavior in general. But if there is no way to do the mode switch after a Rust function has been called, I could imagine that we have multiple interrupt calling conventions for such architectures, e.g. arm-interrupt-supervisor-mode, arm-interrupt-user-mode, etc. This would be similar to AVR, where we have both avr-interrupt and avr-non-blocking-interrupt.

in the ARM case the mode can be changed at any time, but some modes have distinct stacks, so a mode switch is "exceedingly" unsafe to put it lightly. Basically impossible to do correctly outside of assembly because you can never tell when the stack pointer is maybe being used.

Thanks for clarifying! Sounds like multiple interrupt calling conventions to switch to different modes are the best solution then. I pushed a small update to clarify that this is possible:

I think this is ready for the next stage, so I created a proper RFC pull request at:

2 Likes