A Stable Modular ABI for Rust

[ckaran]

@robinm

Note: if for some reason, a library libA must have some of its public function consumed using the ABI B, and the rest using the ABI C, it is always possible to split it in 3: libA_core would contains all the functionalities, but shouldn't be consumed directly, libB would be a thin wrapper over the part of libA_core that would be consumed with the calling convention B (and depends on libA_core), while libC would be a thin wrapper over the part of libA_core that would be consumed with the calling convention C (and likewise would depends on libA_core).

Two (possibly dumb) questions:

• How do you write glue code? It looks like libA_core is unable to call to libB or libC, so if libB tries to pass through to libC, it can't be done.
• Would this prevent the compiler from doing things intelligently? Under the earlier suggestions, the code (including the representation annotations) are available to the compiler from the start (they are likely to be in the same crate, if not in the same module), whereas this proposal sounds like the compiler will only be able to do linkage-level optimization tricks. If the compiler has full information and access right from the start (with the source code), it may be able to choose a layout in memory that is in the intersection of both ABIs¹, making the translation a zero-cost abstraction.

¹The example I'm thinking about involves byte alignment of certain types. Some ABIs require alignment along some byte boundary, whereas others will permit misalignment. If the compiler is aware that some pair of objects are going to be interrelated in some manner, it may choose to align everything along the byte boundary that both ABIs find acceptable.