Cross-language safer ABI based on Rust?

[m4b]

Yea I’m definitely for that! And I think its actually the way to do it - start off specifying a smaller (and likely easier) subset of the language, nail it down, and then extend when some of the kinks are worked out to the more complex features.

But I’d be also be lying if I said I wasn’t more interested in figuring out a really great abi for a language(s) with generics

[zackw]

I’d encourage everyone thinking about this stuff to read through the “Itanium” C++ ABI document. This was developed originally to promote interop among C++ compilers for Itanic, but is now (as far as I know) used by both GCC and LLVM for all CPUs and all operating systems other than Windows (where the MSVC++ ABI is used instead). I had a small hand in the development of this spec and its implementation in GCC, and my former boss at CodeSourcery (Mark Mitchell) was the lead author.

The most important thing you should notice is that this ABI has deeply seated dependencies on details of the C++ object model, and the majority of the text is devoted to C+±specific issues, such as the multiple vtables required to account for each phase of a complex C++ object’s lifecycle, the rules for type identity, etc. In many places you have to have guru-level understanding of the C++ standard just to know what problem it’s trying to solve, let alone what the solution is. Rust could conceivably make use of the name mangling spec (the primary function of name mangling is not to permit function overloading, IMNSHO, but to render caller-callee function signature inconsistencies across a separate-compilation boundary detectable at link time) but very little else.

I would argue, based on my experience with this spec, that it’s an open research question whether it’s even possible to develop a cross-language ABI for languages with a rich object model. That doesn’t mean it’s not useful to try! But it does mean that you should start very, very small. Try to solve concrete and bounded problems, one at a time. Here’s an example of a concrete and bounded problem in this area: “How can I make Result<T,E>, for some relatively limited set of types T and E (start with () and integers), usable as the return value of system calls when my OS kernel is written in Rust but its user space applications could be in any language?”

[josh]

The specific set of problems I’d like to solve in a cross-language ABI, in order of priority:

• Can we safely passing strings and arrays without either language having to use unsafe code working with the length?
• Can we distinguish between owned and borrowed objects, so one language can call another with a pointer and know whether it can free that pointer afterward?
• Can we support passing and returning tuples and simple algebraic data types (Rust-style enums)?
• Can we support returning a pointer to an object that maintains references to the internals of one of the passed arguments, and thus must not outlive that argument?

Note that for all of these, both languages have the responsibility to propagate the appropriate constraints to the rest of their code.

[m4b]

Every item on this list is awesome

[zackw]

The specific set of problems I’d like to solve in a cross-language ABI, in order of priority:

All of these seem doable to me. #2 and #4 are “a simple matter” of defining a name mangling scheme that encodes all of these properties - the C++ ABI I linked to can already distinguish various kinds of pointers and references, so #2 would be just some more modifier letters. #4 is more involved because you’d have to invent an encoding for lifetimes and you’d have to include the return type in the mangling (C++ doesn’t bother, although it really ought to). I don’t know how this information is currently propagated across crate boundaries, but for a cross-language scheme, name mangling is the path of least resistance since you can continue to use the existing linker.

#1 and #3 involve nailing down object layout, and I had the impression the compiler team didn’t want to do that just yet. Maybe it would be OK for simple things like strings and small-arity tuples and enums containing only scalars.

[bjorn3]

Struct layout doesnt need to be fixed. Object files (rlib) can contain metadata about the struct layout.

Edit: enums can also be supported by adding an abstract vm to determine the discriminant. (This supports null pointer optimization and optimizations of the same kind)

[zackw]

Object files (rlib) can contain metadata about the struct layout.

Specifying that metadata in enough detail that other languages can reasonably use it is probably a lot more work than committing to a fixed layout for some subset of types.

enums can also be supported by adding an abstract vm to determine the discriminant

Same, and now also you have a weird machine and it may be subvertable.

[bjorn3]

Specifying that metadata in enough detail that other languages can reasonably use it is probably a lot more work than committing to a fixed layout for some subset of types.

Rustc already generates and uses that metadata. I just propose to put it in the object files.

[rkruppe]

The metadata rustc emits is entirely unstable and entirely unsuitable for any compiler other than the version of rustc that emitted it. For starters, it contains all sorts of super-internal data structures (HIR, MIR, …), and (AFAIK; at least today) it does not contain layout information, just the data from which layout information is computed.

[eternaleye]

I wonder if one way of tackling this would be to define ABI “levels”

1. Exposing functions from and to, as well as (read-only) statics and constants of, primitive types ([iu]size, uN, iN, fN, [T; n], fn(...) -> ...)
2. Exposing functions that take & and &mut references in their arguments, including slices and str
3. Exposing item-less marker and lang-item traits and impls (Copy, Drop, Send) and allowing types marked with them to be used where primitive types (if Sized) or slices and str (if ?Sized) are
4. Exposing other unary traits and impls, for use with static dispatch (in the side consuming types) or dynamic dispatch (across the FFI boundary)
5. Exposing lifetimes (including references in return types)
6. Exposing unbounded (save for markers?) generics
7. Exposing bounded generics

In addition, for every level N there is a corresponding level Nu, which permits marking functions as unsafe, statics as mut, and (independently) *const and *mut may be used anywhere primitive types may.

• At level 1, you have the “toy examples” of FFI, such as factorial(n: u64) -> u64.
  • Interop is trivial.
• At level 2, you have the “small examples” of FFI, such as character counting, or predicate evaluation.
  • Interop requires enforcement of aliasing and concurrent mutation rules.
• At level 3, you gain String, and other non-parameterized heap types, enabling things like whitespace trimming.
  • Interop requires enforcement of move semantics.
  • May be worth being a bit stricter than Rust here, for the sake of linear languages?
    • On the other hand, no Turing-complete language can actually guarantee linearity. You could always enter an infinite loop, and fail to consume values.
• At level 4, you gain access to Clone, and a host of user-defined traits.
  • Interop requires support for traits, interfaces, multiple inheritance, or some other suitable proxy.
• At level 5, you can take advantage of split_at() without sacrificing safety
  • Interop requires region subtyping.
• At level 6, you gain Box<T>, Vec<T>, and some other useful containers
  • Interop requires (highly constrained) Rust codegen.
• At level 7, you basically have full Rust interop.
  • Interop requires effectively unconstrained Rust codegen.

Most levels would either

1. Add kinds of symbols that couldn’t exist before (like traits) or
2. Add type specifiers to symbols that couldn’t exist before (like references or lifetimes)

As a result, not only should determining what level an API requires be possible merely by examining (mangled) symbols, it should also be possible for a client to only expose the subset of symbols permitted by the levels it supports. This probably requires the mangling expose the return type.

Ideally, it might be possible to make the mangled symbols themselves sufficient to generate bindings from.

[kornel]

As far as I understand lifetimes don’t exist on the ABI level. Calling a function that takes a “thin” reference is the same as calling a function that takes a raw pointer.

For specific lifetime declarations the solution might need to be something closer to source-level descriptions, conceptually closer to C headers.

[eternaleye]

That’s true at the calling conventions level of ABI, but what about the name mangling level? I’d argue these two functions really, really ought to be mangled differently in a “safer ABI”:

fn foo<'a, 'b>(&'a u32, &'b u32) -> &'a u32;
fn foo<'a, 'b>(&'a u32, &'b u32) -> &'b u32;

This takes you from an ABI that allows API creators to go as low as a -7 on the “hard to misuse” scale with lifetimes, and moves you all the way up to a mandatory +9, where the linker won’t let you get it wrong.

It’s the difference between “a one-character typo in a header causes non-deterministic segfaults” and “a one-character typo in a header causes the build to fail.”

[eddyb]

I think those have different mangling if the type is still involved, because the hashing (which TypeId thus Any also uses) is supposed to take those lifetimes into account (after canonicalizing them), so 0 vs 1 will be hashed at one point.

[Storyyeller]

I think nailing down the unsafe code guidelines for Rust would be a prerequisite to any effort like this. You can’t expose & and &mut in an ABI if you haven’t even decided what they really mean yet.

[zackw]

I’d argue these two functions really, really ought to be mangled differently in a “safer ABI”: […]

Concur.

the hashing (which TypeId thus Any also uses) is supposed to take those lifetimes into account

I don’t think a cross-language, baked-in-stone mangling scheme should be based on hashing. It’s both harder to extend compatibly. and harder to debug.

[eddyb]

I agree, I was simply stating the status quo.

[eternaleye]

I disagree - in safe code, the meaning of & and &mut are quite well-defined: & is multiple aliases may exist but mutation may not occur while borrowed, and &mut means that only one such reference may exist, and mutation can occur through it.

Unless the Nu levels are implemented, this is sufficient, and work on this can proceed.

Furthermore, even aside from that, a “safer ABI based on Rust” does not necessarily need to have exactly Rust’s definition of unsafe - it may, in fact, end up being prudent to ban the Nu levels entirely, or say that the aliasing/mutability guarantees of references from this ABI may never be violated, even within unsafe.

Either of those would suffice.

[Storyyeller]

I disagree - in safe code, the meaning of & and &mut are quite well-defined: & is multiple aliases may exist but mutation may not occur while borrowed, and &mut means that only one such reference may exist, and mutation can occur through it.

I think things are a lot more complicated than that. In particular, your definition doesn’t actually hold, even in safe Rust.

In particular, multiple mutable references to the same object can exist at the same time, it’s just that the borrow checker ensures that you can only use one at any given time. Anyway, unless you propose to have a standardized cross-language borrowck, this isn’t particularly helpful for defining an ABI.

Apart from that, unsafe code is a reality and any realistic proposal would have to deal with that. You can’t even write the Rust standard library without unsafe code.

Anyway, it seems like the safe Rust <-> unsafe Rust “abi” would be the ideal case for working out compatibility concerns (starting with the fact that it’s not actually an ABI). If we can’t do that, there’s no hope of making languages interoperate.

[eternaleye]

There’s a reason I specified #2 and #5 in the order (and at the distance) that I did: #2 permits one to pass access to data to functions without passing ownership (and does not permit retaining that access). Meanwhile #5 does indeed require a full borrow checker. That’s why it’s the last step before the ones that require Rust codegen.

The onus #2 places on the client language is quite small: It must ensure &mut arguments to functions do not alias and are not observed during the execution of the function, and it must ensure that & arguments to functions are not mutated during execution of the function.

This is really no worse than restrict in C, with the added benefit that making the mangling capture it allows client languages stronger than C to actually enforce such guarantees statically.

[cuviper]

it must ensure that & arguments to functions are not mutated during execution of the function.

This may be a nitpick, but would you exclude UnsafeCell and its derivatives from this ABI? (wherein & only refers to sharing, and internal bits can be mutated.)