Is anybody tracking what Swift is doing with their ABI?
I wonder to what extent an extern "swift" would be possible in Rust. This would be amazing for macOS/iOS, where currently it requires an awkward path of Rust -> C -> ObjC -> Swift (or direct use of ObjC runtime from Rust, which isn’t nice either).
I know Rust’s own stable ABI is far far off, so maybe some Swift interop could also be a way to have some (subset of) stable binary ABI for Rust itself?
How far fetched of an idea is it to imagine an item layout API for the compiler? If a procedural macro crate could provide even a limited #[repr(swift)] or #[repr(cplusplus)], that gives Rust a far better interop story than any language I can think of.
Swift’s implementation of generics is significantly different – there was a talk on this at LLVM’s 2017 dev meeting: https://llvm.org/devmtg/2017-10/#talk15
It seems to me this would be a huge hurdle to getting Rust and Swift talking. Maybe you could do without generics, but then I’m not sure you’re much better than extern "C".
That’s a very helpful link, thanks!
I understood the talk correctly, I still think it should be possible to have some interoperability with Swift’s generics, just without true monomorphisation. Rust’s monomorphised version would just be a still generic-ish code calling Swift’s runtime.
Swift provides “witness tables” for generic types that describe their size, alignment and a sort of vtable for basic operations like copy or destroy. I don’t know if that can be made to agree with Rust’s move semantics. I suppose in the worst case non-Copy objects will all need to be SwiftRc<T>.
I’m not entirely sure how dynamic the tables are. Can the compiler read size/alignment, or is it really so dynamic that it could change at run time later? Lack of const fn size_of could be a problem for Rust. But OTOH Swift does monomorphisation as an optimization.
And finally Swift allows generic types like struct Pair<T> to be treated as opaque types accessed only via witness tables. For Rust that would be a question whether it should be exposed explicitly as methods or whether “direct” field access could be actually dynamic for #[repr(swift)] (desugaring to calls in MIR?).
And finally one thing I’ve noticed that the ABI does not define generic type bounds (where T: Eq). It assumes that for type checking the compiler has access to the source. That’s a bit of a bummer, because that means generics won’t be safe to use only based on ABI alone, and will need equivalent of bindgen to get the bounds.
And the other way, AFAIK, making Rust code usable from Swift requires “just” generating witness tables for everything (including a witness-table-based instantiation of each generic function). It would be the slow dynamic path without monomorphisation. Since Swift doesn’t have borrow checker, possibly only Copy and Arc would be safe to use in this context?
It would be neat if the LLVM cousins (Rust, Swift, Kotlin, etc) could one day in the distant future inter-operate like the JVM languages do.
Swift ABI is now stable, and will ship with the next macOS/iOS release.
This is from a while ago, but actually Swift does encode generic bounds into a compiled function. Specifically that is encoded as part of the mangled name of the function.
e.g. $s4main3foo1xyx_tAA1XRzlF is the mangled name of a function: foo<T>(x: T) -> () where T: main.X, in a module named "main".
Unfortunately this means that in order for Swift and Rust to interoperate directly the Rust compiler must have a Swift name mangler, in addition to calling convention support and everything else.
I don't think this is avoidable, e.g., if we wanted to interoperate with C++, we'd need a C++ name mangler as well, and to interoperate with C, we currently already have a name mangler for it (it is just a trivial one).
BTW, how would that look like? Currently we have (IMHO unnecessarily split) extern "C" and #[no_mangle].
if there was extern "swift", would it also require something like #[mangle(swift)] to work?
IMO #[no_mangle] just means to not mangle a symbol name. You could use that on a Rust function, and call its symbol from wherever, and it would work using the Rust calling convention, if it panics a Rust panic is raised, etc. Right now, extern "C" is a bit stronger than that, in that it does not only use C name mangling (#[no_mangle]+extern "C" should IMO be unnecessary) but it also makes sure and assumes that the C ABI will be followed, which allows us to warn about non-repr(C) types being used, etc. For C++ and Swift I'd just expect extern "C++" and extern "Swift" to do a similar thing (following these languages platform ABIs, throwing the appropriate exceptions, etc.). I could imagine repr(C++) and repr(Swift) attributes that allow you to, for example, give some subset of Rust enum the layout of a C++ std::variant type or of a swift enum, and if you try to use an incompatible enum in a extern "C++" function, you'd get a warning / error alerting you of that.
I get about ABI and repr, but if the current philosophy of separating ABI from name mangling is kept, then extern "swift" functions won’t be able to call any actual swift functions, because they won’t match their names (extern "swift" foo() will compile to __ZN3foo3foo17, and not $s4foo$swiftysomething)
So either extern has to combine ABI and mangling, or there needs to be more mangling special attributes to go with it.
This is what I argue for: extern "X" should imply X's mangling scheme.
extern "C" fn is very useful for defining functions that are passed as callbacks into C APIs, you don’t want to have to make up a unique name for these just so they can be #[no_mangle], marking them as #[no_mangle] pub is how you explicitly export them to the global namespace.
From what I remember (it’s been a while since I’ve touched FFI) functions imported via extern "C" {} are implicitly #[no_mangle], so if there was support for importing Swift functions via extern "swift" {} I would expect those to have the appropriate mangling applied. (While defining Swift ABI functions via extern "swift" fn should just change the ABI and a separate step should change the mangling to make them globally visible).
I’ve argued in the past that it can be automagically solved by making private extern "C" functions mangled with Rust’s private scheme, and C-mangle only publicly exported functions (pub extern that is in a publicly-accessible module). This way you can have C callbacks with any name you want, and Rust’s normal privacy rules and namespacing applies.
You could potentially be making an extern "C" fn publicly visible to be consumed by other Rust code (maybe it’s used as a callback fn argument in a sibling macro). I guess that could be niche enough that it should be handled by #[mangle(Rust)] pub extern "C" fn foo(), but it may be backwards incompatible to change it (and having extern "C" fn and extern "swift" fn inconsistent seems like a bad idea).
Actually, I have spent a bit of time on thinking about supporting the Swift ABI from Rust:
Todo list for Swift ABI (search LLVM for “swift” PRs):
self pointer arguments (LLVM PR).#[export_name = "$s..."] everwhere#[no_mangle] won’t work due to the “$s” prefix on all normal Swift symbolsfn({i32, f32}, i32) and fn(i32, {f32, i32})
are both represented as fn(i32, f32, i32)
Would it be possible to define some kind of generic attributes for non-standard calls and offload the actual understanding of the swift abi to an external tool, possibly with something like procedural macro?
Adding extern "swift" and #[mangle(swift)] means declarations of the swift functions in rust syntax are still needed, so there would probably have to be a tool that will generate them from the actual swift code similarly to what bindgen does for C.
That tool could then understand the mangling, and emit the #[export_name = …]s, but maybe similar approach could be used for the other features. An attribute for adding the llvm attributes, and maybe even some kind of llvm IL template that would take care of the parameter exploding?
That way the implementation of the actual swift ABI could be developed outside the actual rust compiler—without having to stick to its release schedule and being slowed down by the long CI runs—and independently of support for further ABIs like C++.
Coming back to this…
Would now be a good time to begin working on this? Or rather, as the canonical first step of working on a Rust compiler feature is to create a tracking issue, should a tracking issue be created for Swift ABI support?
I think for this, if experimentation needs to be done in-tree (i.e. it needs access to compiler-internal information), someone should write an eRFC (experimental RFC) for what the benefit is and what it would look like.
If an ABI shim can be created outside of rustc, then an informal working group should probably be formed to experiment out-of-tree, as this will allow work to progress at its own pace rather than being dependent on rustc CI.
I still think that an approach that allows for a minimal compiler plugin to define ABIs (i.e. what extern "plugin" means) would be the best way to go, which might even be an #[extern(plugin)] proc macro system.