rdylib is a hypothetical compiler option to create a Rust binary that can be loaded and unloaded at runtime by another Rust program. It will communicate with the host program via a Rust native ABI. The goal is to ensure that if both program are proved safe, and the compiler/loader checked that the ABI constraints are fulfilled, when they are working together, the system is still safe.
Today, Rust still not able to dynamically link two binaries written in Rust, except from
unsafe C ABI.
This is the biggest obstacle towards replacing all unsafe C programs with Rust. The Redox OS team has been forced to use static linking from the beginning. Recently, I just saw a Gentoo blogger blamed Rust for its lack of ability to dynamic linking.
It would not be possible to ship native dynamic linking in 2021 edition. However, if we do not start working now we would not see it in 2024 edition. So I really want to see it become the first priority for the post-2021 development.
The following is some thoughts about how to achieve this.
Apart from the variants of C ABI, the only popular enough alternative is the COM ABI which is Windows only. I haven't dig into the relatively new Swift ABI and have no comment on it. But I am familiar with the COM ABI, which is based on passing
Rc if using single thread appartments) everywhere in Rust's point of view. In the day the COM ABI was invented, dynamic dispatch was started to be popular thanks to SmallTalk, C++ and OOP.
It is interesting because there was no successful attempts that bring generics to ABIs. I can see the huge difficulty - if we allow generics to appear in the ABI exported entry points, we need a code generator that runs at link time.
So, I guess the Rust native ABI should be more like the COM ABI which is a subset of the language without generics. However, this ABI still should support the following:
- Lifetimes. This is a big challenge since we know lifetimes were erased at compile time. But without lifetimes, we lost the ability to check that the ABI is in fact safe at load time, and this would be no better than an unsafe C ABI. so this is what makes the ABI different. However, any progress on this direction would be very beneficial - the same idea can be used in Web APIs, RPC APIs etc.
- Dynamic dispatch. The COM ABI proved this can work, so we should be able to do the same.
(I didn't put
async in the list because I think it should be implied by the above)
On top of the lifetimes in the ABI, the unsafely today exists in many dynamic linking systems includes dangling pointers to DLL owned data when the DLL was unloaded. This creates another big challenge: a pointer that is
'static in a DLL is not
'static in the host program.
Having the issues I described here, I believe we will overcome it and have a good solution.