I’ve been trying to get to a point where I can dynamically link a Rust crate seamlessly with another Rust crate from a potentially different version of Rust, but it seems like it is currently impossible. I want to get some feedback on whether or not there is a way to accomplish what I am trying to, or else find out what is necessary to get dynamic linking in Rust if there is not.
Note: I do not have deep understandings of either the Rust compiler or Assembly and the way that linking works under-the-hood. I’m doing my best to figure out how things work as I try to solve my problem.
If there is a way to do this already, then this topic should have gone on the Rust users forum, but I’ve done a lot of looking into this and I’m pretty sure that there isn’t a way to do this without compiler changes, so I’m putting this topic here.
Dynamically linking Rust crates to other Rust crates over different versions of Rust is unsupported because of the unstable Rust ABI. Dynamic linking is supported to C libraries because the C ABI is stable, but this doesn’t present a feasible way to link Rust crates to each-other without having to manually define an interface over which they can communicate.
There is a crate, abi_stable_crates, that allows dynamically linking other Rust crates like a kind of plugin, but it requires manually specifying the interface over which the crates can communicate and does not allow the plugin crate full access to the public API of the application crate.
My goal is to allow you to dynamically link to a crate without there being extra limitations or manual effort required to use the crate’s public API like you would if it were statically linked.
To help you understand why I want to do this, my specific use-case is to compile a game engine core, like Amethyst, to a shared library and to dynamically link game code and game mods to the engine. This has the following purposes:
- To drastically improve the game compilation time so that you can start multiple game projects quickly without having to re-compile the 380+ crates that are exactly the same for the different games and mods.
- To enable you to use the game engine core as a library that can be linked into a native Python extension for a Blender plugin.
- To prevent using exorbitant amounts of disc space by duplicating the game engine core inside of mods and the Blender plugin.
- To allow game mods to have full access to all of the structs, traits, functions, etc., that are in the game core so that you do not have to provide a special or alternative API which they must use to extend the game.
The full architecture that I’m trying to achieve can be found here. The advantages of dynamic linking in my particular use-case are impossible to provide through any other means that I can think of.
How to Create a Stable ABI?
Using the C ABI
One thought I had was to have a new Rust library type like
rdylib where Rust will automatically create
extern "C" functions for all of your Rust functions, and provides a way to import the crate like
extern "C" crate crate_name that would automatically create extern blocks so that you could use those functions. I don’t know that this would be a good way to go about it. The advantage would be if you could manage to automate the process without changing
rustc through macros or through a compiler plugin, but I don’t know that you could make that work or not.
Getting Rust’s ABI Stable
If there is no other way to get dynamic linking working, I want to know what it would take to get Rust’s own ABI stable. I saw that a new name mangling scheme was recently merged which seems like a large step in the right direction, but what else is required to get the ABI stable and how does the Rust community feel about the possibility?