EDIT: Please forgive the bad example. Unloading libraries is super risky. My issue stems from using Vulkan, which is less risky.
System resources frequently have hierarchical relationships which cannot be adequately expressed using Rust's lifetime system. Consider the simple case of loading shared libraries at runtime. It typically goes like this:
- Load the library.
- Get function pointers from the library.
- Use the library through the function pointers.
- (Optional) Unload the library.
As long as the steps are performed in this order, all is well. Changing the order in any way results in catastrophe, however. In particular, we must never unload the library before we are done using it, since any function pointers after that point will be dangling. (Note that in this scenario, the library owns the functions pointed to by the function pointers. That is, the pointees are part of the library.)
Now, in Rust, the libloading
crate provides an abstraction over OS-specific dynamic linking facilities. The Library
and Symbol
structs represent a loaded library and symbol from some library. To ensure that a Symbol
cannot outlive the Library
it is from, it contains a PhantomData
reference to its source. Thanks to Rust's lifetime semantics, this ensures that steps 1-4 must be performed in order! Here is a simplified example (that does not compile):
Code
use std::marker::PhantomData;
struct Context {}
impl Context {
fn create_handle<'a>(&'a self) -> Handle<'a> {
Handle {
marker_: PhantomData,
}
}
}
impl Drop for Context {
fn drop(&mut self) { /* destroy context */ }
}
struct Handle<'a> {
marker_: PhantomData<&'a Context>,
}
impl Drop for Handle<'_> {
fn drop(&mut self) { /* destroy handle */ }
}
struct App<'a> {
ctx: Context,
handle: Handle<'a>,
}
impl<'a> App<'a> {
fn new() -> App<'a> {
let ctx = Context {};
App {
ctx,
handle: ctx.create_handle(),
}
}
}
However, there is a catch: due to the phantom reference, getting a Symbol
from a Library
borrows the Library
. This means the Library
cannot be moved, even though the reference is phantom and we only care about the Library
's overall lifetime as an object, rather than its lifetime at the memory location when we borrowed it. This severely restricts the ways in which we can organize code. In particular, we cannot store a Library
and its Symbol
s in the same struct, due to the phantom circular reference. Essentially, we can only use them inside the same block.
The case of loading libraries is really the simplest kind. The main reason I bring this up is because I ran into this problem when trying to code with Vulkan in Rust, which is rife with these parent-child relationships. Furthermore, the Vulkan object model stipulates that, unless otherwise specified, children must be destroyed before their parents. The vulkano
crate works around this and offers a safe interface by mandating that essentially everything is wrapped in Arc
. This problem has also appeared in the SDL2 binding for Rust: Trouble with using `Texture` when bound by lifetime · Issue #351 · Rust-SDL2/rust-sdl2 · GitHub. I'm certain that others can contribute more examples.
What we really want here:
- That resources are automatically released in a correct order.
- That using such structs is safe.
- Ideally, that such structs can be specified and constructed safely.
- That no unnecessary allocations are made.
- That such structs are simple to specify and construct.
One potential solution (that would likely be overkill and possibly anathema to Rust's design) would be to enable us to tag values as immovable. This is possible in C++20, since it is possible to return types with no move, copy or assignment constructor. However, all we really need is that .drop()
is called in the right order. There may well be simpler and better solutions. (I am not personally involved in Rust's development, nor am I intimately familiar with its design.)
However, this is overall a big ask. To the best of my knowledge, the following are the only current options for working with such quasi-self-referential structures:
Put everything on the stack. Pros: Safe and efficient. Cons: Makes code really awkward to write. Large allocations may cause stack overflow.
Leak the Library
instance.
Pros: We can give references to it a lifetime of 'static
, allowing us to place its Symbol
s anywhere.
Cons: We give up on the possibility of unloading (and possibly incur a heap allocation). Also, this does not work if .drop()
needs to be called.
Use Rc
or Arc
to replace the phantom reference.
Pros: No more worries about lifetimes. We can place Symbol
s freely. The Library
will get dropped when appropriate.
Cons: We incur a heap allocation and Symbol
now contains two pointers instead of one. It is much harder to tell the order in which objects will be dropped.
Use unsafe interfaces to generate a safe and efficient combined Library
and Symbol
struct using a macro
Pros: Does what we need.
Cons: Doesn't generalize, since there are many more complex use cases. Need to put everything in a macro. Writing procedural macros is tricky, especially since it would need to unsafe itself.
Use rental
, ouroboros
or owning_ref
Pros: Safe.
Cons: They require heap allocation. rental
(abandoned) and ouroboros
use fairly complex macros.
I hope that with this post I can draw a bit more attention to this issue. Alternatively, it would also be good to know if there are any insurmountable obstacles with safety or Rust's design, so I can instead focus on making better workarounds.