I have an application where I need to pin all objects in an object graph, at least temporarily, which means that Pin
don't seem like it applies. My hope in this post is to raise awareness, and see if there are any solutions possible. I suspect that this would require some kind of compiler intrinsic or language keyword, like a new type of scope within which all objects are pinned.
My problem is that I want to serialize/deserialize object graphs, and have them reconstituted to be equivalent to the original graphs. For example, if I have an object graph that has cycles in the graph, I want the deserialized graph to have the same cycles. This is not what serde is currently capable of (see the docs for Arc). That said, if you can force all objects to remain pinned in memory, and you can get the address of each object, then you could create a new object that implements the Serialize, Serializer, Deserialize, and Deserializer traits that acts as an intermediate layer between what we're serializing, and the 'real' backend serializer. It would maintain a single object, the data_store: Hashmap<usize, Option<Vec<u8>>>
. The keys are the memory addresses of the object being serialized, and the value is the value of the object after it is has been serialized. At this point, you can perform a simple depth-first search to store everything in data_store
.
- For non-pointer types, you store the address of the object as the key, and the value is the serialized value of the object.
- For pointer types, you do the following:
- Get the address of the pointer itself. If it is already in
data_store
, then continue, otherwise:- Create a new entry in
data_store
with the address of the pointer as the key, andOption::None
as the value. - Recurse through the pointer, restarting this algorithm.
- When you pop back up, replace this pointer's value with
Option::Some(address of the object being pointed to)
- Create a new entry in
- Get the address of the pointer itself. If it is already in
Once you're done with this, you have an object that has no cycles as it doesn't have any actual pointers. data_store
can then be fed through any serde serializer. Deserialization is just doing the above backwards. Note that the memory addresses of the reconstituted objects can be different from the original memory addresses; the only reason I needed the original memory addresses was to act as unique names for the objects that were being serialized.
This also shows how weak this is; if an object is moved while the object graph is being serialized, then its name has effectively changed, breaking all my assumptions. So, is there any way to guarantee that all objects that are recursively reachable starting at some object or pointer are pinned? Given that Unpin
exists, I can't rely on Pin
.
The only thing that I can think of is to change the language so that creates a new scope which marks all objects in that stack frame and below as being immobilized. So, something like the following would be legal rust:
use serde::{Deserialize, Serialize};
use serde_json::Result;
#[derive(Serialize, Deserialize)]
struct Cyclic {
cycles: Option<Rc<Cyclic>>
}
fn serialize(c: &Cyclic) -> -> Result<String> {
immobilize {
serde_json::to_string_pretty(c);
}
}
immobilize
would also have to be nestable, with the internally nested immobilize
scopes being no-ops. Only when the highest immobilize
scope is exited will normal semantics (everything can move) be returned to.