Potential solution for types that shouldn't be moved, like StaticMutex

The current situations is that StaticMutex contains the implementation details (e.g. pthread_mutex_t) and is either used in a static (through its public methods taking &'static self) or in Mutex<T>, as a private Box<StaticMutex> field.

This is done because StaticMutex should never be moved in memory.

One might consider appropriate to add immovable types to Rust, as seen here. But that is largely unfeasible, such types could never be constructed other than by struct literals (or enum variant constructors) and would thus require public fields - not really what you want for Mutex<T>.

I believe I’ve found a middle ground, a rule that could accommodate most existing code, while removing the indirection in Mutex and obsoleting StaticMutex: Such types can be freely moved as long as they haven’t been borrowed. Borrowing observes the address, which is what needs to remain constant for each “object” in memory of that type.

If we use a trait with a default impl and opt-out, it could look like this:

unsafe trait Relocate {}
impl Relocate for .. {}
impl<T> !Relocate for Mutex<T> {}
unsafe impl<'a, T> Relocate for &'a T {}
unsafe impl<'a, T> Relocate for &'a mut T {}
unsafe impl<T> Relocate for *const T {}
// This impl makes Box and Arc implement Relocate implicitly.
unsafe impl<T> Relocate for *mut T {}

struct MyStruct { data: Mutex<Vec<i32>> }
fn make() -> Arc<MyStruct> {
    let mutex = Mutex::new(vec![1, 2, 3]);
    // Calling mutex.lock() (or even just doing &mutex) at this point
    // would borrow the mutex and prevent moving it later.
    let my = MyStruct {
        data: mutex
    };
    // Same here for &my, &my.mutex or my.mutex.lock().
    Arc::new(my);
}

The real problem in designing this lies in the interaction with generics, e.g:

// In std::ptr:
fn read<T: Relocate>(x: *const T) -> T {...}

ptr::read is used by mem::swap which is used by mem::replace which is what Option::take does.

If we end up with T: Relocate bounds just to use Option::<T>::take, the fallout might be too much. I think we should at least try to devise some kind of bound deduction scheme.

One such scheme could be based on argument and return types: if nothing is known about them, assume they implement Relocate:

impl<T> Option<T> {
    // &T always implements Relocate even if T doesn't
    fn as_ref(&self) -> Option<&T> {...}

    // &mut T implements Relocate, but the returned T might not
    fn take_unwrap(&mut self) -> T {...}
}

// &mut T implements Relocate, so T is not restricted
fn take_and_print<T: Debug>(x: &mut Option<T>) {
    // But that means calling take_unwrap will error,
    // as it requires Relocate
    println!("{:?}", x.take_unwrap());
}

I am not certain how this scheme would be implemented in the compiler, but it has the smallest amount of annotation overhead I could think of.

If we can do this, maybe we should have a similar story for Sized - where only localized uses of ptr::read (directly or transitively) that are not reflected in, e.g. the return type, would require a bound.

As an interesting aside, this scheme means that T: ?Sized would never be required, only T: Sized (and that presumably only rarely).

cc @nikomatsakis @pnkfelix @nrc @alexcrichton @aturon

It seems like an uninit pointer-type (don’t remember if there’s another blessed name for the type, no time to look it up right now) could address your concern regarding constructing immovable types?

impl<T> Mutex<T> {
  fn create_in(&uninit self, value: T) {
    // function has obligation to populate `self` before return, or else
    // compilation will fail.
  }
}

That is much harder to design, implement and to use, than “types which cannot be moved after they were first borrowed”. And it solves none of the issues around generics (this late, you want most Foo<Mutex<T>> uses to work, unless they are wrong).

XMPPWocky mentioned in IRC that types that are !Sized also can’t be moved, so I think it is possible to implement NoMove in a library with something like this:

struct NoMove<T>(T);
impl<T> NoMove<T> {
    pub fn new(x: T) -> Self { NoMove(x) }
}
impl<T> !Sized for NoMove<T> {}

And add some Deref and DerefMut traits (and possibly some unsafe operations), and you should be able to use Box<NoMove<T>> to get your desired semantics.

Types that aren’t Sized can’t be placed on the stack or inside structures (except in a tail position, but not even that is implemented properly!).

That makes your solution unusable for Mutex<T> - again, “getting immovable types” is a faux dilemma because truly immovable types are largely useless.

1 Like

@eddyb I mean that all uses of NoMove must appear as a pointer (e.g. Box<NoMove<T>>).

Sample implementation that (probably) works (different to my original proposal, but the same spirit)

use std::ops::{Deref, DerefMut};

pub unsafe trait NoMove: DerefMut {}

pub fn new_no_move<'a, T: 'a>(x: T) -> Box<NoMove<Target = T> + 'a> {
    Box::new(NoMoveConcrete(x))
}

struct NoMoveConcrete<T>(T);
impl<T> Deref for NoMoveConcrete<T> {
    type Target = T;
    fn deref(&self) -> &T { &self.0 }
}
impl<T> DerefMut for NoMoveConcrete<T> {
    fn deref_mut(&mut self) -> &mut T { &mut self.0 }
}
unsafe impl<T> NoMove for NoMoveConcrete<T> {}

playpen

I’m not sure though whether this has a large performance impact due to dynamic dispatch.

I proposed a simpler solution to @alexcrichton that involved implementing a trait for Box<T>, Arc<T>, etc. (each smart pointer that implements Deref<T> and doesn’t move its allocation) and using that in std::sync::mutex to add methods to Box<Mutex<T>>, Arc<Mutex<T>>, and any others (with just one Mutex-specific impl).

The problem is that the approach above cannot replace many uses of mutexes - which are embedded in a structure, itself encapsulated in some allocation that doesn’t move around.

Your sizedness hack is interesting, but appears to be (at least) as limited in practice to the idea @alexcrichton rejected.

Sorry, not really up to speed, just curious about the background - why? It has pointers to it which are not known to the borrow checker?

For what it's worth, my thought was that "immovable types" are unnecessary to design in explicitly: we can rely on the borrow-checker's existing behavior to prevent moves of borrowed values, and use uninit pointers to allow constructing a structure containing a self-borrow:

struct Immovable<'a, T> {
  value: T,
  anchor: PhantomData<&'a mut T>,
}
impl Immovable<'a, T> {
  fn create(storage: &'a uninit Immovable, value: T) {
    *storage = {
      value: value,
      anchor: PhantomData,
    };
  }
}

I do take some issue with:

Immovable types can be used to resolve a significant issue with current Rust that prevents simple translation from other mainstream languages in Rust's target market: they could allow reasonable construction of self-referential structures. (I appreciate the approaches @reem described in that link, but I don't think forcing a user to call a freeze method to work with a structure is an ergonomic solution.)

Making immovable types useful in Rust probably requires &move and &uninit pointers. (As @reem's contribution to the question above showed, Rust already supports immovable types, they just aren't that easy to construct or work with.) &move pointers also solve the issue with generics. (Option<Immovable<_, T>> is difficult or impossible to make work, but Option<&move Immovable<_, T>> should be OK, even without &move being precisely defined: &move is a pointer and its referent is not moving. To try to preempt a potential argument, I don't know if Box<> is changing meaning from meaning "heap-allocated value" to something else, maybe new Box<> obsoletes &move. If so, I think this discussion would still apply with some names changed.)

For what it's worth, it wasn't my intention to argue against your proposal. But I don't yet share your belief that immovable types would be useless or impractical in Rust.

(For reference, previous discussion about &uninit and &move pointer types.)

StaticMutex holds pthread_mutex_t (or an equivalent thereof) which is supposed to be stored in memory, at a significant address - that is, either in a global or an allocation which is not relocated or deallocated before calling pthread_mutex_destroy.

This isn’t a rare pattern in a C API, but most of the time we put a Box around it (or manage an allocation on the C side) and forget about it.

While self-borrowing types are interesting, they have limited value outside of single stack frames and applying that technique in Mutex would break all uses while still requiring out-pointers which we are highly unlikely to get for 1.0.

The self-referential structures you mention also prove my point: moving them in memory is harmless before their address was actually observed, so having them be immovable is overly restrictive (and requires at least out-pointers).

I wonder how much safer you could make an intrusive linked list if you would have the “borrowing freezes the address of the object” semantics apply there.

So there’s one point I’d like to make, and one place where I hope you can clarify something:

  • Self-borrowing can be useful outside of single-stack frames, so long as there’s a mechanism to support constructing such values without relying on the stack. A 'static value should easily be able to contain self-references (any self-borrow would also be 'static, so the lifetime should not be a concern), and a Box<> value could possibly support them through something like:
impl Box<T> {
  fn new_in(fill: F) -> Box<T> where F: Fn(&out T) { ... }
}
  • Regarding your point about self-referential structures, I am having a hard time imagining how such a value might be constructed using the Relocate semantics you describe. Approximating an RAII style would usually mean that the borrow would be necessary as the structure is being initialized, wouldn’t it? I find concrete examples often help, so consider the example from the stackoverflow question:
struct Computer<'a> {
  ram: Ram,
  mmu: Mmu<'a>,
  cpu: Cpu<'a>,
}
struct Ram;
struct Mmu<'a> {
  ram: &'a Ram,
}
struct Cpu<'a> {
  mmu: &'a Mmu,
}

With &out pointers and placement-in, that could possibly be constructed something like:

impl<'a> Computer<'a> {
  fn create(&'a out self) {
    in self Computer {
      ram: Ram,
      mmu: Mmu { ram: &self.ram },
      cpu: Cpu { mmu: &self.mmu },
    };
  }
}
let x: Computer;
x.create();

In this case, the lifetimes of the interior borrows are constrained to equal the lifetime of the containing structure, which would usually be the constraint you’d want for a self-referential structure. I know there’s a lot of detail to the design of &out pointers I’m hand-waving past, but once the basic idea is understood, this isn’t that hard to write or use. At least I don’t think it is, though there is the significant ugliness that making an internally-borrowed value mutable would likely require a Cell.

Still, since I hope to understand the design space, I’d like to ask: How might this structure be implemented if your Relocate idea were adopted? Or is this pattern something you didn’t intend to address?

Thank you

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.