Adding shared mutability, while still maintaining memory safety

language design
1

I believe that rust should add some form of safe shared mutability (without the need of RefCell)

let mut idk = 5;

let shared_one = &shr mut idk;
let shared_two = &shr mut idk;

*shared_one += 5;
*shared_two += 5;
// no errors, because &shr mut is used, not &mut

let shared_two = &mut idk; 
// error, because u shouldnt be able to make an unshared mut reference where a shared mut reference or immutable reference exists

How will this work?

there will be a trait called Shareable (or something else, but lets just use this name), which structs can implement. if a struct implements it, all its members and itself can be shared mutably, and the &shr mut keyword would work on it's instances. a struct cannot implement it if at least one of its members is not Shareable to ensure memory safety. For example Vec won't implement it, since resizing it may delete memory that may still be accessed if its shared mutably. and any struct that has a field of Vec won't be able to be Shareable.

Programmers can bypass this with unsafe, but it should be encapsulated in a struct and using the struct normally should not cause any invalid memory access.

How will this be useful?

Vec would definitely lead to issues if shared mutably, so of course this wont implement Shareable

So the rust standard library can then add another Vec like say, SharedVec that is indeed Shareable. it's implementation would work in a way that shared mutability on it would NOT lead to invalid memory access. it will of course have more overhead, but u would only use this when u need it.

of course, you can't have both &mut and &shr mut to the same reference. so the idea that there can be multiple references but only one &mut applies (but with this concept, mutiple &shr mut).

How about in a multi-threaded environment?

In a multi threaded environment you would still need stuff like RwLock and Mutex because this concept only works with single threaded environments

Why make this? shared mutability is the root of all evil!

  1. It opens more ways to program things that are still memory safe, while still maintaining the "pay only for what you use" idea that rust follows.
  2. It helps new rust devs easily blend into the rust ecosystem, since they would be able to code things more easily like they did in other programming languages, and possibly optimize it later.
  3. It helps with quicker prototyping.
  4. For stuff like game dev, making video games in rust can be frustrating sometimes, and part of it is because of the lack of shared mutability. this concept can help with it.

I know that rust does not seem to like shared mutability, but honestly I think sometimes it's necessary. and having something like &shr mut, the shr part will make programmers know that "oh yeah the value could be changed somewhere cuz it has the shr part"

Why not just RefCell?

RefCell has to do runtime checks, which has overhead, and can even throw errors in scenarios where shared mut wont lead to any issues. So that's: runtime overhead and, sometimes, unnecessary errors

Thoughts? do u think this is a good idea? do you think this is possible? please share your thoughts.

2

How is this different from Cell<T>?

  • zero cost
  • shared mutability
  • can be created from &mut T
  • &Cell<T> can be copied
  • can not be shared between threads
  • can not be converted to &T -- another &Cell<T> may exist, so only supports copying get or replacing the value
7 Likes
3

Cell<T> doesn't solve some cases.

As a form of interior mutability, you have to replace the entirety of it's data (to change state), and get a copy of the entirety of it's data (to get state)

that works great for simple types like i32, i64, f32 etc, but for larger types there will be the overhead of having to copy or replace the entirety of the state. the larger the struct, the more the overhead.

Also, when you use Cell<T>, any modification requires replacing the entire value. This means if two different parts of the program attempt to update different fields of a complex structure, one update could unintentionally overwrite the other. It's rare, but still possible

4

Yes, but that is required for soundness. Creating any kind of reference to the inner value would be unsound, because one could change the value behind the back of said reference through the Cell, which is UB.

How does your proposed solution tackle that?

3 Likes
5

That is solved by field projections, which some crate implement via macros, and admittedly I'd like to get language support for, but this doesn't need a completely new kind of references.

What's more, Cell is even more powerful, since you can use it when only part of the fields are shareable (but you only need them).

6

In my proposal, only types implementing a hypothetical Shareable trait can support &shr mut. Implementing this trait to a struct is essentially saying that modifications to the inner state remain safe , even with multiple &shr mut references.

A type implementing Shareable must ensure no UB arises from having multiple shared mutable references. Thats why i stated that something like Vec wont implement it, but something like i32 would. because changing the value of i32 in a single threaded environment won't cause any UB even if shared mutably. but Vec is a no no, because someting else iterating through it could access deleted memory if a resize occured during the iteration

and for a struct to be able to implement Shareable, All their fields must implement Shareable as well. (unless u go unsafe)

7

that is indeed true. Making cells in only the fields that require mutability seems to be a great approach. though I have a few concerns

what if you wanted a collection to be able to be shared mutably? Cell<Vec> would be pretty expensive if u have to copy the collection everytime u want to check the contents in it. How would you go about this? my proposed way is to make a Vec type maybe called SharedVec which is implemented in a way that modifying it while something else is reading it wont cause UB (with the use of &shr mut)

also, the boilerplate of incrementing a value in a cell, might cause code to be a bit harder to read. eg

instance.cell.set(instance.cell.get() + 1)

you mentioned some crates makes this easier. What are the popular crates that do that?

8

You don't have to copy the collection -- you even can't, since Clone needs a &Vec not a &Cell<Vec> -- but you can swap in something else.

Reusing a recent conversation, one way is this:

10

Late response because I was doing what you were talking about. And yes, using cell does work. the issue is the amount of bloat involved with it sometimes. to return a copy of a value nested in another value (which are stored using cell) took a decent chunk of lines of code

#[derive(Default)]
struct B{
    number: Cell<i32>
}
struct A{
    nested: Cell<B>
}
impl A{
    fn get_b_number(&self) -> i32{
        let gotten_b = self.nested.take();
        let number = gotten_b.number.get();
        self.nested.set(gotten_b);
        return number;
    }
}

would be nice if there was an in built feature to make this shorter

11

Why not? Because these wouldn’t be sendable between threads, shouldn’t there be no issues? It just seems not very useful, since not many extra operations would logically be supported through a shared mutable reference compared to a shared immutable reference.

I can’t actually figure out what the point of the Sharable trait is because any type should be usable through a shared mutable reference as long as &shr mut T can’t be used for &mut self methods (which it shouldn’t be able to, since &shr mut T is very different from &mut T).

What specific use case(s) did you have in mind for this feature? It seems like very few types would support many operations for it, it would require extra thought for library developers. The only thing that I can think of is potentially aliasing mutable function arguments where Cell is too much overhead (which is rare, because the compiler can often optimize away swap + modify a field + swap back), but that seems like a very niche use case for adding a new fundamental type. Responding to your 4 already stated use cases:

  1. Cell already fills this niche, and I think the lack of ergonomics isn’t bad enough to justify a new reference type for how commonly it’s used.
  2. I don’t think it would help much with new Rust devs, shared mutability is rare for most use cases.
  3. Polonius (the new borrow checker) will fix many of the common issues with the current borrow checker that make iteration speed slower, but actual shared mutability is generally rare, including when prototyping.
  4. Game development shared mutability generally involves multithreading, so this proposal wouldn’t help much.
1 Like
12

But how would that be implemented? And in what ways is this new reference type needed for such an implementation (i.e. couldn't it use a normal shared reference?)

13

It's worth explicitly noting that this isn't really true, not with current game engine architecture. Game logic is still largely, almost exclusively, done from a single thread. There's significant noise around improving this situation, since single threaded bottlenecks are not great, but for the majority of gameplay scripting, single threaded is the practical reality.

Game engines do utilize multithreading, but in a very large scale subsystem-based way, e.g. you have the "UI thread," the "Gameplay thread," the "Physics thread," the "Graphics thread," the "Audio thread," etc., and most interaction between these worlds passes through (sometimes hidden from the API consumer) message passing buffers.

Gamedev usage really would love having some kind of &cell T instead of &Cell<T> because of the real ergonomic benefits of a "proper" reference type, but ultimately I agree that it doesn't carry its full weight. Especially since the gamedev usage also needs to use some custom reference/handle type to support any cross-frame references between objects as well as the tangled graph of semi-mutable game state.

3 Likes
14

Vec won't implement it, because u could borrow one of the elements references, and that reference could be invalid if the Vector resizes later, so Vec is not fit to implement the Shareable trait.

Meanwhile for i32, no matter what u do to it (except for, of course, free it), as long as its not in a multithreaded environment, it won't lead to UB, even if other parts of the program are referencing it while it's being mutated.

I agree that not much operations will find this useful, and Cell does solve these issues, however, cell can be very verbose, so it will be nice if there was a much less and more natural way to implement it, not necessarily the &shr mut way. Also, niche areas like gamedev can benefit from it.

#[derive(Default)]
struct B{
    number: Cell<i32>
}
struct A{
    nested: Cell<B>
}
impl A{
    fn set_b_number(&self, new_num : i32) {
        let gotten_b = self.nested.take();
        gotten_b.number.set(new_num);
        self.nested.set(gotten_b);

    }
    fn get_b_number(&self) -> i32{
        let gotten_b = self.nested.take();
        let num = gotten_b.number.get();
        self.nested.set(gotten_b);
        return num

    }
}

These implementations are pretty verbose. and it would be even more verbose, if B did not implement Default. And even more verbose, if it was not a number we were trying to set/get and instead another non copy type i was trying to get or set. however in my proposed way you could do

#[derive(Shareable)]
struct B{
    number: i32
}
#[derive(Shareable)]
struct A{
    nested: B
}
impl A{
    fn set_b_number(&shr mut self, new_num : i32) {
        self.nested.number = new_num
    }
    fn get_b_number(&self) -> i32{
        self.nested.number
    }
}

It's less verbose, and would definitely be more performant.

there now could be an Rc called RcShr that implements a hypothetical DerefSharedMut, if T implements Shareable, that would let u access these methods.

let mut shared = RcShr::new(A::new());
let mut other = shared.clone();
shared.set_b_number(4);
other.set_b_number(6);
shared.set_b_number(5);
// no UB has occured. no error has occured, because the Shareable trait ensures 
//that the types will have no UB even when mutated while other parts of the
//program are referencing it

Now this, i believe, is very useful. i dont have to do

shared.borrow_mut().set_b_number(4); //it has runtime checks

or that long operation i showed with cell

of course, it does not need to be the &shr mut and Shareable concept i put. it could be something similar. It could even just be &mut, but u can only use multiple &muts on a type if it implements Shareable. I'm no programming language design expert so there should be a better way to do this. The idea is that it's not verbose, enables shared mutability, without any UB

15

It is wanted for more than just ergonomics, right? I've seen some interest in projections from &Cell<Struct> to &Cell<Field>, which I think is only possible right now with a bit of unsafe and Cell<T>::as_ptr.

16

This is also wanted for lots of other types too, though -- like MaybeUninit<Struct> -- so the correct answer for it is not a new reference type, but a way to express that for library types without needing a new reference type.

(Because we're not adding &cell and &maybeuninit and &shared_ptr and ...)