Less restrictive mutability rules, again

language design
1

I'm seeing postings on other forums about new languages whose main feature seems to be a more relaxed borrow checker than Rust. This has come up many times before. I had a go at this back in 2024. The consensus seemed to be that it wasn't enough of a pain point to justify language changes.

But now we are seeing more activity in this area. Everybody has a new "improved Rust" language:

The Ante people identify two problems. One is the scope of a borrow extending beyond the last point of use. See the link above for an example. That's supposed to be fixed in the Polonious borrow checker.

The other is the good old back references problem. That can be handled in Rust with Rc, RefCell, Weak, downgrade, and upgrade. Ir works, but the resulting code is clunky. Dangling weak pointers are possible, although they lead to detected errors. Circular ownership and resulting memory leaks are possible and not easily detected.

So what's current thinking on this? Both problems seem fixable without a new language.

2

This old discussion again. My opinion is still that for the most part what Rust does works well, and it can even help guide you towards a better design, one that avoids pointer soup. Pointer soup is bad for two reasons: the code becomes harder to understand and reason about, and it is usually less efficient to execute since it tends to be cache unfriendly.

Usually some sort of arena design is a good better substitute since it at least increases data locality if nothing else.

3 Likes
3

Once again, I would be the one that offer a broader more open minded perspective. It is crucial to avoid being conservative, instead, one should be innovative, acknowledging both weaknesses and strengths, and not stopping at good enough but keep trying to solve the limitations. A limitation of the "borrow checker" in safe Rust is its inability to facilitate "shared mutability" without incurring a performance penalty (e.g., through Rc or RefCell). I encountered this firsthand while building an arena allocator: without resorting to unsafe Rust, I was forced to pay a performance cost for Rc and RefCell just to implement safe shared mutability, a cost that shouldn't actually be necessary, but arises solely due to Rust’s rigid "aliasing XOR mutability" design

Is there a solution that allows for shared mutability, without using unsafe and without the associated performance overhead? Rust could introduce a new type of reference: one that permits shared mutability while remaining guaranteed safe against issues like dangling references, use after free, unaligned addresses, etc, it would retain all the safety guarantees of standard safe references, with the sole exception of the one from exclusive mutability. This would represent a massive improvement over raw unsafe pointers

4

The new reference is an "unsafe" reference, but it is far safer than a raw pointer

5

there's already a safe shared aliased reference with no unnecessary overhead: &Cell<T>. Sadly it's just not very ergonomic, iirc there's some work on making it more ergonomic by allowing you to easily get &Cell<Field> from &Cell<StructWithField>

6

No, you are quite mistaken. Cell is not the solution the user needs because it does not provide a reference, rather, it employs a "copy on read" mechanism. Cell does not return a reference, every time you read its value, a deep copy is performed, which incurs a performance cost. It is why, it only works for types that implement the Copy trait. This means it can not be used for Vector, String, Box, or other complex, built in types, as doing so would result in an expensive deep copy (malloc call + memcpy). That is not the solution required here

7

Cell can be used without needing to deep-copy allocating types, e.g.:

/// this has basically the same cost as normal Vec::push
pub fn push_to_cell_vec<T>(cell_vec: &Cell<Vec<T>>, item: T) {
    let mut v = cell_vec.take(); // no allocation
    v.push(item);
    cell_vec.set(v); // also no allocation
}

with the Cell field access language feature:

pub struct MyType {
    pub items: Vec<Type2>,
    ...
}

pub fn f(v: &Cell<MyType>) {
    // idk what syntax will be selected, so I picked ->
    push_to_cell_vec(v->items, Type2::new());
}
8

This is fundamentally incorrect, it’s not "sharing," but rather "move", meaning only one instance can exist, you might as well just remove the cell entirely because the whole point of a "shared mutable reference" is to allow mutation without having to move the underlying value

Furthermore, because it operates "by value," you can not create two cells that point to the same vector, so it fails to solve the problem

Furthermore, it doesn't solve the limitation that only one mutable operation can occur at a time, the mutation process is still restricted to happening only once at any given moment

9

That's mostly not true. You do not need to perform a copy, nor need the Copy trait. In addition to what @programmerjake shared, there is also an unstable get_cloned which allows you to perform the "copy on read" without requiring Copy, although this cannot be implemented for all types implementing Clone. Most notably however this allows cloning a Rc<T> out of a Cell<Rc<T>>, which unlocks a lot of usecases. Ante has a similar problem though! There are different Clone traits based on the kind of reference you want to use to clone, and not all types can implement both of them.

Ultimately you cannot get a shared or mutable reference to the insides of a &Cell<T>, but that's also true of the "shared mutable" references in e.g. Ante.

Again, Ante for example has very similar limitations. Their "shared mutable" cannot give you access (any kind of access, including "shared mutable") to the insides of a unstable-stable type (which includes Vec, String, Box and enums, exactly the types you're complaining cannot be used with Cell!).

I should mention though that Ante appears to have "stable" references that allow you get this kind of access, but they have a different tradeoff: you are not allowed to mutate the shape of any unstable-shape type, or any type that could contain them.

Overall Ante and other similar languages are not magic. Most "shared mutable" references have very similar semantics as &Cell<T> with some syntax sugar on top for some "simple" usecases. They show that we can have a much better dev experience using Cell, but fundamentally there are limitations that both have to share and which make writing some kind of code impossible.

10

The getter method for the cell performs a copy. What @programmerjake is doing stems from a fundamental misunderstanding specifically this :

So the unstable get_cloned is not the needed solutiom. It still performs a "copy on read"

Edit: the name _cloned() already communicates it is duplicating the value. Still it is not the solution for the problem that I brought to the table, so kinda irrelevant

That is exactly what I’ve been saying all along: it’s a limitation of Rust. I also proposed a solution for it above

I wasn’t referring to Ante at all, I just wanted to point out a certain mindset, the tendency to think, "This is good enough," and then stop, which reflects a lack of pursuing the best possible solution

I provided my own solution above

11

It's totally sharing, just Vec doesn't have any API for Cell<Vec<T>>:

let v = Cell::new(Vec::new());
let a_bunch_of_shared_references = [&v; 4];
push_to_cell_vec(a_bunch_of_shared_references[0], a_bunch_of_shared_references);
// now v contains a bunch of references to itself

// push some more references, now through a different reference:
push_to_cell_vec(a_bunch_of_shared_references[3], a_bunch_of_shared_references);

you can't have two separate Cell's containing the same (by identity, not equality) value ever. what you can do, however, is have multiple references to a Cell<T>, which is basically like how C++ pointers/references work, the C++ equivalent of &Cell<T> would be something like T * or T &.

That's inherent to having a type (Vec) that is not designed to be modified simultaneously. Languages that don't stop you from modifying something simultaneously generally misbehave when you do, e.g. C++ std::vector has iterator invalidation which means it's generally UB to modify it simultaneously like that. IIRC Java has similar issues, just it causes misbehavior instead of UB. Python also misbehaves, e.g. here on my computer it skips the 3 when iterating:

l = [1, 2, 3, 4]
for i in l:
    print(i, l)
    if i == 2: del l[1]
    print(i, l)

outputs:

1 [1, 2, 3, 4]
1 [1, 2, 3, 4]
2 [1, 2, 3, 4]
2 [1, 3, 4]
4 [1, 3, 4]
4 [1, 3, 4]
1 Like