Relaxation of reference rules for single-threaded applications

language design
1

I want to start by admitting that I am absolutely not an expert in the field of language design. I am not even an expert at Rust.

I have a problem that I suspect could be solved by allowing the rust compiler to be configurable.

The sample code for this problem can be found here: Rust Playground

This example is contrived. I am familiar with how to solve this problem using traditional Rust idioms.

However, if we consider the error:

error[E0499]: cannot borrow `context` as mutable more than once at a time
  --> src/main.rs:21:18
   |
20 |     for bar in &mut context.bars {
   |                -----------------
   |                |
   |                first mutable borrow occurs here
   |                first borrow later used here
21 |         do_thing(&mut context, &bar);
   |                  ^^^^^^^^^^^^ second mutable borrow occurs here

I wonder what problem is this error solving in this particular program? So, I run off to the Rust documentation (References and Borrowing - The Rust Programming Language):

The restriction preventing multiple mutable references to the same data at the same time allows for mutation but in a very controlled fashion. It’s something that new Rustaceans struggle with because most languages let you mutate whenever you’d like. The benefit of having this restriction is that Rust can prevent data races at compile time. A data race is similar to a race condition and happens when these three behaviors occur:

  • Two or more pointers access the same data at the same time.
  • At least one of the pointers is being used to write to the data.
  • There’s no mechanism being used to synchronize access to the data.

Data races cause undefined behavior and can be difficult to diagnose and fix when you’re trying to track them down at runtime; Rust prevents this problem by refusing to compile code with data races!

But, where is the data race? Where is the concurrent access? I don't see one, and if we aren't solving that problem which problem are we solving? Because the limitation/solution is applied assuming the problem is there. In my application, both this contrived example and what I am developing outside of this example, I am 100% single-threaded and have no concurrency mechanisms in use whatsoever.

It occurs to me that this could be a configurable/setting on the compiler to promise that concurrency is not present, and so this use-case is fine.

2

It's not just data races, it's things like iterator invalidation, non-overlapping memcpy, and alias-based optimizations more generally.

See also this recent topic for some conversation on how being single threaded isn't enough, even if there was an interest in creating a single-threaded Rust dialect (which there generally isn't).

24 Likes
3

Consider what would happen if your do_thing added new elements to context.bars, potentially reallocating it elsewhere, while the calling scope is iterating over those elements.

It's not just about concurrent modification, but also that holding a &mut T means that actions not using that reference won't access that T.

6 Likes
4

Yeah that's an interesting point. It makes me think I'd want a mechanism to grab an immutable reference to the collection/iterator, while being able to make mutable changes to the elements. Which seems fine, I think?

5

I'm not familiar enough with the problem of non-overlapping memcpy and alias-based optimizations to comment on these. I think that thread you linked is interesting though. There's definitely a case to be made for global static mutable being made safe to use in single-threaded applications though. Right now its possible using unsafe.

6

Maybe you're looking for Cell, which allows mutation through shared references, by disallowing sharing across threads.

2 Likes
7

Hmm. I don't think so? Having just now read some of the docs on Cell, it looks like it operates dominantly off of copying. The I'd definitely want mutable access to the value it holds and not copy values around, if possible. I've read some about RefCell, but if I'm understanding properly it moves the problem into runtime and panics there instead of having a compiler error. Maybe I am misunderstanding the usage of these though.

8

It's really like I want a MutVec where I can pass a immutable reference to MutVec such that what the list contains isn't mutable, but the elements in the list are each individually mutable.

9

.iter_mut() or .as_mut_slice()?

10

Doesn't this require a mutable reference to the thing you're calling iter_mut() on?

11

It gives you an immutable "container" with mutable access to the elements. .as_mut_slice() works too and can also give an iterator. Do you also want something else to hold write access to the container part? I'm not sure what one can do with such a view on a container though….shrink_to_fit()? .try_reserve()?

I suspect some GhostToken-like permission split mechanism could work here, but without the power to reallocate, I'm not sure what the separate container access is supposed to achieve.

12

The problem with this is that you would still be able to get shared references to the items from the original collection, then the mutable references yielded by the iterator would allow you to invalidate them.

let foo: Vec<Vec<Something>>  = ...;
let first: &Something = &foo[0][0];
// Pretent this somehow yields &mut references
for mut_ref in &foo {
    mut_ref.clear();
}
// first now points to invalid memory, so this is bad
println!("{first}"});

Also mandatory (IMO) read: The Problem With Single-threaded Shared Mutability - In Pursuit of Laziness

6 Likes
13

Ok, that was an excellent article. It makes me think we should be able to attach qualifiers to the references we take. The ability so say something like:

  • Reference-A cannot be equal to Reference-B.
  • Take a reference to a Vec that allows mutability of the existing elements, but not the length of the list.

So, some kind of semantics that establish qualifications on the references you take, as well as providing those qualifiers via an API for collections like Vec, so that Vec can write conditionally unsafe code.

Edit: the purpose of such features would be to keep the checks at compile time instead of just deferring checks to runtime via RefCell and friends.

14

I'm not so sure about what you're proposing. For example:

This is just a &mut [T], which you can already get from a Vec.

How would those qualifications look in practice? What would they allow that's new? And how would the compiler check for them?

2 Likes
15

I don't know that I have a ton to offer here, at least not much more than misunderstandings of the issue and ramblings on how I think it might be addressed. To the extent that I have any ideas I'll try for guess at what I might do if I was actually in the position of knowing what to do and being capable fo doing it in terms of applying changes to Rust.

struct Foo {
    pub value_1: u32,
    pub value_2: u32,
    pub value_3: u32,
}

fn main() {
    let mut foo = Foo {
        value_1: 1,
        value_2: 2,
        value_3: 3,
    };

    let v1 = &mut foo.value_1;

    do_thing(&mut foo, v1);

    *v1 *= 5;
}

fn do_thing(f: &mut Foo, u: &mut u32) {
    f.value_2 = 10;
    f.value_3 = 10;
    *u *= 2;
}

The above code does not compile, even though there is actually nothing wrong with the program. The compiler is clear about why:

error[E0499]: cannot borrow `foo` as mutable more than once at a time
  --> src/main.rs:21:14
   |
19 |     let v1 = &mut foo.value_1;
   |              ---------------- first mutable borrow occurs here
20 |
21 |     do_thing(&mut foo, v1);
   |              ^^^^^^^^  -- first borrow later used here
   |              |
   |              second mutable borrow occurs here

For more information about this error, try `rustc --explain E0499`.

However, it seems odd because it seems like it could be detectable that we are mutating foo in a way that doesn't violate memory safety, right? It's not like in "do_thing" the alias for "f.value_1" and alias for "u" are actually in conflict. Is my understanding of this right? Am I missing something here?

So, this type of thing seems detectable (again not an expert in this whole langauge design business). And in a similiar aesthetic and even technical mechanism for how lifetimes can be written and elided when their usage is obvious, we could have similar markup to denote the precise accesses a reference can have.

So we could write something like:

fn do_thing(f: &'a @[value_2, value_3] mut Foo, u: &'a mut u32) {
    f.value_2 = 10;
    f.value_3 = 10;
    *u *= 2;
}

Ignore the syntax ugliness. The idea is to convey that the first reference only intends to modify value_2 and value_3. The function should then only be able to modify value_2 and value_3 through that alias. And then it doesn't matter what is passed into the alias "u" because the compiler knows that whatever reference is aliased as "u" must not be in conflict with "f".

I also realize I could have written this another way:

struct Foo {
    pub value_1: u32,
    pub value_2: u32,
    pub value_3: u32,
}

fn main() {
    let mut foo = Foo {
        value_1: 1,
        value_2: 2,
        value_3: 3,
    };

    let v1 = &mut foo.value_1;
    let v2 = &mut foo.value_2;
    let v3 = &mut foo.value_3;

    do_thing(v1, v2, v3);

    *v1 *= 5;
}

fn do_thing(v1: &mut u32, v2: &mut u32, v3: &mut u32,) {
    *v2 = 10;
    *v3 = 10;
    *v1 *= 2;
}

However that gets tedious as the application gets larger. There's a desire I have heard repeated by a lot of new-to-Rust devs where it's desirable to pass around a context object of some kind instead of having to split the context up into itty-bitty-pieces and pass each of its fields in as an individual reference. This would help there. Although you could just get tired of the tedium of writting out @[value_2, value_3] as well. Which would be a far complaint. But we could bundle these into some kind of reference-aspect that a type provides.

impl Foo {
    pub @some_name_here = [value_1, value_2] 
}

...

fn do_thing(f: &'a @some_name_here mut Foo, u: &'a mut u32) {
    f.value_2 = 10;
    f.value_3 = 10;
    *u *= 2;
}

I'm not sure I'm making sense, or maybe I am making sense but I might just lack a ton of knowledge that would make it obvious to me how wrong this suggestion is.

16

This is generally called "partial borrows". There was a topic just a couple days ago on experimenting on them Idea: Experimental attribute based support for partial borrows

3 Likes
17

You can have mutable aliasing on a single thread already if you use the &Cell type. See as_slice_of_cells

18

Cell requires copying and for my case that's unfortunately a non-starter.

19

Yeah, so something like this or adjacent to it is what I am looking for. I'm not trying to be particularly prescriptive with the syntax, just the desire for some kind of split/partial borrowing that has better compile time support and also a desire for more expressive syntax to help with disambiguating the things the compiler might struggle to sort out.

20

No, this doesn't require copying. It's possible to cast &mut [u8] to &[Cell<u8>] at no cost, without copying. cell[x].set(val) is the same thing as slice[x] = val, except noalias guarantees.

3 Likes