Making the `mut` keyword less misleading

[doublequartz]

The forums just saw yet another ^[1] topic about the confusing behavior of the mut keyword. This was my old pet peeve; Though there have been musings about changing the syntax of either the mutable references or bindings, none have made it out of brainstorming.

The mut keyword is misleading to beginners because it's overloaded with two, subtly different meanings. let mut and & mut looks very similar, so people inevitably assume that they're one and the same feature, when they really aren't.

In another case of overloaded syntax, patterns, there would be a light at the end of the tunnel in which everything will make sense if you study it enough. mut has no such mercy: it gets worse the more you learn about it.

No one to my knowledge has addressed this problem directly. The suggested changes would've eliminated the overloading, but not as its primary intention.

The suggestion to rename the unique/mutable reference is presumably cancelled due to stability reasons, nothing unusual. The suggestion to remove let mut on the other hand have met actual oppositions. Note that even if we concede that let mut is useful, the syntax is still misleading.

If I had my way, I would forbid putting a whitespace between the & and mut to give it an intuitive nudge that &mut is atomic and attempts to understand it as a concatenation of & and mut is meaningless.

Lifetime-ascribed reference might be delimited with a different character:
&'a mut Foo → &'a:mut Foo

I'm aware none of these syntactic changes are likely to happen. References, even limited to mutable and lifetime-ascribed ones, are common enough that the churn becomes hard to justify, though the ability to soft-deprecate syntaxes between editions helps a bit.

Perhaps we should look out for ways to improve the teachability of &mut vs. let mut without changing the language at all. That might be done through lints or learning resources. What kind of plan do you think are viable? I would like to discuss what we can do here.

---

1. This comes up again and again. A cursory search on URLO has revealed 7 whole topics on the matter, with more likely to be found. ↩︎

[m-ronchi]

@doublequartz

Lifetime-ascribed reference might be delimited with a different character: &'a mut Foo → &'a:mut Foo

or &mut 'a Foo to enforce the atomic aspect

[kornel]

I'm afraid that it's too late for major changes to Rust's basic syntax, so something like &'a:mut is going to be difficult. Migration would cause a lot of churn, and for years people would be asking what's the difference between : and space. Although, I do like &mut 'a T.

Warning about & mut sounds like a good idea. Rustfmt already cleans it up, so the warning shouldn't be too noisy for existing users.

[tczajka]

I don't think it's misleading. C++ has the same kind of "overloading":

const int x; // x is immutable
const int *x; // what is behind the pointer is immutable

[kornel]

I don't think that's a good example. C declarations are quirky, e.g. some users wish it was int* x not int *x, and how pointers and qualifiers apply to more complex types gets even trickier.

const in C is weird and misleading, e.g. const char ** pointer is not compatible with a const char *const * function argument.

[tczajka]

@kornel

C declarations are quirky, e.g. some users wish it was int* x not int *x

Just like in Rust, whitespace here is irrelevant. It's true that C declarations have quirky precedence, but the point of the example wasn't about that. It was that const can refer to either the variable or to what the pointer points to, just like mut in Rust.

@kornel

const in C is weird and misleading, e.g. const char ** pointer is not compatible with a const char *const * function argument.

That doesn't seem to be true: compiler explorer.

[kornel]

Sorry, I wrote that from memory and got lost where the consts go.

    // seems like an obvious rule, but it's actually an exception
    char *a;
    const char *b = a;

    // this doesn't compile
    char **c;
    const char **d = c;

or a puzzler why two levels work, but three don't:

void foo1(const char *const *const x) {}
void foo2(const char *const *const *const x) {}

void bar1(const char **x) { 
    foo1(x);
}

void bar2(const char ***x) { 
    foo1(*x);
    foo2(x); // nope
}

Explanation.

[tczajka]

@kornel

    // this doesn't compile
    char **c;
    const char **d = c;

The Rust equivalent also doesn't compile:

fn foo(c: &mut &mut u8) {
    let d: &mut &u8 = c;
}

error[E0308]: mismatched types
 --> src/lib.rs:2:23
  |
2 |     let d: &mut &u8 = c;
  |            --------   ^ types differ in mutability
  |            |
  |            expected due to this
  |
  = note: expected mutable reference `&mut &_`
             found mutable reference `&mut &mut _`

[kornel]

Yeah, Rust &/&mut is much more consistent about it (and mut isn't even in the same game). C needs to be weird about it because it allows mutable and const pointers to alias, Rust doesn't.

[tczajka]

@kornel

C needs to be weird about it

What do you mean? All the examples so far work (or don't work) exactly the same in C as they do in Rust.

[doublequartz]

@tczajka

I don't think it's misleading. C++ has the same kind of "overloading".

That C++ has a given language feature hardly speaks for its clarity─if anything, it indicates otherwise. I was hoping to not have to do this, but while it's clearly not misleading to you, as I've mentioned on a footnote, others need not share your level of understanding^[1]. I think mut is encouraging a particular misconception among newcomers. Because the dichotomy is highly nuanced, it takes significant effort to try and correct the mindset after it has once taken root.

@kornel

I'm afraid that it's too late for major changes to Rust's basic syntax, so something like &'a:mut is going to be difficult.

Reasonable, yet tragic position. Has it gotten worse since #[warn(bare_trait_objects)]?

---

1. Not saying this is the majority or anything. But suffices to say, this happens at least occasionally. ↩︎

[kornel]

@doublequartz

Has it gotten worse since #[warn(bare_trait_objects)]?

Maybe? 2018 edition was an unusually big change, and Rust userbase was ~10x smaller than today. Rust is now also more careful about macro_rules parsing :tt vs :expr which unfortunately makes all syntax changes theoretically breaking, so now we have :expr_2021.

Bare traits were super confusing and dyn fixed that well.

OTOH stronger grouping of & + mut is only a partial solution. The mut binding mode remains. The &mut for exclusive access remains.

It would be nice to solve the mut/&mut confusion.

Maybe using &mut 'a T would work, and we could pretend that the order of mut and lifetime is flexible?

But is this path a good cost-benefit tradeoff?

Will it work, or make users confused why it's &mut:'a T here but T: 'a elsewhere?

Maybe if we're doing mass find and replace, changing &mut to &ex (exclusive) would be better?

[doublequartz]

@kornel

Will it work, or make users confused why it's &mut:'a T here but T: 'a elsewhere?

Fair point that I haven't noticed. Might be another argument for &'a:mut T, but I'll leave that for others to evaluate.

Admittedly, even soft deprecations can turn out to be difficult. Rust has a history of that, too: elided_lifetimes_in_paths is also "deprecated", but recent developments seem to backslide on warning it by default.

[tczajka]

@doublequartz

I was hoping to not have to do this, but while it's clearly not misleading to you, as I've mentioned on a footnote, others need not share your level of understanding[1].

People are confused about different levels of indirection, but that doesn't mean the notation is "misleading" in any way, just that multiple levels of indirection is an abstract, nuanced concept and difficult to grasp by beginners. I doubt a slightly different notation or whitespace restrictions will help with that.

[doublequartz]

@tczajka

const int x; // x is immutable
const int *x; // what is behind the pointer is immutable

You've phrased the distinction between the two forms of immutabilities as above. At a glance this appears consistent, but further analysis can reveal how it isn't.

In particular, immutability as defined by let var fails to inherit through &mut references, allowing it to behave superficially similar to an interior mutability primitive, but not really.

let x = &mut 0; // x is immutable
let mut y = &&mut 0; // *y is immutable

*x = 1; // Succeeds. Even though x is immutable, it happily hands out
        // &uniq access, allowing the dereference.

**y = 1; // Fails. Once you have a shared reference, any
         // transitively reachable fields are also considered shared
         // and therefore immutable without an UnsafeCell.

Explaining this in its full generality requires a notion of precedence, despite there being no binary operators. In a nested data structure of the form var = A(B(C( ... X(Y(Z)) ... ))):

1. If there are any & reference or UnsafeCell, their innermost occurrence determines mutability of Z
2. Otherwise, if there are any &mut reference, Z is mutable
3. Otherwise, Z has the mutability of the place expression var

Since var is guaranteed to be the outermost layer, removing 2. would allow us to merge 3. into 1. as well. Looking from the other direction, this means the presence of step 2. is also responsible for 3.

Is this really where we want to be spending our complexity budget? Removing &uniq access from Rust would allow us to just say "all immutability is transitive". Exposing it for other indirections as you suggest would instead make it even harder to determine mutability of Z. Our algorithm would now look like this:

1. Traverse the access chain outward from the innermost element
2. If you never find anything that changes uniqueness or mutability, Z has the mutability of the place expression var
3. Otherwise, if you find a &, &uniq or UnsafeCell before impl DerefUniq, that type determines mutability of Z
4. Otherwise, continue traversing
5. Z is immutable if you find an & before UnsafeCell

All in all, written in Rust.

Confused? Yeah, me too. This is way harder than an average Rust programmer can be expected to wade through.

Indirections are unintuitive and hard to reason about. That's why Rust has the borrow checker. It lets a Rust programmer treat borrowed values like owned values, without the cost of pervasive cloning.

Why, then, undo this benefit, by separating the mutability of the pointer from that of the pointee? Did anyone in the history of Rust ever explicitly ask for let x = &mut y pattern? I won't be convinced until these questions are answered. Rust with &uniq would be self-consistent, yes, but I know a lot of other impractical things are self-consistent as well.

[tczajka]

@doublequartz

You've phrased the distinction between the two forms of immutabilities as above. At a glance this appears consistent, but further analysis can reveal how it isn't.

It's all consistent.

@doublequartz

let x = &mut 0; // x is immutable
let mut y = &&mut 0; // *y is immutable

*x = 1; // Succeeds. Even though x is immutable, it happily hands out
        // &uniq access, allowing the dereference.

**y = 1; // Fails. Once you have a shared reference, any
         // transitively reachable fields are also considered shared
         // and therefore immutable without an UnsafeCell.

The first y derefence yields a shared object. Hence the second dereference isn't an "exclusive mutable" dereference (since it's not exclusive) and falls back to a shared immutable dereference. So you end up with a shared immutable place after **y.

You'd get the same behavior for user-defined objects if DerefMut was defined to take &uniq self. *y is not uniq so it can't use that, so it falls back to Deref instead.

[doublequartz]

I know this very well. I was simply highlighting the difference between "shared-immutable" and "unique-immutable" which you papered over by just saying "immutable". The two, overloaded use-sites of mut (or lack of it) are not analogous, because there are two different immutabilities in play.

@doublequartz

Why, then, undo this benefit, by separating the mutability of the pointer from that of the pointee? Did anyone in the history of Rust ever explicitly ask for let x = &mut y pattern? I won't be convinced until these questions are answered. Rust with &uniq would be self-consistent, yes, but I know a lot of other impractical things are self-consistent as well.

You did not answer my questions.

[tczajka]

@doublequartz

You did not answer my questions.

OK I will try to answer that question then.

@doublequartz

Did anyone in the history of Rust ever explicitly ask for let x = &mut y pattern?

I don't know the history of this, so I can't say much about how it historically happened. But personally I like knowing that x won't change (i.e. start pointing to something else). So you can say I have "asked" for it. This makes it easier to think about the code. It's for the same reason that I prefer:

let n = x.len();

over

let mut n = x.len();

This way I don't have to think about whether n will be different throughout the function.

When I code in C++ I even type in the keyword const in the right places to get that const-correctness benefit:

const int n = ...;
int *const p = ...;

[doublequartz]

So you think object identity of x is at least somewhat useful. I can understand that. But is it worth the excessive complexity that comes with it, as I have described?

[tczajka]

@doublequartz

So you think object identity of x is at least somewhat useful.

I wouldn't call it that. I don't think it's useful to say that changing a variable changes its "object identity". For example, I would say n is the same object throughout here. It's just a mutable object, so its value changes.

let mut n = 5;
n += 1;

Edit: I guess you're talking about the object identity of *x. Yes that makes sense. It's useful to me to know that *x is the same object throughout.

@doublequartz

But is it worth the excessive complexity that comes with it, as I have described?

Personally I think it's worth it. As proof: other languages (C, Javascript) added const functionality over time because programmers wanted it.

But what I was really disputing are the claims that it's "misleading" and "inconsistent".