Local closures borrowing ergonomics

I think within this thread what we want is guaranteed-inlined closure, otherwise we'll get mutable alias.

The only bad part of the macro I see is its syntactical noise, or ugly in short. And I believe it would be less ugly with macro 2.0 syntax(TBA).

let mut add = |v| acc.push(Ok(v));
// vs
macro add($v: expr) => { acc.push(Ok($v)) }
// vs
macro_rules! add { ($v:expr) => (acc.push(Ok($v))) }
3 Likes

One downside is that macros don't actually capture acc -- that's how they avoid aliasing here, but it also means they're vulnerable to shadowing at each call site.

edit: nevermind -- macro hygene works!

My proposal was going to be something like this:

let mut acc = Vec::new();
macro fn add(v: _) { acc.push(v); }

Everything is pass-by-value by default, but you could have pass-by-name too:

macro fn iif(condition: bool, macro then: _, macro otherwise: _) -> _ {
    if condition { $then } else { $otherwise }
}

impl<T> Option<T> {
    macro fn unwrap_or(self, macro fallback: T) -> T {
        match self {
            Some(v) => v,
            None => $fallback,
        }
    }
}
3 Likes

Agreed, it would be nice to be able to express that in Rust. For instance, in C++, I have used local lambdas many times to simplify code like this.

Interesting. Maybe a solution based on this would be to allow access to the closure's variables, something like:

let mut acc: Vec<Result<Item, Error>> =  Vec::new();
let mut add = |v| acc.push(Ok(v));
if cond1 {
    add(x1);
    add(x2);
}
if unlucky {
    add.acc.push(Err(e)); // :-/
}
if cond2 {
    add(y3);
}

Though this would become complicated with Rust 2021's disjoint capture in closures.

I could imagine a Rust where using variables/members mentioned in a closure reborrowed them from the closure's state while necessary rather than the originally mentioned binding, to allow code like the discussed to work. Using a keyword such as pub to opt in to this behavior also doesn't seem completely foreign.

The problem is the "while necessary", though. The rules for when it would reborrow from the closure are way more complex and reliant on later borrow checking passes than I would like. How a closure borrows its context is already subtle, but this is controlled by just the body of the closure, and can be avoided by using move.

If this does happen, I think it should always be opt-in by a keyword (e.g. pub), the closure should be treated as dropped with side effects (i.e. it's dropped at end of scope rather than being invalidatable by latter borrows), and all mentions of the borrowed bindings should be reborrowed from the closure until the closure is known-moved-from (i.e. has been dropped on all code paths).

But this still doesn't work for the explicitly reborrowed bindings pattern ({ let captures = (...); move || { let captures = captures; ... } }) because the binding(s) used by the closure is different from the binding(s) used by the outer scope.

With all of the complications on making closures smart enough to share borrows in this way, I think I'm fine with saying that the right way to handle this is macros 2.0, which do express temporarily borrowing from their environment just for the executions quite easily and natively.

3 Likes

I could easily see capture borrows continuing to be based on strictly the closure body, and simply errorring later if the capture borrow is not strong enough. Thinking on it, there is existing syntax which does this:

let mut a = ...;
let mut clos = || {
    let a = &mut a; // ←

    // use only `&a`, so the closure would capture only `&a`
    println!("{}", &a);
};

clos.a.mutate(); // errors without `let a = &mut a;` above

This is one of the first papercuts I encountered when learning Rust years ago, and I've run into it multiple times since. I'd be a big fan of making this code "just work".

I see a connection between this and two-phase borrows. The borrow created when creating the closure could be seen as the "reservation" phase of a two-phase borrow, whereas calling the closure would cause the "activation".

Doing anything else with the closure, such as passing it to an external function, could also be seen as causing activation. So, for example, this useless code would be allowed:

let mut foo = 1;
let my_closure = |x| { foo += 1; x };
foo = 2;
some_iterator.map(my_closure);

…but passing my_closure to some_iterator.map() would be considered to borrow foo (or rather, activate its deferred borrow), not just borrow my_closure itself.

Given that AFAIK rustc still only emits noalias attributes on function arguments, I'm not sure there's any situation where this could cause a problem today. If there is such a situation, or if rustc emits noalias attributes more aggressively in the future… well, it could avoid emitting noalias in the these situations, though admittedly that's a form of feedback from the borrow checker to code generation, something that I believe doesn't currently exist.

At least for me, it's common to want more than one closure in the same function. I don't see any way this could be compatible with a "main function must borrow variables back from the closure" approach, whether that happens explicitly or implicitly.

4 Likes

A solution would be nice, since today I was hit by this:

use rand::Rng;

// Error: cannot borrow `*rng` as mutable because previous closure requires unique access
fn first_attempt(vec: &mut Vec<i32>, rng: &mut impl Rng) {
    let mut push_random = || vec.push(rng.gen());
    
    push_random();
    
    while rng.gen_bool(0.9) {
        push_random();
    }
}

// Error: type annotations needed
fn second_attempt(vec: &mut Vec<i32>, rng: &mut impl Rng) {
    let mut push_random = |rng2| vec.push(rng2.gen());
//                                        ^^^^ consider giving this closure parameter a type
    push_random(rng);
    
    while rng.gen_bool(0.9) {
        push_random(rng);
    }
}

// Error: `impl Trait` not allowed outside of function and method return types
fn third_attempt(vec: &mut Vec<i32>, rng: &mut impl Rng) {
    let mut push_random = |rng2: &mut impl Rng| vec.push(rng2.gen());
//                                    ^^^^^^^^
    push_random(rng);
    
    while rng.gen_bool(0.9) {
        push_random(rng);
    }
}

// Works
fn fourth_attempt(vec: &mut Vec<i32>, rng: &mut impl Rng) {
    fn push_random(vec: &mut Vec<i32>, rng: &mut impl Rng) {
        vec.push(rng.gen());
    } 
    
    push_random(vec, rng);
    
    while rng.gen_bool(0.9) {
        push_random(vec, rng);
    }
}

which is not great...

2 Likes

...but this doesn't yet solve the motivating use case does it?

add(x);
acc.push(Err(e));

acc.push() will fail because the borrow has activated on add(x)?

P.S. so long as the closure

  • has been neither moved
  • nor is currently borrowed
  • including the case when closure's borrow time has ended

could these borrows done by the closure be treated as "inactive", "released"?

1 Like

Is it feasible to extend borrow checker to reborrow assuming struct borrows the binding or a field of the binding? Maybe even const functions will work. I.e.

let mut a = (15);
let b = &mut a.0;
a.0 += 20;
// Automatically insert another "let b = &mut a.0;"
*b *= 5; 

But

let mut a = [15];
let b = a.get_mut(0).unwrap();
a[0] += 20;
*b *= 5; // Still illegal since b is computed at runtime.

If this would make the rules on borrowing too complicated, there can be a macro-like utility.

let mut a = (15);
let b = &mut a.0;
a.0 += 20;
reborrow!(b);
*b *= 5; 

reborrow! would be defined as "Find the initialization site of the binding and create a new object using the same (const) code". Though limiting this to non-move closures is probably preferable.

Actually, this works with no changes to compiler:

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=79579940ef5e0c07cd7bcdcb17d4a5ac

Can be prettier if macro hygiene can be turned off.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.