Closure ergonomics/movable whole bindings

Just spitballing here but, what if you could unshadow names?

There are many ways to have lexical unscoping. Currently Rust has one:

{
  let x = ...;
}
// no x

But what if we had more? Something like this:

let move x = ...;
let y = move || x;
// no x

could be convenient with clone:

let x = Arc::new(());
let move x = x.clone();
let y = move || x; // maybe even ditch the `move` here entirely?
let z = move || x;

or, if willing to ignore/overlook guaranteed soundness issues with existing stack-pin macros:

let x = Arc::new(());
let x = x.clone();
let y = move || x;
del x; // bikeshed the keyword name
let z = move || x;

let x = some_unpin;
pin_mut!(x);
del x;
let y = x; // eep!

but anyway just, thoughts on non-hierarchical lexical scopes?

With del, you could also do stuff like this, where you temporarily unshadow it inside an inner scope:

let x = foo;
let x = bar;
{
  del x;
  x.foo();
}
x.bar();

(note that let move wouldn't be able to do this one, unfortunately... or fortunately.)

Anyway we may have talked about this before but we don't remember it going anywhere previously. But this thread is inspired by this: Clone into closures · Issue #2407 · rust-lang/rfcs · GitHub

(also related but different language 'Non-linear lexical scoping (again) and an idea about __unset' - MARC )

Wouldn't this make pin_mut! unsound?

2 Likes

Nah, let move is pretty tame. The discussion about del is mostly... for context. We mean, that was our original idea back on the Lua mailing list, and let move is inspired by that.

Fwiw, the only issue with the existing pin_mut! is that it does:

let mut $x = $x;
let $x = unsafe { ... $x };

and would be fine if it were:

let mut storage = $x;
let $x = unsafe { ... storage };

(because of hygiene)

I think this is a bad idea because it would be unclear to the programmer which binding is being used. The to-be-allowed code currently has a meaning – to use the moved-from binding – and this would change that existing stable behavior. The existing meaning (deliberately) doesn't compile, but the code still already has a clear and obvious meaning.

The advantage of this is purely in a fixed, small reduction in the number of unique names a programmer has to write in a program, at the expense of then later reading the same program.

Note that it is possible to use a shadowed binding via hygiene, if you were able to use it previously:

let x = 0;
macro_rules! m { () => (x); }
let x = ();
dbg!(m!()); // prints 0
4 Likes

It is a common pattern (at least we find ourselves using it often) to do

{
  let x = x.clone();
  foo(move || x);
}

We don't see what's confusing about

let move x = x.clone();
foo(move || x);

A reduction in indentation doesn't seem like a big deal but it's really easy to overdo indentation when using fairly simple macros, like impl_trait! from the impl_trait crate. Mostly because it already forces 2 indents on you, compared to the "conventional" way of implementing traits: one for the impl_trait! macro itself, and one for the impl trait block inside the inherent impl. This is compounded by us following the 80 columns rule, 16 (20%) of which are just to get into the function itself in impl_trait!:

impl_trait! {
    // 4 columns
    impl Foo {
        // 8 columns
        impl trait Bar {
            // 12 columns
            fn foo() {
                // 16 columns(!)
            }
        }
    }
}

We especially noticed this being an issue with the hexchat-plugin crate. But anyway, we digress, the main point is that the ability to move a whole binding into a closure seems more ergonomic to us?

It sounds like the principal motivation is getting rid of temporaries consumed by moves into closures. Can't you write this instead?

foo({
    let x = x.clone();
    move || x
})

If that's still too much indentation (maybe the typical closure body is a lot more complicated than the example suggests?) you could hide it in a macro:

foo(move_binding!(x = x.clone(), || x))

And that suggests to me the purely syntactic extension

foo(move |x = x.clone()| x)

which would desugar to the same thing as the macro expansion.

2 Likes

pin_mut! already has to move the to be pinned value before pinning it as macros can refer to the shadowed variable if they are defined before the shadowing location.

let foo = 0;
macro_rules! get_foo { () => { foo } }
let foo = 1;
assert_eq!(get_foo!(), 0);

Yes, but pin_mut! sadly introduces two new bindings to the current scope, both with the same name.

It doesn't rely on hygiene, but on the assumption that nobody would ever suggest introducing unshadowing/non-hierarchical lexical scopes to Rust. (movable whole bindings would still be sound, tho.)

I had no idea this was possible - it seems pretty concerning to me.

I would have thought that both foo variable definitions should have the same hygiene, which should make the first one completely inaccessible after it's shadowed

Isn't the whole point of macro hygiene to require that symbols used in the macro be unaffected by code in the invoking scope? What other behavior could you have? If you wanted a macro to refer to a variable in the invoking scope, you have to pass the symbol in. Otherwise it would behave like a C macro.

1 Like

Both definitions of x are present in the same lexical scope, and the macro body isn't actually parsed until it's invoked. I would have thought that the def-site hygiene used for the token x (it's being used as a local variable, so the mixed-site hygiene turns into def-site hygiene) would only take the lexical scope of the macro definition into account. However, it appears to also consider the location of the macro definition within the lexical scope.

Ah, I see the problem then. The macro definition isn't acting like an item, but instead like a statement.

Though you could argue that variable shadowing implies the existence of an implicitly narrower scopes, so the idea of there being one lexical scope here doesn't sit right; there are two foos, so there are two lexical scopes. Actually, each let statement starts a new child scope with new variables, so that code is basically equivalent to:

{let foo = 0;
macro_rules! get_foo { () => { foo } }
{let foo = 1;
assert_eq!(get_foo!(), 0);
}}
3 Likes

So here are our thoughts about movable bindings:

  1. Movable bindings are moved into the next inner context they're used in.
    let move x = true;
    let y = x && x; // moves x into a temporary context
    // no more x in this scope
    
    let move x = foo;
    let y = &x; // error about temporaries
    // no more x in this scope
    
    let move x = foo;
    let y = x; // moves the binding into the expression, the value into y
    // no more x in this scope
    
    let move x = foo;
    let y = || x; // moves the binding into the closure
    // no more x in this scope
    
  2. But how does this interact with nested contexts?
    let move x = foo;
    let y = || {
        let z = || x; // does x get moved into this closure?
        // can x be used here?
    };
    // no more x in this scope
    
    Also:
    let move x = foo;
    let y = {
        let z = || x; // does x get moved into this closure?
        // can x be used here?
    };
    // no more x in this scope
    
    There are 2 rules available here: "move into inner context only" and "move into used context". Neither have great ergonomics, but "move into used context" is more flexible, because you can rebind it:
    let move x = foo;
    {
        let x = x;
        {
            x...;
        }
        // x is still available here
    }
    // no more x in this context
    
  3. If the binding gets moved into a closure, the value does too.
    let move x = foo;
    let y = || {
        // x is in this context
        let z = || { x };
        // no x in this context, let z consumed x.
    }; // y is an FnOnce
    

Oh hmm wait this doesn't work... The problem with moving it into the context where it's used is that temporaries also count. The problem with moving it into the context where it's moved is that you'd have to use move closures anyway to move the value into the closure, but then you'd lose the "move" tag on the binding? And it wouldn't work with non-move closures.

We really want this idea to work, but we're not sure how to make it work in a way that's simple to understand and teach.

You're right. For future reference, here's the code that actually implements that logic:

Hmm, what about:

  1. Move the binding whenever the value is moved.
  2. Closures move values from move bindings. (once)

This means you could do something like:

let move mut x = String::new()
x.push_str("hello "); // borrows x
x.push_str("world"); // borrows x
let y = || x; // moves the whole binding

It does mean move bindings get special-cased by closures, however, we think that'd be okay, because part of the point is to have finer control over move captures.

let state = State::new(...); // interior mutability
let move mut data = state.get_data(...);
let closure = |...| { data.update(...); state.set_data(..., data); }

(yeah, we know how to do this with current rust. yet we still think there are significant ergonomic benefits to be explored here.)