Proposal: explicit reference to `self` for closures

[earthengine]

My original concern of using self is that it would not allow self being used in the closure body. But I think in that case we can let the user to move self to another variable, then captures it into the closure.

using self have other benefits: it makes the closure looks more like the form FnXXX traits being defined. So when new users looking at the signature of the trait they know how to use them immediately.

So yes, I agree self might be better.

[earthengine]

You are right about Fn and FnMut.

But once we are able to do the above we may look forward to support not just FnBox but also FnCell, FnRefCell, FnRc… etc, in exactly the same way. And it will be hard to decide which one implies another, so it would be good to keep everything consistent.

[H2CO3]

Technically, that may sound very plausible. In reality, however, it’s a huge change in the mental model: whenever one sees self in a closure from now on, they will have to think hard about whether it’s a captured self or it refers to the closure. Sounds quite dangerous to me, it’s very easy to confuse these two, it makes the code surprising to read.

This also makes the code heavily context-dependent. I often refactor code moving it out of and into a closure. Maybe a closure grows too big and needs its own method, or maybe a method becomes trivial and can be inlined. This proposal makes it impossible to perform such refactoring, because the meaning of self changes as you move code from a self-referring closure to a method and vice versa.

@earthengine I don’t really see why it is useful in realistic code. When was the last time you needed a recursive closure that mutates its captures, and why wouldn’t you refactor it immediately to something clearer? The usage of such closures seems to be very bad style to me.

[Centril]

Note: I’m not saying we should necessarily do this; I’m only saying what a non-breaking solution might be

[MajorBreakfast]

I’m 100% with @H2CO3 on this. Changing the meaning of self is not a good idea.

[MajorBreakfast]

@earthengine Can you provide a better use case where you actually need a such a closure? IMO your example would be less complicated if it was written with a normal function that passes the state:

fn fib_internal(curr: i32, next: i32, remaining: i32) -> i32 {
    if remaining > 0 {
        fib_internal(next, curr + next, remaining - 1)
    } else {
        curr
    }
}

pub fn fib (n: i32) -> i32 {
    fib_internal(0, 1, n)
}

Try in playground

IMO that’d be more explicit and easier to understand.

FnBox is still a good option as it makes the above works the same way as the other variants.

Using a boxed function for your code example would have a severe performance impact. You’d get thousands of unnecessary heap allocations if you boxed it. Boxing should only be used if required.

[earthengine]

Needless to say, it can also be done in today’s rust closures. Only that you will need a fixed point combinator, like the following.

Playground link

fn fix<F,T,R>(f:F, t:T) -> R 
    where F: FnOnce(&Fn(T) -> R, T) -> R + Clone
{
    (f.clone())(&|t| fix(f.clone(), t), t)
}

fn main(){
    let fib = |i|fix(move |s:&Fn((u32,u32,u32)) -> u32, v:(u32,u32,u32)|
        match v.0 {
            0 => v.1,
            1 => v.2,
            i => s((i-1, v.2,v.1+v.2))
        }, (i,1,1));
    println!("{}",fib(46));
}

Compare to the solution in the top post, this is less intuitive and harder for new users. Moreover, it depends on the fact that u32 is Copy and so mutation can be simulated. But for other types, this may not be that easy and may need to introduce Cell or RefCell things.

All I want to see for this feature, is not to change anything that works today, it is to allow a new way to express the same intention. So if you don’t like it, don’t use it! This apply to @H2CO3 's refactoring argument, and the @MajorBreakfast 's FnBox argument. (and actually, if we do need FnBox, my proposal is an improvement because the user have full access to the Box and the closure code can decide to move the content out of the box, which would not be allowed in today’s rust closures)

When talking about real usage, well, it is depending on how would you define “complicated”. In the fib case, it seems easy to have a separated function. But this applies to all closures, if you always prefer to have separately defined functions, why do you need a closure? But then closure was proved its value in all modern programming languages.

Having said that, let me just try to give another use case though. Let’s say that an application will listen on a port for connections. It only handles one connection at a time, but each time it receives a connection and handles it, it changes the way it authenticate the client.

let mut password = gen_password();
accept_client(listen_socket, move |mut self, client| {
    handle(client, &self.password);
    next_password(&mut self.password);
    accept_client(self.listen_socket, self)
});

As I have said, you can decide to not use the self part and you don’t have to use self in the body, so you can extract the function easily. But even with self you can still do it like this:

let mut password = gen_password();
accept_client(listen_socket, move |mut self, client| {
    let closure = self;
    handle(client, &closure.password);
    next_password(&mut closure.password);
    accept_client(closure.listen_socket, closure);
});

[ExpHP]

You’d reach for a fix-point combinator? I’m impressed.

Call me a simple man, but I’d just go for this:

let fib: Box<Fn(i32) -> i32> = Box::new(|i| {
    struct Closure { i1: i32, i2: i32 }
    impl Closure {
        fn call(&mut self, i: i32) -> i32 {
            assert!(i >= 0);
            match i {
                0 => self.i1,
                1 => self.i2,
                i => {
                    let old_i1 = self.i1;
                    self.i1 = self.i2;
                    self.i2 += old_i1;
                    self.call(i - 1)
                }
            }
        }
    }
    Closure { i1: 1, i2: 1 }.call(i)
});

[H2CO3]

So if you don’t like it, don’t use it! This apply to @H2CO3 's refactoring argument

Sorry, but this argument still doesn’t work. If a language feature is introduced, it will be used, and eventually anyone will encounter and have to deal with code relying on (and bugs arising out of) that feature.

[earthengine]

I mean, you can’t prevent someone else writing bad code, but at least you can decide to not writing them.

Back to the discussion of this feature, I didn’t see how it should change your view of a piece of code. It is true that when you look at self you need to ask yourself what it refers to, as its meaning depends on context. But this is already happening right now. Check @ExpHP 's code. self 's meaning is also changed in the method body.

[H2CO3]

In what way is it changed? Isn’t it the method’s receiver, as it always was?

[earthengine]

If the outer let is inside a method, then from the outside of the block, self is the outer method’s receiver, which will be a different object than the inner self.

This is exactly the same as in my code: self inside the closure is now refer to a new object, regardless the value being defined from outside or not.

[earthengine]

His code is structured something like let fib:... = ...;, so it is a single let binding, which can be part of a function/method body.

[H2CO3]

If the outer let is inside a method, then from the outside of the block, self is the outer method’s receiver, which will be a different object than the inner self.

That’s not where the problem lies though. In that case, you could still always tell that self belongs to the innermost method.

Instead, the problem is found in code like this:

impl Foo {
  fn bar(&self) {
    self.do_thing(|&self| qux(self.some_member))
  }
}

Here, the self inside the closure refers to the captured variables of the closure. Now if it becomes longer and more complex, I might want to refactor it into a method:

impl Foo {
  fn bar(&self) {
    self.do_thing(Self::frobnicate)
  }

  fn frobnicate(&self) {
    qux(self.some_member)
  }
}

Here, the meaning of self changed from “closure capture” to “method receiver”. It is no longer the case that it is always the receiver of the innermost method; instead, it’s sometimes that, and sometimes a closure capture. And if some_var happens to be in both, the code can still compile and silently do the wrong thing, which is the worst possible scenario.

[earthengine]

you could still always tell that self belongs to the innermost method.

True for the closure as well - you can still always tell once you see the |&self...| heading.

Here, the meaning of self changed from “closure capture” to “method receiver”.

True again. But it happens every time when you attempt to refactoring unwisely. To perform a safe refactoring, you have to be careful for all names the original code referring to. This feature didn’t make it worse or better, just be careful

[H2CO3]

you can still always tell once you see the |&self...| heading

The problem is not with the syntax or that I don’t know what self is; the problem is that its meaning changes.

just be careful

By that reasoning, we wouldn’t need any of Rust’s safety features, and we could just go back writing Heartbleed or goto fail in C. Dangling pointers or double-free? “You weren’t careful enough!” Race conditions? “You should have known better and used atomics!”

50+ years of computing history shows that “just be careful” doesn’t work. Programmers make mistakes no matter how careful they are, and a good language – especially one that claims to be correctness- and safety-oriented – tries to minimize the amount of errors one can make. That is ergonomic. A marginally-useful, but sometimes dangerous special case is not ergonomic.

[earthengine]

Right. So in terms of Rust safety features, your code below

fn bar(&self) {
    self.do_thing(|&self| qux(self.some_member))
}

should not compile, as self was changed meaning. To compile with the proposal, you will have to write:

fn bar(&self) {
    self.do_thing(|&self| qux(self.self.some_member))
}

The first self is the closure itself, and it captures the method self (outer one). So for the proposal, you will need to write self.self to refer to the outer self.

Once you later decided to refactoring it, the compiler shall again prompt you with an error that the outer self does not have a member called self, so your code will not compile if you didn’t remove the extra self

[earthengine]

Another use case.

In current Rust we can write

fn fib_iter() -> impl Iterator<Item=u64> {
    struct FibIter { state: (u64,u64) };
    impl Iterator for FibIter {
        type Item = u64;
        fn next(&mut self) -> Option<u64> {
            r = self.state.0;
            self.state = (self.state.1, r + self.state.1);
            r
        }
    }
    FibIter{ state:(1,1) }
}

With this feature it is

fn fib_iter() -> impl Iterator<Item=u64> {
    let state = (1,1);
    (move |self| {
        impl Iterator for Self {
            type Item = u64;
            fn next(&mut self) -> Option<u64> {
                r = self.state.0;
                self.state = (self.state.1, r + self.state.1);
                r
            }
        }
        self
    })()
}

Looks not much better, but we don’t have to name the structure and don’t have to give every fields a type.

[MajorBreakfast]

Your code example has two tiny bugs (missing let and Some(r)). Here’s a modified version: Playground Link

The current version is both shorter and has easier to understand code. Each new feature should enable something that was either not possible, unergonomic or difficult before. The code you’re showing is even nicer than what I previously posted in this thread. Well done. However, since it is so nice it doesn’t serve as motivation for this feature ^^’

[Centril]

In the case of Iterator and the Fibonacci sequence, I’d propose the following implementation instead using unfold.

extern crate itertools;
use itertools::{unfold, Itertools};

fn fib_iter() -> impl Iterator<Item = usize> {
    unfold((1, 1), move |s| {
        let r = s.0;
        *s = (s.1, r + s.1);
        Some(r)
    })
}

fn main() {
    println!("{:?}", fib_iter().take(10).collect_vec());
}

And here’s one using repeat_with (moving the initial state outside so that we can use different Fibonacci sequences):

fn fib_iter(mut s: (usize, usize)) -> impl Iterator<Item = usize> {
    repeat_with(move || {
        let r = s.0;
        s = (s.1, r + s.1);
        r
    })
}