Proposal: explicit reference to `self` for closures


#22

Another use case.

In current Rust we can write

fn fib_iter() -> impl Iterator<Item=u64> {
    struct FibIter { state: (u64,u64) };
    impl Iterator for FibIter {
        type Item = u64;
        fn next(&mut self) -> Option<u64> {
            r = self.state.0;
            self.state = (self.state.1, r + self.state.1);
            r
        }
    }
    FibIter{ state:(1,1) }
}

With this feature it is

fn fib_iter() -> impl Iterator<Item=u64> {
    let state = (1,1);
    (move |self| {
        impl Iterator for Self {
            type Item = u64;
            fn next(&mut self) -> Option<u64> {
                r = self.state.0;
                self.state = (self.state.1, r + self.state.1);
                r
            }
        }
        self
    })()
}

Looks not much better, but we don’t have to name the structure and don’t have to give every fields a type.


#23

Your code example has two tiny bugs (missing let and Some(r)). Here’s a modified version: Playground Link

The current version is both shorter and has easier to understand code. Each new feature should enable something that was either not possible, unergonomic or difficult before. The code you’re showing is even nicer than what I previously posted in this thread. Well done. However, since it is so nice it doesn’t serve as motivation for this feature ^^’


#24

In the case of Iterator and the Fibonacci sequence, I’d propose the following implementation instead using unfold.

extern crate itertools;
use itertools::{unfold, Itertools};

fn fib_iter() -> impl Iterator<Item = usize> {
    unfold((1, 1), move |s| {
        let r = s.0;
        *s = (s.1, r + s.1);
        Some(r)
    })
}

fn main() {
    println!("{:?}", fib_iter().take(10).collect_vec());
}

And here’s one using repeat_with (moving the initial state outside so that we can use different Fibonacci sequences):

fn fib_iter(mut s: (usize, usize)) -> impl Iterator<Item = usize> {
    repeat_with(move || {
        let r = s.0;
        s = (s.1, r + s.1);
        r
    })
}

#25

@Centril These look nice!

Here are the playground links:


#26

Ok. Challenge time.

Challenge: Use the same technology to turn the following code into today’s rust:

trait NewIterator {
    type Item;
    fn next(&mut self) -> Option<Self::Item>;
}
fn map<T,R>(mut i: impl NewIterator<Item=T>, mut f: impl FnMut(T) -> R) 
    -> impl NewIterator<Item=R> 
{
    (move |self|{
        impl NewIterator for Self {
            type Item=R;
            fn next(&mut self) -> Option<R> {
                match self.i.next() {
                    Some(t) => Some((self.f)(t)),
                    None => None
                }
            }
        }
        self
    })()
}

One possible answer can be found in playground. But I would not post it here to avoid trailing.

If you did this excersise, you will understand why the idea to allow Self being available inside the closure will help.

Again, the intension of this is not to resolve a specific problem like fib, but to help a lot of problems like reforming Iterator and a huge amount of code similar.

EDIT

Minor changes to confirm the current way to define associate types is not to be changed.


#27

The possible solution you posted looks perfectly fine to me.

The code above on the other hand seems not very maintainable to me. This is what’s happening essentially: An anonymous struct with implicit fields is defined. The methods defined on the struct implicitly define the struct fields. An instance of the struct is immediately created and the fields are assigned the values from the outer scope.

There’s a reason why closures are mainly used for very short functions. As soon as they get big, it becomes harder and harder to see which variables are captured where.

I also program in JavaScript and JavaScript lets me actually do something similar. But, Rust is just too different to do it the JavaScript way and the way you propose it should work doesn’t feel right in my opinion.

JavaScript code
function map (iter, mapFn) {
  return { // Object literal. Iterators in JS are just normal objects with a next() method
    next () { // method def: A method is a function and all functions are closures
      let { value, done } = iter.next()
      return done ? { value: undefined, done: true }
                  : { value: mapFn(value), done: false }
    }
  }
}

These are my thoughts on this. TL;DR in Rust, variable capturing is fine for short closures. But, as soon as things get large, using a struct, not a closure, with a proper explicit type definition is IMO the way to go.


#28

@MajorBreakfast

as soon as things get large, using a struct, not a closure, with a proper explicit type definition is IMO the way to go.

I can’t agree more. The point here is just that the code above is in many standards not too long.

It is especially true when I am using map for Option.

fn map<T,R>(mut i: impl NewIterator<Item=T>, mut f: impl FnMut(T) -> R) 
    -> impl NewIterator<Item=R> 
{
    (move |self|{
        impl NewIterator for Self {
            type Item=R;
            fn next(&mut self) -> Option<R> {
                self.i.next().map(f)
            }
        }
        self
    })()
}

(You can do the same thing in another version, but you gain less in percentage as that version is itself longer)

In terms of learning curve, to come up with the solution I posted in the playground, you have to learn (from the errors or from previous knowledge):

  • You can not use impl Trait for the parameter because you have to reuse the type later.
  • You have to use PhantomData as there is a type you need to refer to but don’t have data ralated to it
  • You have to repeat the type you already provided in the function signature, as your structure type have to be generic and cannot use the types that are already in scope
  • You also have to repeat the type constraints

And knowing all the above require you to be a medium level programmer already. If you can get it all right in the first place, congratulations you are an expert (I couldn’t). But even you get it right this time after learnt from mistakes, next time when you have to do similar things you may need to learn it again.

On the other hand, a beginner level programmer can easily repeat what I was did for map when they see example code like

fn take<T>(mut i: impl NewIterator<Item=T>, mut n:usize)
    -> impl NewIterator<Item=T>
{
    (move |self| {
        impl NewIterator for Self {
            type Item=T;
            fn next(&mut self) -> Option<R> {
                 match self.i.next() {
                       Some(t) => if self.n==0 { None } else { self.n=self.n-1; Some(t) }
                       None => None
                 }
            }
        }
        self
    })()
}

And if you cannot get it right in the first place, it just means you are not careful enough this time.

(Note, even you show the new programmer the right way for take, as it does not need PhantomData they will not get it right for map)


#29

Furethermore, kind of such thing is not at all like in Javascript. It is Java’s anonymous class.

Java code:

public <T,R> NewIterator<R> map(NewIterator<T> t, Function<T,R> f) {
    return new NewIterator<R> {
         public bool hasNext() {
             return t.hasNext();
         }
         public R next() {
              return f(t.next());
         }
    };
}

Actually, as the code for take and map are so similar I would expect to write a macro:

fn take<T>(mut i: impl NewIterator<Item=T>, mut n:usize)
    -> impl NewIterator<Item=T>
{
    trait_impl!(NewIterator, {
          type Item=T;
          fn next(&mut self) -> Option<T> {
              ...
          }
    })
}

It will not be too easy to turn this into a macro if we don’t anonymize the types involved.


#30

I am not an expert (yet) so I don’t really understand why do you need the PhantomData ?

I slightly modified your example to see the result and everything seems ok. (I’d be happy if you PM me if this gets too off-topic).

(my background is 98% C) Closures are better used for really small function IMHO. This is adding a lot of “magic” where thing can already be done explicitly with a real type which make the result also compatible with “regular” function not expecting an impl Trait…


#31

I think you are right for not requiring PhantomData and there are better solutions, as I am not expert as well so I will have blind spots.

I have experiences on almost all main stream languages, and I observed a lot of evaluation on languages.

Java introduced anonymous class first, as part of their solution for UI event handling mechanism. Just a few years ago they introduced closures. But their closures are just short hands for anonymous classes. Anonymous classes, by itself was proved to be really useful for Java programmers as they helped them to simplify the abstractions.

Rust on the other hand, introduced closure first. And now in this thread I realized only slightly modify its grammar we can support anonymous type on top of it.

In Rust, I believe anonymous type can also help encouraging the use of light weight traits, which will help programmers to build programs on top of abstractions, not on concrete behaviors.

So we now have two alternatives:

  1. Anonymous type. Require new grammar like
fn take<T>(mut i: impl NewIterator<Item=T>, mut n:usize)
    -> impl NewIterator<Item=T>
{
    move impl NewIterator {
          type Item=T;
          fn next(&mut self) -> Option<T> {
              ...
          }
    })
}

(the move keyword is needed as impl NewIterator is not Copy nor Clone, so i have to be moved to the anonymous object)

Then we will have self referencing closures for free (similar to what Java currently allows):

let mut password = gen_password();
accept_client(listen_socket, move impl FnOnce(Client) {
    extern "rust-call" fn call_once(self, args:(Client,)) {
        let (client,) = args;
        handle(client, &self.password);
        next_password(&mut self.password);
        accept_client(self.listen_socket, self)
    }
});

A macro can be introduced for this:

let mut password = gen_password();
accept_client(listen_socket, move self_closure!((Client,), {
        handle(client, &self.password);
        next_password(&mut self.password);
        accept_client(self.listen_socket, self)
});
  1. Self referencing closure. Then we have anonymous types for free.
fn take<T>(mut i: impl NewIterator<Item=T>, mut n:usize)
    -> impl NewIterator<Item=T>
{
    move |self| {
        impl NewIterator for Self {
            type Item=T;
            fn next(&mut self) -> Option<R> {
                ...
            }
        }
    }
}

(I originaly thought we need to call the closure but actually this is not necessary. The compiler knows the type of the closure, so it can pickup the trait implementation).

Again, a macro can simplify this:

fn take<T>(mut i: impl NewIterator<Item=T>, mut n:usize)
    -> impl NewIterator<Item=T>
{
    trait_impl!(NewIterator, {
            type Item=T;
            fn next(&mut self) -> Option<R> {
                ...
            }        
    })
}

In either solution, one concern is how to handle the code generation if the trait contains a generic method. If there is no generic methods, the compiler can generate the type and methods after monomorphization, as all types are known. So initially, I would suggest a restriction on traits that only contains non-generic methods. We can always fallback to use named types so this will not restrict anything.


#32

Instead of the impl NewIterator {...} grammar, an alternative is to allow struct {} and then we write impl NewIterator for struct {} instead. This is more verbose but will not be confused with the impl MyStruct {} grammar.