Closures that can return references to captured variables

language design
1

Closures currently cannot return references to captured variables. For instance the following code produces an error saying "references to captured variables can't escape the closure".

fn foo() {
    let arr = [1, 2, 3];
    let c = move |i: usize| &arr[i];
}

However, with GATs an alternate Fn trait could be defined like this:

trait LendingFn<Args> {
    type Output<'s>;
    fn call(&self, arg: Args) -> Self::Output<'_>;
}

This would allow return types that are tied to the lifetime of self and is similar to the discussed lending variants of Iterator and Stream. In my opinion it would also be useful to provide a GAT variants of Fn and FnMut once GATs are stable. The compiler could then accept the function foo from above creating a closure implementing LendingFn.

A downside of including Lending* variants for the closure and iterator traits is a further increase in complexity, considering there are already three different traits for closures. Since non-lending closures can be implemented in terms of the lending trait variants, I wonder if it were possible to change the existing traits instead, by adding a generic lifetime parameter to their associated types. While, currently, this would be a breaking change, it might be possible to add the lifetime parameter in a compatibility-friendly way.

The compiler could accept "downgraded" implementations of traits with GATs. In implementations not using the generic parameters of the associated type, the compiler could allow referring to the associated type without specifying generic arguments. This could work since such implementations constitute "special cases" of implementations that do use the GAT parameters. Thus all existing implementations of the non-GAT traits would still be accepted after adding the lifetime parameter, making it a non-breaking change. This would also be beneficial for the Iterator and Stream traits, avoiding redundancy with potential LendingIterator or LendingStream variants.

Do you think closure traits using GATs would be useful?

Is is realistic to change the existing closure and iterator traits?

2 Likes
2

They can if you don't move the captured variables into the closure, i.e. if you remove the move keyword from your example.

Maybe that's not important to the rest of your argument (I stopped reading because GAT's are kinda over my head), but I wanted to point this out nonetheless :slight_smile:

3 Likes
3

Yes, you are totally right! The limitation only exists for returning references to variables owned by the closure. I should have made clearer that I am specifically talking about that case.

4

For Fn{Mut|Once} specifically, it wouldn't be (stable[1]) breaking, because it's currently impossible to implement or otherwise name the trait(s) except through the Fn(Args...) -> Output form. Adding the GAT would thus be compatible, so long as the existing Fn() alias continues meaning Output is independent from call(&self)'s '_.

[1] of course, it'd be breaking for nightly-only implementers (but we don't care about that, because nightly opts out of stability) unless we added a rule that you can implement a GAT without the lifetime generic, and it just accepts and ignores that input lifetime. Underspecified, but I think possible, somewhat realistic given other lifetime inference rules, and potentially desirable if it allows us to reuse existing traits for the lending versions.

5

FWIW, what you want here is still expressible through a HRTB over FnOnce:

for<'lt>
    &'lt F : FnOnce(usize) -> &'lt i32
,

so no need to be (nightly-)breaking or whatnot: both Fn and FnMut are, in fact, shorthands, that technically could be removed just for higher-order FnOnce signatures where the return type does not mention the lifetime of the borrow over Self:

F : Fn[Mut](Args…) -> R,
⟺
for<'__>
    &'__ [mut] F : FnOnce(Args…) -> R
,

Alas, in practice, these higher-order bounds are both unmet by closures, and unusable when met, so one needs to manually hand-roll those, defeating the point of using closure sugar to begin with :weary::

#![feature(fn_traits, unboxed_closures)]

struct Example<T, const N: usize> /* = */ (
    [T; N],
);

impl<'lt, T : 'lt, const N: usize> FnOnce<(usize, )>
    for &'lt Example<T, N>
{
    type Output = &'lt T;
    
    extern "rust-call"
    fn call_once (
        self, // : &'lt Example<T, N>, /* only shorthand form is accepted for rust-call */
        (i, ): (usize, ),
    ) -> &'lt T
    {
        &self.0[i]
    }
}

fn main ()
{
    let mut f = Example([1, 2, 3]);
    // dbg!(*f(0)); // ERROR
    dbg!(f.call_once((0, ))); // OK: &1
    f.0[0] = 42;
    dbg!(f.call_once((0, ))); // OK: &42
}
3 Likes
6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

8

I am re-opening this topic, as @Ddystopia expressed interest in continuing this discussion in light of the fact that we do have stable GATs now.

1 Like
9

To summarize, this feature will be very convenient and changing FnMut trait will not cause stable breakage. Taking into account that GATs were stabilized in 1.65, I see no reason why it should not be implemented.

Is it worth making a Pre-RFC for this? I could work on this and on the PR .

10

Adding a GAT to FnMut would make it non-object-safe.

11

I asked about this a while ago on URLO I think and was given an additional reason against: FnMut: FnOnce, but closures can't return a reference to owned data in a method that takes a self receiver.

12

Recent Zulip discussion: https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/Lending.20closures.20and.20Fn*.28.29.20-.3E.20impl.20Trait