[Pre-RFC] Returning traits by value

[stebalien]

Obligatory: I apologize if this has already been discussed.

Motivation

Currently, when writing a function that returns:

iter::range(1u, 10u).map(|x| x*x).filter(|x| x % 2u == 0)

One has to write one of:

// option 1
fn test<'s>() -> iter::Filter<'s, uint, iter::Map<'s, uint, uint, iter::Range<uint>>> {
    iter::range(1u, 10u).map(|x| x*x).filter(|x| x % 2u == 0)
}

// opiton 2
fn test<'s>() -> Box<Iterator<uint> + 's> {
    box iter::range(1u, 10u).map(|x| x*x).filter(|x| x % 2u == 0)
}

Option 1 is painful to write and option 2 requires a heap allocation.

Primary Proposal

My primary proposal is to allow one to write the following:

fn test<'s>(square: bool) -> Iterator<uint>+'s {
    iter::range(1u, 10u).map(|x| x*x).filter(|x| x % 2u == 0)
}

This would be equivalent to option 1 except that the caller would only be allowed to access methods accessible through the Iterator trait (this would also help with the problem of exposing private types in public methods).

Additional Proposal

Unfortunately, this only handles the case of a single return type. However, this isn’t always the case:

fn test<'s>(square: bool) -> ??? {
    if square {
        iter::range(1u, 10u).map(|x| x*x).filter(|x| x % 2u == 0)
    } else {
        iter::range(1u, 10u).filter(|x| x % 2u == 0)
    }
}

I propose, again, that the following syntax be allowed:

fn test<'s>(square: bool) -> Iterator<uint>+'s {
    if square {
        iter::range(1u, 10u).map(|x| x*x).filter(|x| x % 2u == 0)
    } else {
        iter::range(1u, 10u).filter(|x| x % 2u == 0)
    }
}

which would be functionally equivalent to:

fn test<'s>(square: bool) -> _test_ReturnType<'s> {
    if square {
        A(iter::range(1u, 10u).map(|x| x*x).filter(|x| x % 2u == 0))
    } else {
        B(iter::range(1u, 10u).filter(|x| x % 2u == 0))
    }
}
enum _test_ReturnType<'s> {
    // One for each possible return type
    A(iter::Filter<'s, uint, iter::Map<'s, uint, uint, iter::Range<uint>>>),
    B(iter::Filter<'s, uint, iter::Range<uint>>)
}
impl<'s> Iterator<uint> for _test_ReturnType<'s> {
    fn next(&mut self) -> Option<uint> {
        match self {
            &A(ref mut inner) => inner.next(),
            &B(ref mut inner) => inner.next()
        }
    }
}

Note: This is just an illustration, not a implementation recommendation. For the actual implementation, I would resolve funtions on the returned “union” the same way rust currently does on a boxed traits.

Drawbacks

The primary drawback I can see is that methods that return traits by value would not be callable on trait objects because, like methods that return Self, the underlying return type of these methods is implementation dependent.

[huon]

RFC PR #105 is related (NB. closed by the author for revision, not rejected.)

[aturon]

I closed the RFC in part because it was decided that this feature, while desirable, should wait till after 1.0.

As an aside, note that Iterator<uint> is already a valid type under DST, so it probably doesn’t work to simply use that type name as-is.

[P1start]

Given that this feature is almost necessary for unboxed closures, as their type cannot be specified in the normal way, it seems strange to close this feature until after 1.0, as the old closures should be removed by then. Does this mean we’ll have no way of returning closures other than using Box, which allocates?

[aturon]

That’s right, you will likely not be able to return an unboxed closure, because the relevant traits are likely to be feature-gated (since they may want to change if VG happens).

1.0 is largely about putting together a coherent core language/set of libraries that we feel comfortable promising backwards compatibility for. We will be adding plenty of new things afterwards. But it’s important that we actually get a release out, and that means making some hard decisions about what has to be done now.

[eddyb]

I should mention that some/most of the compiler machinery for implementing RFC PR #105 is already there (due to unboxed closures being anonymous types anyways), it would be a real shame to not have it in 1.0 solely because of supposed implementation complexity. Are there any standing issues or unanswered questions with it?

[aturon]

@eddyb Note that, as far as I know, you will not be able to return an unboxed closure at 1.0, which is the relevant bit here.

Implementation effort was part of it, but also at the time this was written we had not designed, let alone implemented, associated items. Clearly there is interaction between the features, and a new design needs to be carefully developed. In addition, there were several additional points of complexity raised in the comments for the original RFC that will need design work. In short, I don’t think the design as presented in my original RFC is the right one, and we need more careful thought and experience before putting forward a new design.

Even for associated items, only a portion of the RFC has been implemented and there’s much more work left to do.

Finally, it’s worth emphasizing again both that 1.0 is not very far off, and that we’ll be on a six week release schedule. The 1.0 release is a beginning, not an ending, and features like this (and HKT, and VG, and many others) can be pursued once we have this critical stable base to work from.

[glaebhoerl]

There've been calls to have a DST RFC, and I think that would also be helpful in this context:

@aturon

As an aside, note that Iterator<uint> is already a valid type under DST, so it probably doesn't work to simply use that type name as-is.

It's very hard to think about the potential connections, similarities, and differences between these two interpretations of Iterator<uint> when DST itself isn't precisely specified.