Async/Await series

[aturon]

That said, I agree that Unpin is an awkward name.

[seanmonstar]

I’m very excited that we can soon await while having borrowed some arguments!

fn poll(self: Pin<Self>, ctx: &mut task::Context)

Is it possible to not modify Future to need to take Pin<Self>? It feels like a leaked abstraction, that if I’m implementing Future, I need know something about pins, when its expected that almost everything will not need to be pinned. Particularly, nothing else is going to need to take Pin<Self>, so it sticks out to me.

Could the pinning not be dealt with at the await keyword/macro, wrapping the expression in a pinned type that implements Future for it?

[Nemo157]

Could the pinning not be dealt with at the await keyword/macro, wrapping the expression in a pinned type that implements Future for it?

Note that not all immovable futures are defined via async/await. Just today I manually implemented a StableStream that uses an internal buffer written to via DMA from a radio peripheral. It’s so nice to be able to avoid the standard static global/externally allocated buffer implementation that’s necessary to ensure safety.

[borealis]

One challenge of the current async situation is that any function annotated with #[async] returns impl Future which is an unnameable type. This is somewhat viral in nature: any other Future which uses it also has an unnameable type. And the only escape hatch, is to put it into a Box<Future>. So libraries can’t use #[async] without forcing users box up futures returned from the library. The end result seems to be that #[async] provides nice benefits for a library author, but by using it, the library provides futures which are less useful to client - a strong incentive not to use it. Authors of client code who use #[async] are also in an awkward position - on the one hand, libraries that avoid #[async] are more useful, on the other hand, if the client author wants to dig into the library’s code they have to read futures written in a somewhat different style than they may be accustomed to.

I thought I read something a while ago about plans to make impl Future return types nameable and I was wondering if / how that fit into the current await plans?

[Ixrec]

If I understand all the impl Trait-related RFCs correctly, the current answer to “making impl Trait types nameable” is https://github.com/rust-lang/rfcs/pull/2071 modulo a fresh syntax bikeshed when this is all closer to stabilization

[withoutboats]

Is it possible to not modify Future to need to take Pin<Self>? It feels like a leaked abstraction, that if I’m implementing Future, I need know something about pins, when its expected that almost everything will not need to be pinned. Particularly, nothing else is going to need to take Pin<Self>, so it sticks out to me.

It is not possible if you want to be able to treat the return type of an async fn as impl Future, because that means it has to be pollable, but polling it advances its state, creating self-references. And we certainly want the return type to be a future, so we can call combinators on it without heap allocating it.

The key point is that Pin is isomorphic to &mut if the type implements Unpin. So for manual futures the change is fairly insignificant (this is discussed more in the post).

[seanmonstar]

It is not possible if you want to be able to treat the return type of an async fn as impl Future

I meant that the wrapped value would still implement Future, just that the details that it needs to be pinned would be specific to that one future, not to all manual implementations of Future. Such that an async function state machine becomes:

struct __AsyncGenerated {
    inner: Pin<__State>,
}

impl Future for __AsyncGenerated {
    fn poll(&mut self) {
        // the state is pinned, and we deal with the pin in here, because we have to
    }
}

The key point is that Pin is isomorphic to &mut if the type implements Unpin. So for manual futures the change is fairly insignificant

Oh sure, but my thoughts were more that seeing self: Pin<Self> at all, even if I’m not writing a self-referential future, seems like leaking the Pin/Unpin concept.

I find it similar to if poll required self: Arc<Self>, or something. I’d be left wondering… “huh? why?”

[seanmonstar]

I’ve since reasoned through why my suggestion of wrapping a Pin cannot work: you can’t move around the wrapper that has a pin, you need to pin at the outermost future, and forward the pin through.

I still think it’s leaky to have to receive Pin<Self> universally in every single manual Future implementation, and so hope to still find a way to not have to do so.

[twmb]

This work is very nifty, but all of it combined with the futures-await crate is still a bit over my head. I’m looking forward to a how-to to show up in https://aturon.github.io/apr/async-in-rust/chapter.html.

[withoutboats]

I think another way to state what you’ve realized is that the Pin type and the PinBox type both indirect through pointers. You can’t just have a Pin newtype - moving that moves the inner value. There’s no way to have the functions return a pointer unless they heap allocate every time, which is untenable.

We’ve explored a lot of different API designs here - at least a dozen - and I’m pretty confident that you can’t get less “leaky” than this API while maintaining the performance properties we care about.

[Nemo157]

If I understand all the impl Trait-related RFCs correctly, the current answer to “making impl Trait types nameable” is https://github.com/rust-lang/rfcs/pull/2071 modulo a fresh syntax bikeshed when this is all closer to stabilization

Unfortunately the current #[async] macro does not support this, there is nowhere to set the name that you want to use instead of impl Future. Hopefully this can become possible in the future though.

[kirillkh]

Please forgive my ignorance, but I wonder how this compares to Kotlin’s coroutines. Has anyone looked at those from Rust community? I successfully used Kotlin coroutines in production and can attest first-hand that they make writing nonblocking code so incredibly easy it feels like magic.

[rpjohnst]

I’m a fan of Kotlin’s coroutines. They’re similar in a lot of ways, though there are some important differences I’ve brought up before:

• Syntactic: https://www.reddit.com/r/rust/comments/6zy8hl/kotlins_coroutines_and_a_comparison_with_rusts/
• Semantic: Pre-RFC: CPS transform for generators

[kirillkh]

I second everything you wrote there. To be honest, after working with Kotlin coroutines I just wish every language would build its nonblocking implementation on top of that kind of mechanism. It really feels like a much-needed breakthrough after all the callbacks/futures/RX jungle. I’m not sure about how performance compares to other approaches, though.

[harrychin]

I also concur with the sentiment that Kotlin coroutines have a lot of ergonomic and intuitive advantages. I’ve personally used Kotlin’s Coroutine library for an Android UI and it’s both more ergonomic and much more efficient than alternative solutions. The system’s flexibility to support the front-end API’s reminiscent of other languages like Go’s switch or async/await while not causing substantial increases in memory usage feels very optimal.

[glaebhoerl]

W.r.t. Kotlin experience reports, can we be explicit about what the alternatives being compared against are? C# async/await? Go/Haskell green threads? Algebraic effects? I have no trouble believing it’s much superior to callbacks, but that’s presumably true of most options.

[withoutboats]

I haven’t looked closely at kotlin but it seemed analogous to the fact that kotlin has TCP preserving closures, which we don’t have & which is very difficult to integrate with ownership/borrowing and our absence of a runtime.

[kirillkh]

I compared Kotlin’s coroutines to Java futures and ReactiveX. I haven’t used an async/await framework before, so it’s hard to comment there. But KC implement async/await and, while that functionality has its uses, the syntax is much more verbose than vanilla coroutines. The primary advantage of KC over everything else I tried is that it doesn’t obscure your logic with boilerplate and allows you to keep it ordered in the same way like it was blocking code, i.e. it allows you to lay out your code the way you want to, not the way the framework imposes on you. That’s hard to overestimate, really. Another advantage is that it integrates well with legacy code based on callbacks and futures. I used it in combination with Vert.x, which is based on futures. While a modest amount of glue was necessary, it primarily involved just calling .await() on a returned future and felt completely fluid.

If you know any other async frameworks that have the same qualities, I would be very interested to learn about them.

[rpjohnst]

What do Kotlin’s coroutines have to do with its TCP closures? The two don’t really interact, and even if they did they implement TCP without relying on the GC or a runtime (they just inline the closures)…

[rpjohnst]

If you know any other async frameworks that have the same qualities, I would be very interested to learn about them.

Its closest relations are C#, Python, and Javascript’s async/await. They also allow your code to stay ordered the same way as blocking code, and integrate with existing callback and futures-based code.

The main difference with Kotlin coroutines, as far as I’m aware, is the presence of the await keyword, and the way you write the return type of the function. In C#, calling an async function gives you a future (Task<T>), which you then await to suspend and get its value:

async Task<int> AccessTheWebAsync()
{
    HttpClient client = new HttpClient();
    string page = await client.GetStringAsync("https://example.com");
    return await FurtherProcessing(page);
}

While Kotlin coroutines are much more syntactically similar to sync code (using a fake API to match the C# example):

suspend fun accessTheWeb(): int {
    val client = HttpClient()
    val page = client.getString("https://example.com")
    return furtherProcessing(page)
}