Proposal: New channels for Rust's standard library

[anon19897381]

This is actually a pre-RFC written in the form of a blog post:

[DDOtten]

A well written proposal and I fully agree with deprecating the mpsc module for the channel module. The transition would be pretty easy. Simply changing mpsc::channel to channel::unbounded and mpsc::sync_channel to channel::bounded for most users with the additional benefit of making your code more readable. It feels like we can have our cake and eat it by having a multiple producer multiple consumer channel with a better performance than the current multiple producer single consumer channel while being able to leave the confusing terminology behind.

[mjw]

Is omitting any form of selection clearly the right thing?

I remember first looking at mpsc after 1.0 and being very surprised that selection was missing (it feels like one of the basic primitives: Occam’s ALT).

From the history it looks more like select! was omitted from 1.0 because it was unfit for stabilisation, rather than because anyone positively thought that channels without selection make a coherent feature which is worth including in a standard library.

Maybe it’s worth looking to see what proportion of crossbeam-channel's users use selection.

[matklad]

I think we should deprecate mpsc long-term. Channels without select are a bad interface, because they are not composable.

I also think that, long-term, channels should be in std, because they are such a core primitive. However, I am not sure that we are ready for std::sync::channel right now. I think it perhaps makes sense to wait for async/await to stabilize, to be confident that sync::channel works both in blocking and in non-blocking contexts.

Is improving mpsc performance really beneficial? Perhaps adding a note to the docs “use crossbeam-channel” would be more helpful for end users?

OTOH, if we can decrease the maintainence burden of mpsc by simplifying code, that seems like a good idea. So perhaps we can introduce nightly only sync::channel with the right interface, and make mpsc a shim on top of that? We can than move sync::channel to stabilization in the future, together with the Stream abstraction.

[kornel]

Replacing of the existing implementation with a slightly smaller, better version seems like a definite improvement to me. If the missing Sync could be added without breaking things, that’d be even better.

I’m not sure about deprecation and replacement.

mpsc works OK for basic cases. I’ve used it in a few places and never actually needed select, so to me mpsc isn’t broken.

If we agree that mpsc shouldn’t have been in the stdlib in the first place, then adding another channel module just repeats the mistake. We’re already sending people to use 3rd party crates for relatively basic things, so I don’t think proper channels have to be in the stdlib. The stdlib doesn’t have queues (rayon), which I consider a more common and more basic primitive.

[kornel]

And excellent points about inconsistent naming. That’s something we can mostly fix right away by updating the docs and examples.

[gnzlbg]

I agree with the deprecations, but once we put something in standard, we can never remove it.

If we add a different channel API and implementation to the standard library, and in 5-10 years time it turns out that it’s also “bad”, then that would be two deprecated channel implementations that we would have to maintain forever.

I agree that channels are a core concurrency primitive, but so are many things in many fields (like, rayon is a core parallelism primitive, but that’s not enough of a reason to move rayon into the standard library). The question is whether channels are a core standard library primitive. That is, whether they are a vocabulary type / trait, or whether it is impossible to implement them in stable Rust efficiently because some compiler magic is required (think: Atomic types).

AFAIK these APIs can be implemented without issues outside the standard library. We already made this mistake once, let’s learn from it and not make it again. So my vote here is for no, let’s not move crossbeam channels into the standard library.

I wouldn’t be opposed for the deprecation message to recommend users to use crossbeam channels directly (as opposed to some more generic deprecation message that does not recommend any particular library).

EDIT: also another downside could be having to maintain two different channel implementations if crossbeam and the standard library diverge. I don’t know what’s our manpower for this, but if the issue is one of discoverability, or that the Rust book has a policy of only using libstd primitives, then we should improve those instead and try to keep libstd lean.

[djc]

At first after reading this blog post I was really excited about getting a new std::sync::channel library that contains the core of crossbeam-channel, but in the end I think I agree with @gnzlbg that it might make more sense to deprecate std::sync::mpsc and point to the crossbeam-channel project directly. And if we do decide to deprecate it, it’s not even clear it makes sense to spend the effort to simpler channel implementation for the current std API.

Another argument that @gnzlbg didn’t mention is that it’s clear that a lot of evolution has happened already between the release of std::sync::mpsc and the current iteration of crossbeam-channel, and it’s not clear that crossbeam-channel has matured enough (in the sense of the pace of change slowing down – not in the sense of being mature) that it would be a good fit for the “forever frozen” promise we have for std.

I’m not sure if we already have clearly defined guidelines for what kinds of things should be allowed to go in std, but “core primitives”, “compiler magic is required” might be interesting first steps to define something like that.

If you want to argue that the crossbeam-channel library contains fundamentals that lots of projects need, I think we should also consider making chrono part of std (610k vs 740k recent downloads). By which I mean, that’s a slippery slope and not a very clear-cut argument IMO.

[joelgallant]

Speaking as somebody relatively new, the common guidance in the community is “just use crossbeam”. I don’t think that’s a bad thing, since, as seen already, “never breaking” APIs are really really hard to get right. There’s a high bar to be included in stdlib, and frankly I think it should be higher than it is.

The current stdlib implementation does work for what it is, and those who care can find (and we can help them) better options.

Think about how long some boost libraries in C++ took to be included in their standard library.

[joelgallant]

From a surface level, it sounds like deprecation with a pointer to alternatives is better than the confusion of two implementations in stdlib (to me, that wouldn’t inspire much confidence - though it’s not unheard of in other languages, java’s vector/arraylist comes to mind). I’m all for fixing the current implementation in whatever way we can though.

[vorner]

My humble opinion here is: replacing the guts of mpsc without changing the interface doesn’t prevent the deprecation or switchover later on. If it’s already (mostly) written, it might be the first step no matter if the deprecation happens later on or not. And much faster than agreeing on new naming, if it should be deprecated, etc.

I’m not sure how far we can get with fixing the current API with soft-renaming (keeping the old one deprecated and hidden from docs, calling the new one from within it) methods and such. It would be nice if we didn’t have two eg. receiver types that would be incompatible and if mpsc and channel were actually only aliases and the types were completely the same.

[gnzlbg]

@anon19897381

I want to remind that the mpsc implementation has been broken for years and needs to be fixed one way or another as soon as possible. So we need to do something about it. [...] And even if we fix the bug, I still think the API has so many flaws it would be better to not have it at all.

Are you arguing that we should fix it even though we deprecate it?

I don't think that is a good use of resources, but I'm not going to complain if someone decides to do that.

[cuviper]

Besides batteries-included availability, are there any special advantages for including channels in std? For example, are there improvements that could be made with unstable features?

[borealis]

If a new API is provided, it would probably make sense to consider trying to align it with the futures channel API: https://rust-lang-nursery.github.io/futures-api-docs/0.3.0-alpha.13/futures/channel/mpsc/index.html

[Nemo157]

futures-channel::Sender will still require a function that does the current operation of disconnect to satisfy other traits it implements, specifically Sink requires a way to “close” the sink from a mutable reference.

Other than that, I think simplifying the API as much as possible and standardising it across implementations would be a great idea.

[toc]

@anon19897381

• The docs for Mutex , thread::spawn , and Iterator::for_each use channels.
• There’s a bunch of tests for threads, networking, allocators, and others that use channels. See this list.
• The testing framework libtest (that’s the #[test] thing) itself uses channels internally.

This is actually really interesting to me. Channels being a basic primitive are one thing, but there's no reason not to leave them in a crate. But other parts of std benefitting from channels, or a desire to standardize channels across crates (so they can talk to each other with channels) are two good reasons to include it. We should be absolutely sure we can provide a good, general, performant, and stable interface and implementation though.

It sounds like replacing the guts of mpsc and deprecating it both have 0 downsides. I'd like to know if we can use channels more thoroughly in std (just for tests seems kind of meh).

[scottmcm]

@anon19897381

Now, this doesn’t mean we need channels just because of these uses , but I think these uses mean channels are really common: pretty much as soon as one calls thread::spawn() , chances are they’ll also want channels, and channels are typically more handy than Arc<Mutex<_>> .

I don't think that's necessarily true -- once can also end up with Arc<Mutex<_>> because of rayon, or something like that, and never use channels at all.

I think my preference here is just to deprecate and wait. It can develop outside of std: new test frameworks can use whatever they want, rayon seems like it's fine outside of std, etc.

[dcarosone]

More than anything related to where they live, what I’d most like to see fall out of this opportunity is clear consistency of design across the various channel types and implementations. You get at a lot of that in the discussion about naming and terminology. I think the futures::channel and/or async case doesn’t perhaps get as much attention in the conclusion and direct proposal, and embedded channel constructs are also starting to appear. The reasons for that more-limited discussion scope are clear and fair, but the need remains.

If there is to be some minimal / base case of channels in std, one way of slicing things I haven’t seen suggested : std has only the ‘rendezvous’ or ‘zero-capacity’ implementations of channels.

• This perhaps allows for simpler implementation (I have no idea if it really helps, and in the ways that matter most, but it seems at least plausible)
• Perhaps even for some common traits (or at least design patterns naming conventions) that can then be shared and consistent
• Crate-based implementations like crossbeam can add to those to include the additional complexities of buffering and bounds management, performance and contention optimisation over this shared content, etc.
• Potentially, if this avoids allocations and GC for buffers it can even be useful for the no_std case

[kornel]

@gnzlbg

Are you arguing that we should fix it even though we deprecate it?

It sounds bad the way you state it, but I think may still be worthwhile: even after deprecation it will stay in the library ~forever. Even after deprecation it will have remaining users.

[jethrogb]

Agreed we should fix, fix & deprecate, or replace. We shouldn’t just deprecate.