Proposal: New channels for Rust's standard library

[gnzlbg]

I agree with the deprecations, but once we put something in standard, we can never remove it.

If we add a different channel API and implementation to the standard library, and in 5-10 years time it turns out that it’s also “bad”, then that would be two deprecated channel implementations that we would have to maintain forever.

I agree that channels are a core concurrency primitive, but so are many things in many fields (like, rayon is a core parallelism primitive, but that’s not enough of a reason to move rayon into the standard library). The question is whether channels are a core standard library primitive. That is, whether they are a vocabulary type / trait, or whether it is impossible to implement them in stable Rust efficiently because some compiler magic is required (think: Atomic types).

AFAIK these APIs can be implemented without issues outside the standard library. We already made this mistake once, let’s learn from it and not make it again. So my vote here is for no, let’s not move crossbeam channels into the standard library.

I wouldn’t be opposed for the deprecation message to recommend users to use crossbeam channels directly (as opposed to some more generic deprecation message that does not recommend any particular library).

EDIT: also another downside could be having to maintain two different channel implementations if crossbeam and the standard library diverge. I don’t know what’s our manpower for this, but if the issue is one of discoverability, or that the Rust book has a policy of only using libstd primitives, then we should improve those instead and try to keep libstd lean.

[djc]

At first after reading this blog post I was really excited about getting a new std::sync::channel library that contains the core of crossbeam-channel, but in the end I think I agree with @gnzlbg that it might make more sense to deprecate std::sync::mpsc and point to the crossbeam-channel project directly. And if we do decide to deprecate it, it’s not even clear it makes sense to spend the effort to simpler channel implementation for the current std API.

Another argument that @gnzlbg didn’t mention is that it’s clear that a lot of evolution has happened already between the release of std::sync::mpsc and the current iteration of crossbeam-channel, and it’s not clear that crossbeam-channel has matured enough (in the sense of the pace of change slowing down – not in the sense of being mature) that it would be a good fit for the “forever frozen” promise we have for std.

I’m not sure if we already have clearly defined guidelines for what kinds of things should be allowed to go in std, but “core primitives”, “compiler magic is required” might be interesting first steps to define something like that.

If you want to argue that the crossbeam-channel library contains fundamentals that lots of projects need, I think we should also consider making chrono part of std (610k vs 740k recent downloads). By which I mean, that’s a slippery slope and not a very clear-cut argument IMO.

[joelgallant]

Speaking as somebody relatively new, the common guidance in the community is “just use crossbeam”. I don’t think that’s a bad thing, since, as seen already, “never breaking” APIs are really really hard to get right. There’s a high bar to be included in stdlib, and frankly I think it should be higher than it is.

The current stdlib implementation does work for what it is, and those who care can find (and we can help them) better options.

Think about how long some boost libraries in C++ took to be included in their standard library.

[stjepang]

Thanks everyone for commenting! You gave good arguments for not having any channels in the standard library and, honestly, I’d be happy with that outcome.

The current stdlib implementation does work for what it is, and those who care can find (and we can help them) better options.

I want to remind that the mpsc implementation has been broken for years and needs to be fixed one way or another as soon as possible. So we need to do something about it.

A serious bug that is trivial to trigger was reported two years ago but the complicated codebase makes it difficult to fix. There was a PR attempting to fix it, got blocked on removing mpsc_select (will be a big refactor), and was subsequently closed. The bug is now even documented as a known issue, which is a bad look for Rust and doesn’t inspire confidence in the quality of our code.

And even if we fix the bug, I still think the API has so many flaws it would be better to not have it at all.

[joelgallant]

From a surface level, it sounds like deprecation with a pointer to alternatives is better than the confusion of two implementations in stdlib (to me, that wouldn’t inspire much confidence - though it’s not unheard of in other languages, java’s vector/arraylist comes to mind). I’m all for fixing the current implementation in whatever way we can though.

[vorner]

My humble opinion here is: replacing the guts of mpsc without changing the interface doesn’t prevent the deprecation or switchover later on. If it’s already (mostly) written, it might be the first step no matter if the deprecation happens later on or not. And much faster than agreeing on new naming, if it should be deprecated, etc.

I’m not sure how far we can get with fixing the current API with soft-renaming (keeping the old one deprecated and hidden from docs, calling the new one from within it) methods and such. It would be nice if we didn’t have two eg. receiver types that would be incompatible and if mpsc and channel were actually only aliases and the types were completely the same.

[gnzlbg]

I want to remind that the mpsc implementation has been broken for years and needs to be fixed one way or another as soon as possible. So we need to do something about it. […] And even if we fix the bug, I still think the API has so many flaws it would be better to not have it at all.

Are you arguing that we should fix it even though we deprecate it?

I don’t think that is a good use of resources, but I’m not going to complain if someone decides to do that.

[stjepang]

Are you arguing that we should fix it even though we deprecate it?

Sorry, perhaps I could’ve been more clear. My point was that we should either fix the bug, replace the implementation, or deprecate it.

[cuviper]

Besides batteries-included availability, are there any special advantages for including channels in std? For example, are there improvements that could be made with unstable features?

[borealis]

If a new API is provided, it would probably make sense to consider trying to align it with the futures channel API: https://rust-lang-nursery.github.io/futures-api-docs/0.3.0-alpha.13/futures/channel/mpsc/index.html

[stjepang]

Besides batteries-included availability, are there any special advantages for including channels in std ? For example, are there improvements that could be made with unstable features?

I would say one advantage to including them in std is that we can trivially use them for tests, examples, and implementation of internals:

• The docs for Mutex, thread::spawn, and Iterator::for_each use channels.
• There’s a bunch of tests for threads, networking, allocators, and others that use channels. See this list.
• The testing framework libtest (that’s the #[test] thing) itself uses channels internally.

Now, this doesn’t mean we need channels just because of these uses, but I think these uses mean channels are really common: pretty much as soon as one calls thread::spawn(), chances are they’ll also want channels, and channels are typically more handy than Arc<Mutex<_>>.

Other than that there are no advantages. So the question is: are channels essential enough to have in the standard library, just like we have basic collections? My feeling is people will start missing mpsc if/when we deprecate it. But I could be wrong and appreciate that some disagree. It’s a tough call.

[Nemo157]

futures-channel::Sender will still require a function that does the current operation of disconnect to satisfy other traits it implements, specifically Sink requires a way to “close” the sink from a mutable reference.

Other than that, I think simplifying the API as much as possible and standardising it across implementations would be a great idea.

[catenary]

• The docs for Mutex , thread::spawn , and Iterator::for_each use channels.
• There’s a bunch of tests for threads, networking, allocators, and others that use channels. See this list.
• The testing framework libtest (that’s the #[test] thing) itself uses channels internally.

This is actually really interesting to me. Channels being a basic primitive are one thing, but there’s no reason not to leave them in a crate. But other parts of std benefitting from channels, or a desire to standardize channels across crates (so they can talk to each other with channels) are two good reasons to include it. We should be absolutely sure we can provide a good, general, performant, and stable interface and implementation though.

It sounds like replacing the guts of mpsc and deprecating it both have 0 downsides. I’d like to know if we can use channels more thoroughly in std (just for tests seems kind of meh).

[scottmcm]

Now, this doesn’t mean we need channels just because of these uses , but I think these uses mean channels are really common: pretty much as soon as one calls thread::spawn() , chances are they’ll also want channels, and channels are typically more handy than Arc<Mutex<_>> .

I don’t think that’s necessarily true – once can also end up with Arc<Mutex<_>> because of rayon, or something like that, and never use channels at all.

I think my preference here is just to deprecate and wait. It can develop outside of std: new test frameworks can use whatever they want, rayon seems like it’s fine outside of std, etc.

[dcarosone]

More than anything related to where they live, what I’d most like to see fall out of this opportunity is clear consistency of design across the various channel types and implementations. You get at a lot of that in the discussion about naming and terminology. I think the futures::channel and/or async case doesn’t perhaps get as much attention in the conclusion and direct proposal, and embedded channel constructs are also starting to appear. The reasons for that more-limited discussion scope are clear and fair, but the need remains.

If there is to be some minimal / base case of channels in std, one way of slicing things I haven’t seen suggested : std has only the ‘rendezvous’ or ‘zero-capacity’ implementations of channels.

• This perhaps allows for simpler implementation (I have no idea if it really helps, and in the ways that matter most, but it seems at least plausible)
• Perhaps even for some common traits (or at least design patterns naming conventions) that can then be shared and consistent
• Crate-based implementations like crossbeam can add to those to include the additional complexities of buffering and bounds management, performance and contention optimisation over this shared content, etc.
• Potentially, if this avoids allocations and GC for buffers it can even be useful for the no_std case

[kornel]

Are you arguing that we should fix it even though we deprecate it?

It sounds bad the way you state it, but I think may still be worthwhile: even after deprecation it will stay in the library ~forever. Even after deprecation it will have remaining users.

[jethrogb]

Agreed we should fix, fix & deprecate, or replace. We shouldn’t just deprecate.

[Tom-Phinney]

Since it has to stay in the library forever, it needs to be fixed. Whether it’s deprecated in favor of a preferred replacement is a separate matter.

[proc]

I think this also boils down to the same Problem other Langs have: Do we want to support those Projects officially ? Do we want to let them run and hope that the Authors keep up their good work ? For example the Apache libs for Java aren’t really a standard but it’s the defacto standard for many people.

This is also the question of what we count as “Batteries Included”, a slogan Rust had back in 2015 and I’d still count as a big plus. For example having utf-8 everywhere is kind of “batteries included”, the same goes for cargo as dependency management. But you could argue that the standard-stack of many projects are also counting to this.
I still remember being fresh to Rust and trying to write my own Error types. These days I always drop failure or error-chain in my projects, because I know that its okay to do so, that you are not taking too much overhead with that. But also because I know that these libraries are good and I’ve accepted that the std doesn’t have any solution to offer for handling errors of ~20 Dependencies, with all the benefits of failure. Sounds harsh but looking at this from the outside I can’t deny this.

[proc]

Yes I can totally see the problem of removing stuff from the std that’s stabilized. ( Java has still things from 1.0 that are deprecated since then, and they won’t ever go…) But having to tell people that they should ignore the std at some parts and look for crates sounds like a trap game with an std of holes to fall into.