UnixStream on Windows?

Recent Windows releases support AF_UNIX stream sockets. However, std::os::unix::{UnixStream, UnixListener} are only available on Unix OSes.

It would be useful to have these available on Windows as well so that developers don't have to pull in external crates to get functionality on Windows that's available in std for Unix.

Is that assertion in line with the current thinking about what goes in std? If so, what's the best way to go about resolving this? It seems that we would have to move UnixStream and UnixListener out of std::os::unix since that module is full of Unix-specific things that will likely never apply to Windows. Can we move them to std::net?

3 Likes

Moving would be a breaking change, but it's possible to create a new type there, or even just re-export from std::os if they're completely compatible.

However, will all std platforms have this? What about something like WebAssembly?

Or we could add a similar type under std::os::windows, which can even share implementation under the hood. However, what would happen to this on Windows before that support was added? Our platform support states "Windows 7+", so AF_UNIX isn't universal.

A third-party crate could ignore that with only Unix and new Windows supported.

3 Likes

Yes, that's true, we would certainly have to continue to export it from std::os::unix as well.

Exporting it from std::os::windows would be unfortunate because calling code would have to use #[cfg(windows)] and #[cfg(unix)] to pull in the right export. And most won't: they'll just support Unix since that's easier.

Is there a precedent for portions of std that are exported for both Unix and Windows from a single place but are not exported at all on other targets?

For Windows releases before this was added, connecting or binding would just fail with WSAEAFNOSUPPORT, which is probably fine.

3 Likes

Should we expose Unix sockets from Windows?

I interpret the question asked in this thread as: "How can we add AF_UNIX support for more platforms in the stdlib now that those platforms support it?" To me it seems that supporting a feature on more tier-1 platforms would be desirable, even if we can't yet guarantee that we can provide for all platforms. I think adding Unix socket support for the Windows versions that support it would make Rust code more portable, which is progress overall.

Depending on how we implement this (see below) this could mean that existing code that relies on Unix sockets could suddenly work on Windows as well. Without needing to do anything other than upgrade to a new compiler version. Making existing Rust programs work on more platforms by effectively changing nothing seems really good!


How would we expose Unix sockets on windows?

I'd like to separate the questions of "should we" from the "how would we go about it". I feel pretty strong that it would be a good idea to expose Unix sockets on more platforms. But I'm less sure on what the right way would be to go about it.

What's the current situation?

Unix sockets in the stdlib are currently exported like this:

I see two difficulties with this:

  1. This is part of std::os::unix which is often thought of as a counterpart to std::os::windows.
  2. The names of Incoming and SocketAddr conflict with their std::net counterparts. They cannot live in the same submodule as-is.

Option 1

The first option would be to make std::os::unix::net available on Windows as well. This would change the meaning of what it means to live under std::os, but overall this seems like it could be the easiest change for end-users as all existing libraries that depend on Unix sockets would suddenly also work on Windows.

Communication wise there are two tricky parts to handle here:

  1. How do we communicate that parts of the Unix APIs are now also available on Windows?
  2. How do we communicate that not all Windows versions are supported?

I've heard libs teams members express how perhaps the std::os::* submodules may not have been the best design, and at least for these APIs I tend to agree. It seems that it didn't really account for the possibility of platforms gaining support for features from other platforms, which is kind of the root of the awkwardness here.

Option 2

We could introduce a new submodule for the Unix socket APIs. Perhaps something like std::ipc. This has the benefit of being new, so we can define which guarantees it provides. But it would also be easier to type out since e.g. std::os::unix::net::UnixDatagram doesn't particularly roll off the tongue.

This approach doesn't need to be mutually exclusive with the first option. We could a path where we introduce a new submodule, and then slowly phase out the std::os::unix::net family of APIs. Both could refer to the same types under the hood so we'd have the benefit that existing code will work on more platforms. But we take it as an opportunity to improve ergonomics, documentation, and visibility of the APIs.

Option 3

Instead of a new submodule we could add the types to std::net. However Incoming and SocketAddr already exist in std::net. So we'd have to rename them to UnixSocketAddr and UnixIncoming. But the upside of this would be that we could storify it as: "Unix network primitives have become available on more platforms, so they're now available from the main network submodule".

Just like option 2, this doesn't need to be mutually exclusive with the first option and we could provide a path for introducing the new APIs and then slowly deprecating the others.

Discussion

I'm interested in this because I'm currently writing async bindings to named pipes to do cross-platform IPC, and I'm not particularly loving it. Existing code is not very well maintained, and even when using named pipes it's hard to reap the full benefits since we need to account for Unix socket semantics as well.

It'd be nice if Unix sockets were available on more platforms that support them. That would provide a fully portable IPC layer directly from the stdlib. Because right now it seems that Unix sockets are often chosen because they fill that role and are part of the stdlib, but that introduces portability issues for Windows. From a pragmatic perspective I'd prefer we expand support to include some Windows versions instead of the status quo where we have portability problems.

Oh before I forget: WASM was mentioned earlier. It seems the approach the libs team is taking now is to expose all of stdlib in WASI but marking all of their internals as panic!("not implemented for WASI");. That way the API can gradually be filled out as more APIs become available. I feel what we're discussing in this thread feels in line with that.

Anyway, that's about all I had to share here. I hope what I'm saying here makes sense! If this needs to be driven by someone I can probably warrant spending some cycles here in the next few weeks.

4 Likes

I wonder if cfg(accessible) could potentially help here? If we expose UnixStream and similar in a common location across platforms, then people don't have to write a complex configuration of "unix or windows or ...", they can just write cfg(accessible(std::net::something::UnixStream)) to say "if I have UNIX sockets available".

2 Likes

@josh Oh I like that a lot — it would more or less allow cfg(accessible) to work as universal feature detection. That would be really nice.

One question tho: how would you see this interact with WASI's current approach where it provides empty impls for stdlib APIs?

1 Like

That would be a WASI-specific issue. If the stub implementations WASI provides are a problem for an application, that application might have to detect that it isn't on WASI. Alternatively, WASI, might start providing non-stub implementations. That's probably worth a separate conversation with WASI folks about the policy for providing stubs, but I don't think it needs to affect the general strategy of providing UnixStream in a platform-neutral location.

2 Likes

@josh thanks for answering; what you're saying makes sense to me!

1 Like

I created https://crates.io/crates/uds_windows for this.

Open Issue: https://github.com/rust-lang/rust/issues/56533

2 Likes