Announcing the Network Services Working Group (WG-Net)

[withoutboats]

Hey folks,

Do you have an interest in writing network services in Rust? Do you want to see the ecosystem grow and become easier to jump into? Fashionably late, we are kicking off the network services working group.

Our goals of this group are to make writing network services in Rust easier. There are two main thrusts of that effort:

• Make async programming more ergonomic in Rust by improving the futures library and providing language support for async/await syntax.
• Provide a shared foundation for web frameworks in Rust by standardizing around an interface for middleware (like Tower) & encouraging frameworks to use it, as well as writing examples of that middleware.
• Provide extensive documentation about network programming in Rust.

Note that these are preliminary goals and we’d very much like to hear from you about what you’d like to see in the Rust networking space this year.

We have a repo, with discussion issues.

We have a gitter, for chatting in.

Please get involved!

(Also I’m going on vacation next week, so my lovely cohosts @aturon and @cramertj will probably be fielding your initial questions. )

[Nemo157]

a shared foundation for web framework in Rust by providing a standard interface for middleware & encouraging frameworks to use it

Do you have some sort of example of what you mean by this? I don’t see how there could be a standardised interface for web middleware, depending on what protocol is being used there are many different potential interfaces. If you mean an “http framework” I’m all for that. (Or does “web” mean “http” only now and other hyperlinkable network protocols are relegated to being less important).

[withoutboats]

I do mean HTTP, but the middleware definitions we’re looking at can support other protocols if their topology isn’t too different (the traits have associated types for “requests” and “responses”).

[aturon]

I think part of the issue is that “middleware” is pretty overloaded. But the hope is that there are lots of useful foundations that can be shared across protocols.

Have you seen Tower? That’s the best example of the kind of middleware space we’re talking about.

On a personal note: I really appreciate all your help on the futures repo recently, and hope you’ll join in on the working group!

[withoutboats]

On a personal note: I really appreciate all your help on the futures repo recently, and hope you’ll join in on the working group!

I want to second this @Nemo157! Thanks for your contributions, they’ve been very impactful.

[vitiral]

You guys might be (slightly) interested in my new jsonrpc types crate: https://docs.rs/jrpc

[andrewtj]

I’m interested in working on DNS and name-resolution related infrastructure. Are there any plans in that general area?

[Nemo157]

Have you seen Tower ? That’s the best example of the kind of middleware space we’re talking about.

Just took a look at it, and it does seem like it might be possible to have middleware generic over any request/response protocol. The big one I wonder about is something like CoAP, it is mostly a request/response protocol very similar to HTTP, but it also has some pub/sub features. Hopefully it would be possible to use these generic middleware on the request/response paths, while still allowing the pub/sub parts to function independently.

On a personal note: I really appreciate all your help on the futures repo recently, and hope you’ll join in on the working group!

Thanks , I’ll definitely try to help out where I can.

[carllerche]

Protocols that are mostly request / response based but also include a streaming component should work fine. PUB SUB can be modeled as a request for a subscription and the stream of messages is the response.

[Nemo157]

PUB SUB can be modeled as a request for a subscription and the stream of messages is the response.

That would probably work well in a traditional server, but I’m mostly thinking about constrained devices. Commonly they will have that stream of responses spread out over months of real time, thousands of sleep/wake cycles and dozens of full power off reboots. More than likely having to keep the subscription active over multiple firmware upgrades as well.

I do think using generic middleware for the request response parts of CoAP and dealing with the pub/sub parts separately will work fine (especially for devices like I mention above). It’s basically a more standardised variation of HTTP webhooks, so similar to those needs to be tied in pretty deeply to the application layer anyway.

[kainino]

Just want to pop in saying I’d love to see good client side networking for Rust programs running under WebAssembly in a browser. For both games (see also HumbleNet) and web apps.

[ehiggs]

I understand that there is a focus on http and I suppose http2. But I think an early focus should also be a great story around grpc and thrift protocol generation and bindings. pingcap have a decent wrapper around grpc’s c bindings, but installing this requires a go compiler which is kinda weird.

Then there’s a case of managing various protocols to talk well with existing network middleware. rabbitmq, kafka, redis, amazon sqs, etc as well as storage backends (db, file, object, log, telemetry) (postgres, mysql, hdfs, s3, wasb, gs, prometheus). And data/file formats (protobuf, thrift, avro, parquet, orc, rocksdb, sled sled sled sled sled).

A lot of this exists in various levels of quality and I see the WG-Net as a great initiative to make sure the quality of these libraries is top notch.

Further, have we explicit input from e.g. Buoyant who have put their money where their mouth is and roughly reimplemented linkerd (scala) as conduit? Their experience in this area will be invaluable.

[Kane]

I would also second that focus should not be only on http protocols, grpc, thrift and alikes should be considered as first class citizens as well.

[withoutboats]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.