Enabling the formation of new Working Groups

Last year, the Rust Project decided to put a moratorium on creating new working groups. This was put in place for the Rust project to keep some breathing room before the Rust 2018 edition release. While there is still a lot to catch up on, we feel like it’s time to enable people to create official working groups again.

Previously, domain working groups (WGs) were set up at the discretion of the Rust core team and the process was not transparent. We have been thinking about a process to propose and set up new WGs and it isn’t finished. Still, we feel comfortable that we have a good sketch of one. For that reason, we would like to experiment. This means that if you have a lingering, uncompleted formation, you can get back in touch with us now! Please be aware, though, that due to the experimentation nature, we might have requests for changes down the line. Note that this is for fresh domain working groups, not for formation of subgroups of existing teams/WGs. Also, to avoid rush situations, we’d like to create one working group at a time and will give a slight preference to people that have already been in touch with us.

An application should check off this list:

  • create a charter around the following questions:
    • What value do you want to bring to the project?
    • Why should it be put into place?
    • What is the working group about?
    • What is the working group not about?
    • Is your WG long-running or temporary?
      • If it is temporary, when should it spin down?
    • What is your long-term vision?
  • How do you expect the relationship to the project be?
    • How do you want to establish accountability?
  • Which other WGs do you expect to have close contact with?
  • What are your short-term goals?
  • Who is the initial leadership?
    • This is preferably multiple persons
  • How do you intend to make your work accessible to outsiders?
  • Everything that is already decided upon
    • e.g. places to chat, write and mail
  • Where do you need help?
  • Preferred way of contact/discussion

While working groups may sometimes implement their own libraries and frameworks within the project, WGs formed about promoting a specific technology or a specific implementation of a subject may not be accepted on those grounds. We appreciate that this line is not clear-cut, so if you are unsure, please get in touch! It is recommendable to make a clear case why the working group must be inside the project to be effective and what its service to the wider ecosystem is.

The goal of this process is to unblock the current situation in a productive way that provides the governance team and following working groups with learning experiences. We intend to derive a standard process from those experiences.

After completing this process to become an official working group, you’ll be listed on https://www.rust-lang.org/governance so that other folks can find you. The Rust project will provide you with infrastructure, such as mailing list, chat, moderation and similar. Working group leaders will be expected to give periodic public updates on the group’s projects (we’re still working out the right format and cadence for these status updates, but it’ll be something like a 15 minute report in a recorded video meeting once every three months). You’ll also be expected to provide high-level goals for the yearly Rust roadmap. Your working group will get support from the Rust project in fulfilling your goal and will be the first group of experts to be approached when language changes need input. A working group is also a good place to collaboratively propose changes to the Rust ecosystem. You will also be able to attend the yearly all-hands.

We will then contact you and start the process of setting a WG up. Please note that this might take some time, to make sure that we have appropriate capacity for working with you! This is a process of productive discussion and exploration, so there might be changes coming out of that. If you start the process, please be in touch early so that we know about you and can help you as early as possible. This can be in private (write to core@rust-lang.org) or public (just post to https://internals.rust-lang.org), to your preference.

8 Likes

I and apparently other members of the Library team were not aware or did not remember that when we discussed the creation of the Allocators Working Group.

RFC 1398 was accepted in 2016 with a trait for memory allocators. It mentions but does not fully specify making Vec and other containers generic over the allocator type. Since then there has been discussion that is both lengthy (412 comments so far in https://github.com/rust-lang/rust/issues/32838 alone) and fragmented (with a number of threads on this forum), but not a lot of concrete progress. (Other than for GlobalAlloc and #[global_allocator], which specifically left out allocator-generic containers.)

I don’t remember who first suggested an Allocators Working Group (sorry!) but I felt that having a dedicated issue tracker where specific questions can be discussed separately and marked as resolved separately would be very helpful.

After some discussion in Gauging Interest for an Allocator Trait Working Group, we created the rust-lang/wg-allocators repository on GitHub and the t-libs/wg-allocators stream on Zulip.


Now, this thread mentions domain working groups. I’m not sure wg-allocators qualifies as one in the sense that the 2018 roadmap talks about. And this group already exists to some extent, so this isn’t really an application. Still, this list of questions raises good points so I’ll try to answer them for this group.

Some of this is my personal opinion.

  • What value do you want to bring to the project?

Give users of standard library containers more control over heap memory allocation, and enable more use case. Finish designing and implementing a long-standing accepted RFC.

This is an item the 2019 roadmap.

  • Why should it be put into place?

The discussion to resolve all the associated questions has proven too large for a single tracking issue.

  • What is the working group about?

An allocator trait and allocator-generic containers based on this trait.

  • What is the working group not about?

Defining scope more precisely remains to be done. Different people have expressed different desires so far, but it might be better to try to agree on some goals and non-goals.

  • Is your WG long-running or temporary?
    • If it is temporary, when should it spin down?
  • What is your long-term vision?

Temporary, though I have no idea if it won’t end up running for a long time.

The expected outcome is an RFC (or a few RFCs). The group should spin down, to be determined, either when the RFC(s) are accepted, or implemented, or stabilized.

(A possible alternative outcome is a “blessed” crate on crates.io that provides relevant functionality, and consensus that it doesn’t strictly need to be in the standard library. This is more likely IMO if https://github.com/rust-lang/wg-allocators/issues/1 doesn’t work out and we need to have separate container types regardless. The standard library could depend on that crate and expose wrappers, like it now does for https://crates.io/crates/hashbrown and HashMap.)

  • How do you expect the relationship to the project be?

I expect similar to that of an individual contributor for smaller features: submit RFC(s) and implementation PR(s) and follow the usual process for them.

  • How do you want to establish accountability?

Accountability for what?

  • Which other WGs do you expect to have close contact with?

Unclear. Possibly none, unless they have use cases for these features.

  • What are your short-term goals?

File separate issues in https://github.com/rust-lang/wg-allocators/issues for remaining design questions, and get discussion going. This has already started.

  • Who is the initial leadership?
    • This is preferably multiple persons

In Gauging Interest for an Allocator Trait Working Group, @ErichDonGubler volunteered to be a facilitator. This term is used after blog posts by Manish and by Niko.

However I feel that it’s not clear what the responsibilities are or should be. Some “How to start/run a WG” guidance would be appreciated here.

While I am interested in this topic, I lack bandwidth and am specifically not volunteering to be a leader or facilitator. (Though I am the de-facto contact with the Library team.)

  • How do you intend to make your work accessible to outsiders?

Anyone is welcome to participate to discussion on the repository or on Zulip. So far there is no synchronous meeting or formal membership. It’s undecided whether there should be, see https://github.com/rust-lang/wg-allocators/issues/20.

  • Everything that is already decided upon
    • e.g. places to chat, write and mail
  • Where do you need help?

Some guidance or good precedent in other WGs to look at about whether or how to have formal membership, how to build consensus and make decisions (within the WG before bringing them to the wider Rust project and community through an RFC), and more generally what it means to lead on facilitate a WG.

  • Preferred way of contact/discussion

Maybe the issue tracker? I’ve subscribed to be notified of everything in it, and suggested in the README that people who are interested do the same.

5 Likes

@simonsapin Thanks for writing this down! Even if I’m pretty sure that a wg-allocators could just be formed by libs or compiler team just wanting it, I find it great to hear that this is a useful exercise :). As it just happens, I had a discussion today on how I’d find a stable allocator interface important in the mid-term, so I’d be happy to see a group forming :).

I propose creating a gamedev working group. Idea and reasoning behind is also explained in more detail in this blog post I wrote a while back : https://alexene.github.io/2018/11/15/Rust-and-game-development.html

What value do you want to bring to the project?
I will just copy-paste this from my blog post :slight_smile:
The role of this working group is to find and tackle systemic problems that game developers face as they write their games in Rust.
Gathering these pain points sorting and distributing them them to the teams that handle different parts of the ecosystem is one role.
Communicating and teaching through tutorials, a status of what the problems encountered are and general info of what’s happening in this space.
This isn’t going to touch a single area. It impacts multiple parts of the ecosystem and we need to identify and collaborate in solving any pain points found.

Why should it be put into place?
Game development is a big industry that heavily relies on delivering high-performant software and rust is a perfect language candidate for that space.

What is the working group about?
This WG is aimed at solving problems encountered by game developers when using rust.
It will also serve as a voice from the rust team towards game devs, increasing the awareness of subjects where gamedevs should contribute to.

What is the working group not about?
This WG is not about advocating special features in the language specifically tailored at game or engine developers.
This WG is not about working on an official rust game engine, or the official rust entity component system crate. The purpose of this WG is to enable such things to happen, not establish standards or promote one library or engine over another.

Is your WG long-running or temporary?
Given the state of the rust ecosystem around game development tooling I think this wg is long-running.

If it is temporary, when should it spin down?

What is your long-term vision?
Making rust the default language choice in game development.
If that didn’t sound grandiose enough, I have other smaller goals:

  1. Being able to cargo build and get something that can run on a modern console.
  2. Having a AAA game engine done primarily in rust

How do you expect the relationship to the project be?
I see this working group more as a mediator between game developers and various other WGs.

How do you want to establish accountability?
We will adhere to the rust code of conduct and the leadership of this group is responsible to enforce it.

Which other WGs do you expect to have close contact with?
The tooling and embedded working groups are a good starting point.

What are your short-term goals?
Gather a list of problems and topics that game developers have encountered while developed their games with rust.

Who is the initial leadership?
I am happy to lead, organize and mediate the discussions on this and I would super welcome others to share this with.

How do you intend to make your work accessible to outsiders?
I would like us to follow the model that the embedded wg had. I think the rust-embedded github org is also something we should do, at least initially with a the blog.

Everything that is already decided upon
I would vote for a discord room in the rust channel like the other working groups have in order to be easy to find.

Where do you need help?
We can start from having a room in the discord server. I am sure other areas will pop up like mailing list and a lot of other details that I didn’t think about.

6 Likes

I’d love to see this happen (and am willing to contribute wherever possible).

This WG is not about working on an official rust game engine, or the official rust entity component system crate. The purpose of this WG is to enable such things to happen, not establish standards or promote one library or engine over another.

This is important. I’ve seen a lot of focus going to Amethyst and Specs (rightfully so, since the authors have built awesome libraries), but I think it’s important not to put all our eggs in one basket. There are many game developers who like to use their own engines, or don’t use an ECS, so I hope we can foster the creation of many utility crates (one such an example would be https://github.com/kyren/luster), best practices, and community outreach to grow the Rust gamedev community.

3 Likes

Hi,

I nice for pointing this out :). The question was more aimed at how you communicate plans and timeframes and how people can check up on them :).

Oh, I think for communicating timeframes and such things, it will be a combination of newsletter / posts and mailing list. We could just have a what’s in progress / next on a website, I didn’t really think about how these would be organized in detail, but the goal is to be as transparent about them as possible

Consider me signed up for any GameDev WG initiative. As previously stated I’m deeply invested in this effort.

And I do agree that Amethyst is just one small part of a much larger whole here.

1 Like

Actually, considering we already got people asking to be involved, can we split that up into a thread?

I have created this Rust-gamedev WG application discussion in order to not pollute the thread here with gamedev-related things :smiley:

1 Like

Rust Machine Learning Working Group

What value do you want to bring to the project?

The goal of this working group is to take inventory of the existing (and non-existing) machine learning (ML) libraries in Rust and improve or create any crates need to have a fully functional ML environment for Rustaceans.

The working group would start off by taking stock of the current Rust environment for ML, create examples and test their correctness and usability of the crates that currently exist.

After the initial pain point and requirements gathering, the working group would polish, improve, combine, and/or remove existing libraries to have a less confusing environment.

The next phase would be to create any new crates or significantly change any existing crates for use to create ML or supporting frameworks (e.g.numpy/pandas full equivalents).

Why should it be put into place?

Rust is a very good fit for ML developers, but might be very intensive to learn if they do not have a software engineering background to begin with. With the environment more mature Rust would have a significant speed advantage over Python and with the concurrency Rust offers would exceptional for parallel data processing for ML. As it currently stand there is not a lot of up-to-date tooling out there for ML in Rust. There exists crates, but a lot of them are fairly out-of-date and have not seen a commit in ~2 years.

What is the working group about?

The working group is about polishing and getting ML libraries first class in Rust. This would include getting existing ML and supporting crates to “1.0” and developing any gap-filling crates that would need to be created in order to be usable by all ML research/developers (especially those who do not currently use or know Rust). The working group should also set guidelines for any future development in the ML sphere to adhere to (API consistency).

What is the working group not about?

This working group is not about teaching machine learning basics or taking ML/AI to the extreme beyond what currently exists in research. The working group would also not be about promoting anything created inside the working group over anything created outside the working group. We want a cohesive environment for ML in Rust and want those crates to be useable by everyone.

Is your working group long-term or temporary?

This working group would definitely be long-term.

If it is temporary, when should it spin down?

N/A

What is your long-term vision?

The long-term vision for the Machine Learning Working Group would be to have a ergonomic environment for ML researchers/developers to use Rust for ML including new ML research that might not currently exist at the time. The APIs should be usable without the need for a deep knowledge of Rust. The environment should be very ergonomic to the point where it is easy to create new networks on any machine (GPU or GPU based). The group should take all considerations of a Pythonic-like environment as much as that make sense to do. It should be very easy to discover, understand, and evaluate any ML framework or supporting crates for researchers both experienced and new to Rust.

I would love to see tutorials created in pure RustML especially standard ones like the datasets in MNIST.

How do you expect the relationship to the project be?

I believe the MLWG would work closely with Rust project to create efficient algorithms in the lowest level possible with pure Rust. I also see this project working with the entire ML community in order to make sure we have the most up-to-date alogithms created at the time the working group is running. We should also interface with research to make sure we are easy to use for those who are deeply embedded into ML rather than software engineering so we are positive it is easy to use for all.

How do you want to establish accountability?

We will strictly adhere to the rust code-of-conduct and the leadership of this group is responsible to enforce it. The working group is open to everyone and we strongly believe in this.

In terms of the WG accountability I would utilize a Github Pages for the WG to keep goals, timeline, and actions in sync with what the community thinks are important to further ML in Rust. Also including any other medium which is deemed fit by the Rust community and/or academia.

Which other WGs do you expect to have close contact with?

I expect to work closely with any other group contributing to algorithm-heavy parts of Rust development or any working group looking to create strong APIs into GPUs. Also, the core team, and possibly tooling groups.

What are your short-term goals?

The short term goals will be to take a close look at the current ML environment and formulate where needs the most ground work to start creating and adjusting APIs. Including taking suggestions from Rust developers/other working groups and machine learning researchers who do not currently use Rust for ML (rather Python or also importantly C++).

Who is the initial leadership?

I am happy to lead (i.e. mediate, coordinate, note-take, organize, etc.) the working group (I have a foot in both access to machine learning experts, and Rust language experts so more than happy to take the working group lead role). Heavy lifters would be a combination of ML researchers outside Rust as well as Rust developers in the irc #rust-machine-learning room and open to anyone else to join.

How do you intend to make your work accessible to outsiders?

I would plan to use the arewelearningyet.com site, as well as a separate Github Pages for the working group news, schedule, actions, and notifications. Twitter is also a powerful ally in posting small snippets of news. Youtube could also be used if the need for any video announcements arise.

Everything that is already decided upon

IRC and Discord would/cloud be used, I would strongly prefer using only one chat medium and since Discord keeps chat history without significant development I would opt for that. However, are more than open to ideas/what the community thinks is best.

Where do you need help?

Gaining attention to this working group and gather members to make ML awesome in Rust! Would be great to have an ML room on the Rust-lang discord server (I think one existed at some point but disappeared).

Preferred way of contact/discussion

Discord (and most things): deg4uss3r

Or here on internals

7 Likes

This has been floating around in the back of my mind for like 5 years, so I guess I’ll give it a shot :wink:

Rust Cryptography Working Group

What value do you want to bring to the project?

For about as long as I’ve been using Rust there has been an open issue to “Figure out in-tree crypto story”.

Rust already leverages cryptography in many places: most notably in std and in Cargo. These uses are generally fairly ad hoc and serve to scratch a particular itch. While there are places where Rust could benefit from more extensive use of cryptography, and while the language has a vibrant ecosystem of cryptography crates which have formed organically, it remains unclear how std and tools like Cargo should consume them, what they should consume, what cryptographic functionality should go into std proper, etc. I would like to see a working group that can help bring clarity to such issues.

I think a Cryptography WG could serve as a resource for the Core Team, Tools Team, etc when they have questions about cryptography or would like direction on how things like std and Cargo should leverage cryptography, what algorithms should / should not be used, what cryptographic functionality should / should not go in std, etc.

Why should it be put into place?

Many other languages have a standard set of cryptographic interfaces which can be used to compose cryptographic primitives, or implement libraries which consume cryptographic primitives in an abstract way (some examples are Go’s crypto package and Java’s JCE).

I think an interesting deliverable for a Cryptography WG would be such a standard set of traits which can be leveraged for Rust cryptography, delivered as a crate or perhaps in cases where it makes sense included in std (most notably for cryptographic algorithms std is already leveraging). I look to something like the Embedded WG and embedded-hal as a success story for producing a foundational piece of Rust infrastructure, and I see a similarly shaped piece missing for the cryptography space.

What is the working group about?

I see the WG potentially acting in the following ways:

  • As a resource for the other teams/WGs when they have questions about the use of cryptography in Rust projects
  • Providing overall direction for cryptography in Rust, and producing core ecosystem cryptography crates
  • As an interest group and focal point for Rust cryptography in general. Rust has many properties which make it extremely desirable for implementations of cryptographic algorithms, and a bunch of organic adoption, but little direction in terms of building a cohesive cryptographic ecosystem.

Ideally, I would like to see such a WG get us to the place where we can close the “Figure out in-tree crypto story” issue.

What is the working group not about?

When it comes to things like std and Cargo, my personal inclination is for the WG to consult, inform, and assist on the usage of cryptography in these applications, but ideally with a light touch, leaving key decisions to the teams who own the respective code/tools.

I think the goal of a Cryptography WG should be to foster collaboration with the other teams, and assist them in being successful with cryptography-related parts of their project. I don’t think the Cryptography WG should be the “owner” of std crypto or Cargo crypto, just consultants trying to help make it better.

Is your WG long-running or temporary?

I think there could be an immediate deliverable of some core ecosystem cryptography crates, however cryptography isn’t going away and I see the relationship between Rust and cryptography as one that will only go stronger with time.

What is your long-term vision?

An interoperable cryptographic ecosystem within Rust where every crate doesn’t have to build their own internal adapters to support many independent implementations of the same algorithm.

An example might be Cargo crate signing: let’s say hypothetically the Tools Team shipped such a feature. There are many key storage methods available for signature keys: a software-backed key kept in a file, hardware tokens/HSMs, secure enclaves (e.g. MacBook SEP). It would be nice if users of Cargo had their choice of key storage mechanism, without the Cargo team having to go out of their way to support all of them, and making it easy to add new key storage methods.

Making this easy involves creating a standard multi-provider signing interface which Cargo can consume, and that providers for individual key storage mechanisms can target.

Beyond that, fostering and nurturing the ecosystem of cryptography in Rust, as Rust seems like a choice language for these use cases and has seen a lot of organic adoption.

How do you expect the relationship to the project be?

I already talked about this quite a bit earlier but in short: as a resource for the other teams/WGs to consult with when they have questions about cryptography, and hopefully providing direction for the overall ecosystem.

How do you want to establish accountability?

  • The Cryptography WG should be accountable to the growing ecosystem of developers of Rust cryptography crates (implementations of cryptographic algorithms/protocols) to deliver solutions which are useful, secure, and easy-to-use.
  • It should be accountable to the Core Team and other teams/WGs to improve the use of cryptography in their respective projects.
  • Finally, it should be accountable to the Rust ecosystem as a whole (and vicarious users thereof) to lead Rust cryptography in good directions.

Which other WGs do you expect to have close contact with?

The Core Team, Tools Team, and Secure Code WG.

What are your short-term goals?

Immediate short-term goals could include creating resources on modern best practices for using Rust cryptography, including lists of presently-maintained crates, what algorithms they support, state of maturity, etc.

Who is the initial leadership?

I’m interested in leading this but could certainly use help! I know there has been a lot of interest in this topic in the past (some of the polls around the time the Secure Code WG team was formed showed there was perhaps more interest in a Cryptography WG).

How do you intend to make your work accessible to outsiders?

I think we can more or less follow the same approaches we’ve been using in the Secure Code WG (many of which we adopted from other WGs including embedded):

  • GitHub repo / issues as primary coordination point
  • Blog posts
  • Twitter
  • Some sort of chat (Zulip, Discord, or whatever the IRC replacement is when that’s decided)

Everything that is already decided upon

This is my initial pitch for such a WG and I’m largely spitballing here based on my own opinions, so nothing is decided!

Where do you need help?

If other people are interested in getting the ball rolling on this, let’s coordinate to figure out some discussion channels to determine next steps.

I imagine if there’s enough interest, we can start by splitting out discussion into a separate thread here on internals.rust-lang.org, and then maybe discuss alternative communication channels.

18 Likes

Hi all and thanks for your reply :). I’ll add you all to a tracking ticket later today.

3 Likes

The tracking issues here:

The update here is that we will give the gamedev WG preference and start there first. The reasoning here is the long track record of the group to make approaches and the great work at arguing why it’s necessary. As the main reason stated for not taking one on was bandwidth issues, we see no reason to not do it this way. I’ll be in touch over the week for a first meeting.